Building in clean chroot dilemma
Hello, So I currently do not test my packages in a clean chroot because of this issue: https://gitlab.archlinux.org/archlinux/devtools/-/issues/105 It seems a little stupid but I hate sudo being installed on my system, I do not like it, and I do not like when tools force me to use sudo. Without sudo installed the makechrootpkg script breaks and thus I can't use dev tools to test in a chroot environment. I did try and just manually build a chroot environment, but for whatever reason it was not writable to disk, in other words I kept getting "filesystem full" errors when the filesystem was not full, and I couldn't see to find a decent guide on how to actually setup a chroot manually, all recommend using different parts of the devtools. I have posted this for 2 reasons: 1. I want to know if anyone else agrees that doas support should be added to the devtool scripts. 2. I am looking for temporary alternatives from using devtools to build AUR packages in a clean environment. I have in the case of the movim package built it in an lxc container, which are similar to chroots, but the issue is the base arch linux container image contains a few dependencies which could mean I miss out some of the dependencies. And also every time I want to deploy a package to the container I have to copy it over to my production server which is not ideal. If anyone has any solutions please let me know :) Thank you, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Hi, On 24/01/2023 09:54, Polarian wrote:
Hello,
So I currently do not test my packages in a clean chroot because of this issue: https://gitlab.archlinux.org/archlinux/devtools/-/issues/105
It seems a little stupid but I hate sudo being installed on my system, I do not like it, and I do not like when tools force me to use sudo. Without sudo installed the makechrootpkg script breaks and thus I can't use dev tools to test in a chroot environment.
You can run extra-x86_64-build as root, this should work fine.
I did try and just manually build a chroot environment, but for whatever reason it was not writable to disk, in other words I kept getting "filesystem full" errors when the filesystem was not full, and I couldn't see to find a decent guide on how to actually setup a chroot manually, all recommend using different parts of the devtools.
I recommend using extra-x86_64-build as that's a convenience wrapper, it stores build chroots in /var/lib/archbuild.
Hello,
You can run extra-x86_64-build as root, this should work fine.
No this does not work: eureka-notes on master took 9s ❯ doas su [root@PolarianLaptop eureka-notes]# ls PKGBUILD README.md [root@PolarianLaptop eureka-notes]# extra-x86_64-build :: Synchronizing package databases... core downloading... extra downloading... community downloading... :: Starting full system upgrade... there is nothing to do ==> Building in chroot for [extra] (x86_64)... ==> Synchronizing chroot copy [/var/lib/archbuild/extra-x86_64/root] -> [copy]...done ==> ERROR: Running makepkg as root is not allowed. Using doas: eureka-notes on master ❯ doas extra-x86_64-build :: Synchronizing package databases... core downloading... extra downloading... community downloading... :: Starting full system upgrade... there is nothing to do ==> Building in chroot for [extra] (x86_64)... ==> Synchronizing chroot copy [/var/lib/archbuild/extra-x86_64/root] -> [copy]...done ==> ERROR: Running makepkg as root is not allowed. With su: eureka-notes on master took 9s ❯ extra-x86_64-build Password: :: Synchronizing package databases... core downloading... extra downloading... community downloading... :: Starting full system upgrade... there is nothing to do ==> Building in chroot for [extra] (x86_64)... ==> Synchronizing chroot copy [/var/lib/archbuild/extra-x86_64/root] -> [polarian]...done /usr/bin/makechrootpkg: line 561: sudo: command not found ==> ERROR: Could not download sources. Like I have highlighted above, sudo is hard depended on, not having it causes it to break! Thanks, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Hi, On 24/01/2023 13:30, Polarian wrote:
Hello,
You can run extra-x86_64-build as root, this should work fine.
No this does not work: /usr/bin/makechrootpkg: line 561: sudo: command not found ==> ERROR: Could not download sources.
Like I have highlighted above, sudo is hard depended on, not having it causes it to break!
My bad, I thought it was only used inside the chroot and sudo was just a there to evaluate privileges. But no, it's used to download the sources outside of the chroot with --preserve-env which doas does not support. So supporting doas is impossible for devtools as doas has no equivalent option for this. So no dilemma really, just install sudo.
Hello,
My bad, I thought it was only used inside the chroot and sudo was just a there to evaluate privileges. But no, it's used to download the sources outside of the chroot with --preserve-env which doas does not support. So supporting doas is impossible for devtools as doas has no equivalent option for this.
So no dilemma really, just install sudo.
So I am going to make this intentionally harder, screw the devtools, what do I need in order to create an arch chroot manually and enter the chroot using the chroot binary, and then manually cloning the package and building it within the chroot? I can extract the built package from outside the chroot if I like as well, but this is mainly for debugging purposes not for actual building. Thanks for the help, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Am 24.01.23 um 13:55 schrieb Polarian:
what do I need in order to create an arch chroot manually and enter the chroot using the chroot binary, and then manually cloning the package and building it within the chroot?
- create a directory, subvolume, or filesystem - use pacstrap to install base and base-devel - archlinux-chroot into the new system - create a user, and switch to that user - build and test the package If you do this more than once, snapshots come in handy. You might want to share /var/cache/pacman and a directory with your package source files between snapshots, using mount --bind. BR
Wait, I just realised the installation of arch linux is literally this process... I am an idiot!!! Thanks for the help though, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Hello, I ran into a small issue, there is "no available disk space" if this is not done on its own partition, aka if you do not mount a partition to the chroot. Is there any method without having to partition up my disk to allocate storage to a chroot, as currently it has no root storage... Thanks, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Am 25.01.23 um 10:15 schrieb Polarian:
I ran into a small issue, there is "no available disk space" if this is not done on its own partition, aka if you do not mount a partition to the chroot.
Is there any method without having to partition up my disk to allocate storage to a chroot, as currently it has no root storage...
All you need is a directory to store the files of your build environment. Of course the filesystem this directory lives in needs enough free space to hold a minimal Arch installation, which is about 3 GBytes. If the system, you want to try this on, is short of storage, and / is a filesystem that supports reflinks (btrfs and newest xfs), you could snapshot or reflink all files of your running system into the new build-root (skipping pacstrap) # btrfs subvolume snapshot / build-root or # cd / # mkdir build-root # cp --reflink=always -ax /{bin,lib,var,etc,lib64,root,sbin,usr} build-root/ # mkdir -p build-root/{boot,dev,home,opt,proc,run,srv,sys,tmp} then, after chroot-ing into it, use pacman to uninstall everything that is not a dependency of base or base-devel. I did something similar to clean up a machine: # arch-chroot build-root /bin/bash # cd /tmp # pactree -u base | sed 's/[<>=].*$//' | sort -u > base.deps # pacman -Qg base-devel | cut -d' ' -f2 | sort -u > base-devel.deps # sort -um base.deps base-devel.deps > clean-pkgs # pacman -Qe | cut -d' ' -f1 | sort -u > current-pkgs After that, # diff --suppress-common-lines clean-pkgs current-pkgs | sed -n 's/^> //p' | pacman -Rcsn - would uninstall everything that does not belong to base or base-devel. Base-devel.deps is a bit misleading, it does not contain all dependencies, but as only explicit packages are uninstalled later, this seems to work alright. I am not implying that this would be my preferred course of action, but it is fast and space efficient. Next time you write about a problem you ran into, better show everything you did and the error message you got, using copy and paste. BR
On Wednesday, 25 January 2023 at 09:15 (+0000), Polarian wrote:
I ran into a small issue, there is "no available disk space" if this is not done on its own partition, aka if you do not mount a partition to the chroot.
Is there any method without having to partition up my disk to allocate storage to a chroot, as currently it has no root storage...
It may be the issue described in the arch-chroot man page. man 8 arch-chroot:
Note
The target chroot-dir should be a mountpoint. This ensures that tools such as pacman(8) or findmnt(8) have an accurate hierarchy of the mounted filesystems within the chroot. If your chroot target is not a mountpoint, you can bind mount the directory on itself to make it a mountpoint, i.e.:
mount --bind /your/chroot /your/chroot
I had trouble in the past with pacman estimating available disk space in chroots and it was resolved by following this advice. Jaron
Hello,
It may be the issue described in the arch-chroot man page.
Yes it seems like it was!
I had trouble in the past with pacman estimating available disk space in chroots and it was resolved by following this advice.
Yes it worked, thank you so much man, I know it is not conventional to build in manual chroots, and you are meant to use dev-tools but I far prefer this method, so that you so much :) -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Am 24.01.23 um 13:55 schrieb Polarian:
what do I need in order to create an arch chroot manually and enter the chroot using the chroot binary, and then manually cloning the package and building it within the chroot?
- create a directory, subvolume, or filesystem - use pacstrap to install base and base-devel - archlinux-chroot into the new system - create a user, and switch to that user - build and test the package If you do this more than once, snapshots come in handy. You might want to share /var/cache/pacman and a directory with your package source files between snapshots, using mount --bind. BR
Hello, Am Dienstag, 24. Januar 2023 09:54:37 CET schrieb Polarian:
Hello,
So I currently do not test my packages in a clean chroot because of this issue: https://gitlab.archlinux.org/archlinux/devtools/-/issues/105
It seems a little stupid but I hate sudo being installed on my system, I do not like it, and I do not like when tools force me to use sudo. Without sudo installed the makechrootpkg script breaks and thus I can't use dev tools to test in a chroot environment.
you already got one solution, but for the sake of completeness I wanted to add some other solutions, that maybe are less error-prone. 1. Limit the attack surface of sudo by disabling sudoedit and and allowing only the commands necessary for makechrootpkg. Probably sounds bad to you, but in case you ever need to debug your solution it's good to have the reference solution running. 2. https://github.com/AladW/aurutils supports building in a chroot with doas. 3. https://github.com/bionade24/abs_cd is a CD webservice that builds packages in (rootless) docker/podman containers. It does use sudo internally, but only inside the containers. Development has stalled a bit but it's still used by myself & others. Regards, Oskar Roesler
Just a little history lesson and a few words on:
So I currently do not test my packages in a clean chroot because of this issue: https://gitlab.archlinux.org/archlinux/devtools/-/issues/105
It seems a little stupid but I hate sudo being installed on my system, I do not like it, and I do not like when tools force me to use sudo. Without sudo installed the makechrootpkg script breaks and thus I can't use dev tools to test in a chroot environment.
There's a very valid reason for doas' existence. sudo was becoming more and more bloated and that's not acceptable for such a security-centric and critical program. For quite a while sudo would continue to be updated to newer versions with more features, but the sudo that came with OpenBSD was intentionally held back because they were thought to be too risky or introduced more dependencies. I get why you'd want to use doas and avoid having sudo installed. But the doas, or "opendoas," that Arch and some other Linuxes use has not been reviewed at all by OpenBSD people. It's an unofficial, third party port, in contrast to projects like OpenSSH, LibreSSL, OpenNTPD, etc that all have official portable versions for other operating systems handled by people directly involved with the code. The doas Arch uses is from here: https://github.com/Duncaen/OpenDoas But there was also a previous port located here: https://github.com/slicer69/doas In the exciting early days of doas, before the Duncaen version existed, the slicer69 version was what FreeBSD and some Linuxes distributed in their package repos. You'll notice now that pretty much everyone has switched over to the same one Arch is packaging... but why? The original doas port was done very carelessly, without understanding of the differences between OpenBSD and Linux. Specifically OpenBSD has a lot of built-in protections that Linux does not have. This resulted in the first doas port having an awful security bug where anyone could become root without a valid password: https://github.com/slicer69/doas/issues/2 I'm told the version in use by Arch and others is a bit more careful with the porting process, but the point of this long email is to say that I still wouldn't consider "opendoas" to be leaps and bounds above sudo in terms of security. In your case, I think the benefit of having a working devtools setup outweighs the benefits of clinging on to this third party doas port, but that's just my opinion.
Hello, Thanks for the history lesson, it was genuinely kind of interesting, I guess opendoas is not as secure on Linux as everyone makes it out to be, it is still faster and lighter weight which does mean theoretically it should be eaiser to maintain and thus more secure, and also it is less bloated. I guess using sudo for the dev scripts is a good thing, as for once the bloated features have come in useful within the dev scripts. I still would rather avoid setting up sudo just to use the dev scripts. I will try manual chroot building through pacstrap and chrooting as its always more fun to do things manually in my opinion and I will see how that goes. If it goes badly I will take your suggestion, and install and configure sudo for building the packages using the dev scripts, but I at least want a crack at doing it manually. Thanks for the suggestion, and the history :) -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Hey, On 24/01/2023 23:47, Polarian wrote:
Hello,
Thanks for the history lesson, it was genuinely kind of interesting, I guess opendoas is not as secure on Linux as everyone makes it out to be, it is still faster and lighter weight which does mean theoretically it should be eaiser to maintain and thus more secure, and also it is less bloated.
I guess using sudo for the dev scripts is a good thing, as for once the bloated features have come in useful within the dev scripts.
The only thing stopping doas from being supported is --preserve-env support, hardly a bloated feature. I still
would rather avoid setting up sudo just to use the dev scripts. I will try manual chroot building through pacstrap and chrooting as its always more fun to do things manually in my opinion and I will see how that goes.
Try to get --preverse-env supported in doas, then we can look into supporting it in devtools.
Hello,
The only thing stopping doas from being supported is --preserve-env support, hardly a bloated feature.
I guess this is a requested feature... if I have time I will submit it to the Linux opendoas repository Thanks, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
participants (7)
-
Jaron Kent-Dobias
-
Jelle van der Waa
-
Markus Schaaf
-
Markus Schaaf
-
Oskar Roesler
-
Polarian
-
T.J. Townsend