[arch-dev-public] archbuild: building in a chroot for the lazy
Hi all, atm every dev has his own scripts or he just doesn't build his packages in a chroot. Even though mkarchroot and makechrootpkg exist they are not straight forward to use and you need to configure your chroot properly. My long term goal is to create easy to use tools which will use the exact same build environment for everyone. The first attempt is now in testing with devtool-0.9.9. In short: All you need to do now is run e.g. extra-i686-build, testing-x86_64-build, staging-i686-build etc.. This will do all the boring work for you like creating the correct chroot etc.. In more detail: * By default the script will create the chroots in /var/tmp/archbuild. Make sure you have enough disk space here. You might use another dir by using the -r switch. * The -c switch will recreate the chroot even if it already exists. * You can also build i686 pacakges on x86_64. * TUs can use extra-*-build and testing-*-build to build packages for community or community-testing. * You user will need the permission to use sudo * Set PACKAGER, MAKEFLAGS, SRCDEST, PKGDEST in your ~/.makepkg.conf * the first enabled mirror in your mirrorlist will be used for the chroots; local mirrors using file:// are supported. This is in early stage and far from ready. So feedback and feature request are more than welcome. Greetings, Pierre -- Pierre Schmitz, https://users.archlinux.de/~pierre
On 20 August 2010 19:09, Pierre Schmitz <pierre@archlinux.de> wrote:
Hi all,
atm every dev has his own scripts or he just doesn't build his packages in a chroot. Even though mkarchroot and makechrootpkg exist they are not straight forward to use and you need to configure your chroot properly.
My long term goal is to create easy to use tools which will use the exact same build environment for everyone. The first attempt is now in testing with devtool-0.9.9.
In short: All you need to do now is run e.g. extra-i686-build, testing-x86_64-build, staging-i686-build etc.. This will do all the boring work for you like creating the correct chroot etc..
In more detail: * By default the script will create the chroots in /var/tmp/archbuild. Make sure you have enough disk space here. You might use another dir by using the -r switch. * The -c switch will recreate the chroot even if it already exists. * You can also build i686 pacakges on x86_64. * TUs can use extra-*-build and testing-*-build to build packages for community or community-testing. * You user will need the permission to use sudo * Set PACKAGER, MAKEFLAGS, SRCDEST, PKGDEST in your ~/.makepkg.conf * the first enabled mirror in your mirrorlist will be used for the chroots; local mirrors using file:// are supported.
This is in early stage and far from ready. So feedback and feature request are more than welcome.
Very nice, Pierre! But yes, you're right, we all have our own scripts to handle these in one form or another. But thanks to this, now I can delete all the extra dirs in /chroot and my stupid cheatcodes in ~/.bin like chrootpkg-testing :) -- GPG/PGP ID: B42DDCAD
Pierre Schmitz <pierre@archlinux.de> wrote:
* By default the script will create the chroots in /var/tmp/archbuild. Make sure you have enough disk space here. You might use another dir by using the -r switch. * The -c switch will recreate the chroot even if it already exists. * You can also build i686 pacakges on x86_64. * TUs can use extra-*-build and testing-*-build to build packages for community or community-testing. * You user will need the permission to use sudo * Set PACKAGER, MAKEFLAGS, SRCDEST, PKGDEST in your ~/.makepkg.conf * the first enabled mirror in your mirrorlist will be used for the chroots; local mirrors using file:// are supported.
This is in early stage and far from ready. So feedback and feature request are more than welcome.
I get random catastrophes when starting the script while still having some shell open inside: I think it comes from binds not properly dismounted before calling the 'rm -rf' line. Would it be wise to add the --one-file-system option there ? -- Rémy
Rémy Oudompheng <remyoudompheng@gmail.com> :
I get random catastrophes when starting the script while still having some shell open inside: I think it comes from binds not properly dismounted before calling the 'rm -rf' line. Would it be wise to add the --one-file-system option there ?
Gah, even with this change, I still get my packages cache deleted and /dev gets broken (funny things like "/dev/null: permission denied). I can't find the culprit line, makechrootpkg seems to use rsync -x, rebooting...
On Sun, 29 Aug 2010 19:52:10 +0200, Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
Rémy Oudompheng <remyoudompheng@gmail.com> :
I get random catastrophes when starting the script while still having some shell open inside: I think it comes from binds not properly dismounted before calling the 'rm -rf' line. Would it be wise to add the --one-file-system option there ?
Gah, even with this change, I still get my packages cache deleted and /dev gets broken (funny things like "/dev/null: permission denied). I can't find the culprit line, makechrootpkg seems to use rsync -x, rebooting...
I don't really know what you are doing here. ;-) Some more detailed steps how to reproduce it would be helpful. For future versions I could try to add some kind of locking and more robust checks if there are mount point within the target dir. -- Pierre Schmitz, https://users.archlinux.de/~pierre
Pierre Schmitz <pierre@archlinux.de>:
On Sun, 29 Aug 2010 19:52:10 +0200, Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
Rémy Oudompheng <remyoudompheng@gmail.com> :
I get random catastrophes when starting the script while still having some shell open inside: I think it comes from binds not properly dismounted before calling the 'rm -rf' line. Would it be wise to add the --one-file-system option there ?
Gah, even with this change, I still get my packages cache deleted and /dev gets broken (funny things like "/dev/null: permission denied). I can't find the culprit line, makechrootpkg seems to use rsync -x, rebooting...
I don't really know what you are doing here. ;-) Some more detailed steps how to reproduce it would be helpful.
It happens with archbuild (which I used as staging-i686-build), and I didn't tried with makechrootpkg (I don't really want to try anymore). I think I get deterministic behaviour with the following steps: * run archbuild for some package * start a shell in the chroot during the build process * wait for archbuild to finish or interrupt it with Ctrl-C * run archbuild again with the same chroot. -- Rémy.
For the moment, my only proposal is a naïve implementation of a lock system. The corresponding patch (for devtools) is attached. I still don't understand my problems, since they don't seem to come from the "rm -rf" step nor from the "rsync -a --delete" step. -- Rémy
On Thu, 2 Sep 2010 12:19:45 +0200, Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
For the moment, my only proposal is a naïve implementation of a lock system. The corresponding patch (for devtools) is attached. I still don't understand my problems, since they don't seem to come from the "rm -rf" step nor from the "rsync -a --delete" step.
I'll think about a solution for this in future. makechrootpkg may alter the chroot before mkchroot is called which locks it; so it's not that stragiht forward to solve. Until I come up with a clean solution I would suggest not to use the same chroot more than once at the same time. -- Pierre Schmitz, https://users.archlinux.de/~pierre
Hi all,
atm every dev has his own scripts or he just doesn't build his packages in a chroot. Even though mkarchroot and makechrootpkg exist they are not straight forward to use and you need to configure your chroot properly.
My long term goal is to create easy to use tools which will use the exact same build environment for everyone. The first attempt is now in testing with devtool-0.9.9.
In short: All you need to do now is run e.g. extra-i686-build, testing-x86_64-build, staging-i686-build etc.. This will do all the boring work for you like creating the correct chroot etc..
In more detail: * By default the script will create the chroots in /var/tmp/archbuild. Make sure you have enough disk space here. You might use another dir by using the -r switch. * The -c switch will recreate the chroot even if it already exists. * You can also build i686 pacakges on x86_64. * TUs can use extra-*-build and testing-*-build to build packages for community or community-testing. * You user will need the permission to use sudo * Set PACKAGER, MAKEFLAGS, SRCDEST, PKGDEST in your ~/.makepkg.conf * the first enabled mirror in your mirrorlist will be used for the chroots; local mirrors using file:// are supported.
This is in early stage and far from ready. So feedback and feature request are more than welcome. Very nice. Any chance to get an option to avoid to clean the chroot everytime? You know, I use the kde-build scripts and I need to install some package in
On Friday 20 August 2010 13:09:48 Pierre Schmitz wrote: the chroot to build the others. Also, an option to install packages in the chroot is missing. Obviously, I won't to force anybody with my "personal" feature request. I can maintain the previous behavior, but I'd like to keep the same build environment for everyone and maybe someone else needs that options. Cheers -- andreascarpino.it Arch Linux Developer
participants (4)
-
Andrea Scarpino
-
Pierre Schmitz
-
Ray Rashif
-
Rémy Oudompheng