[arch-dev-public] makechrootpkg without aufs2
Hi devs, I always had problems with aufs2 but during the recent rebuilds and KDE development it really annoyed me. It seems the aufs2 kernel module triggers all kinds of (not) so funny errors; especially on high I/O. E.g. building kdebase-workspace with makechrootpkg is a good idea to check i your filesystem or raid correctly recovers from a hard crash. Aside from FS and RAID errors I had funny segfaults of umount on reboot, massive system slowdowns etc.. In general I like the approach of makechrootpkg. Starting from a clean base chroot environment is a good idea to avoid all kinds of packaging issues. So I checked what other options there are. I thought it would be less problematic if it could be handled in userspace. There are solutions like unionfs-fuse; but unfortunately they are quite slow and cannot handle all kinds of special fiels we need (fiofos etc.) In the end I found a much simpler and cleaner way to handle this. All we want is to build our package in a clean chroot and revert all changes when building the next package. So why not just make just a copy of our base system, work on that and revert it when we are done. This can be easily done by changing this in makechrootpkg: mount -t aufs none -o "dirs=$chrootdir/$LAYER=rw:$chrootdir/root=ro" "$uniondir" to rsync -a --delete --progress -h -c -W "$chrootdir/root/" "$uniondir" On my system this sync takes less than 40 seconds in worst case (if uniondir is empty). On the other hand building the package is faster because there is no aufs overhead. And the best thing is that this cannot crash and freeze your system. :-) Pierre PS: We should probably get rid of aufs and merge this change into our devtools. -- Pierre Schmitz, https://users.archlinux.de/~pierre
On 24/01/10 09:08, Pierre Schmitz wrote:
Hi devs, <snip>
This can be easily done by changing this in makechrootpkg:
mount -t aufs none -o "dirs=$chrootdir/$LAYER=rw:$chrootdir/root=ro" "$uniondir"
to
rsync -a --delete --progress -h -c -W "$chrootdir/root/" "$uniondir"
<snip>
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified. Allan
On Sun, 24 Jan 2010 11:29:41 +1000, Allan McRae <allan@archlinux.org> wrote:
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified.
Allan
I know; I just wanted to reduce it to one line to make the idea clear. The actual implementation of this would be a little more complex. We really only need to run rsync if -c is specified and we wont need the handling of the rw layer, the aufs module, mounts etc.. -- Pierre Schmitz, https://users.archlinux.de/~pierre
On 24/01/10 11:42, Pierre Schmitz wrote:
On Sun, 24 Jan 2010 11:29:41 +1000, Allan McRae<allan@archlinux.org> wrote:
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified.
Allan
I know; I just wanted to reduce it to one line to make the idea clear. The actual implementation of this would be a little more complex. We really only need to run rsync if -c is specified and we wont need the handling of the rw layer, the aufs module, mounts etc..
Great. I was just flagging it as a potential issue. I really like this idea as aufs2 craps out on my a lot causing unmount issues during shutdown. Also, when a build fails, it will be much easier to "cd $uniondir" and manually debug it. Allan
On Sat, Jan 23, 2010 at 7:29 PM, Allan McRae <allan@archlinux.org> wrote:
On 24/01/10 09:08, Pierre Schmitz wrote:
Hi devs, <snip>
This can be easily done by changing this in makechrootpkg:
mount -t aufs none -o "dirs=$chrootdir/$LAYER=rw:$chrootdir/root=ro" "$uniondir"
to
rsync -a --delete --progress -h -c -W "$chrootdir/root/" "$uniondir"
<snip>
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified.
I'm fine with this approach. aufs was used originally (well, unionfs) because it was a cheap way to say "don't break this chroot". Copying is fine. BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
On Sat, Jan 23, 2010 at 9:59 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On Sat, Jan 23, 2010 at 7:29 PM, Allan McRae <allan@archlinux.org> wrote:
On 24/01/10 09:08, Pierre Schmitz wrote:
Hi devs, <snip>
This can be easily done by changing this in makechrootpkg:
mount -t aufs none -o "dirs=$chrootdir/$LAYER=rw:$chrootdir/root=ro" "$uniondir"
to
rsync -a --delete --progress -h -c -W "$chrootdir/root/" "$uniondir"
<snip>
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified.
I'm fine with this approach. aufs was used originally (well, unionfs) because it was a cheap way to say "don't break this chroot". Copying is fine.
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
How many people are building clean packages on a size-restricted system? I'm guessing this is an edge case where you might be one of the few exceptions. -Dan
On Sat, Jan 23, 2010 at 11:39 PM, Dan McGee <dpmcgee@gmail.com> wrote:
On Sat, Jan 23, 2010 at 9:59 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On Sat, Jan 23, 2010 at 7:29 PM, Allan McRae <allan@archlinux.org> wrote:
On 24/01/10 09:08, Pierre Schmitz wrote:
Hi devs, <snip>
This can be easily done by changing this in makechrootpkg:
mount -t aufs none -o "dirs=$chrootdir/$LAYER=rw:$chrootdir/root=ro" "$uniondir"
to
rsync -a --delete --progress -h -c -W "$chrootdir/root/" "$uniondir"
<snip>
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified.
I'm fine with this approach. aufs was used originally (well, unionfs) because it was a cheap way to say "don't break this chroot". Copying is fine.
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
How many people are building clean packages on a size-restricted system? I'm guessing this is an edge case where you might be one of the few exceptions.
Tis true, I was just pointing out that it COULD cause issues. For instance, if you home partition is small and you're bittorrenting 8gigs of Friends episodes (like Dan does all the time), you might end up running out and not being able to build until the torrent is done and you move them to some other location. It's definitely not a show stopper, but using rsync isn't without its own problems
Am Sonntag 24 Januar 2010 schrieb Aaron Griffin:
On Sat, Jan 23, 2010 at 11:39 PM, Dan McGee <dpmcgee@gmail.com> wrote:
On Sat, Jan 23, 2010 at 9:59 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On Sat, Jan 23, 2010 at 7:29 PM, Allan McRae <allan@archlinux.org> wrote:
On 24/01/10 09:08, Pierre Schmitz wrote:
Hi devs, <snip>
This can be easily done by changing this in makechrootpkg:
mount -t aufs none -o "dirs=$chrootdir/$LAYER=rw:$chrootdir/root=ro" "$uniondir"
to
rsync -a --delete --progress -h -c -W "$chrootdir/root/" "$uniondir"
<snip>
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified.
I'm fine with this approach. aufs was used originally (well, unionfs) because it was a cheap way to say "don't break this chroot". Copying is fine.
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
How many people are building clean packages on a size-restricted system? I'm guessing this is an edge case where you might be one of the few exceptions.
Tis true, I was just pointing out that it COULD cause issues. For instance, if you home partition is small and you're bittorrenting 8gigs of Friends episodes (like Dan does all the time), you might end up running out and not being able to build until the torrent is done and you move them to some other location.
It's definitely not a show stopper, but using rsync isn't without its own problems
When this change happens, shall we drop the aufs patch too from kernel? I don't use it at all. greetings tpowa -- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa@archlinux.org
On Sun, Jan 24, 2010 at 1:38 AM, Tobias Powalowski <t.powa@gmx.de> wrote:
Am Sonntag 24 Januar 2010 schrieb Aaron Griffin:
On Sat, Jan 23, 2010 at 11:39 PM, Dan McGee <dpmcgee@gmail.com> wrote:
On Sat, Jan 23, 2010 at 9:59 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On Sat, Jan 23, 2010 at 7:29 PM, Allan McRae <allan@archlinux.org> wrote:
On 24/01/10 09:08, Pierre Schmitz wrote:
Hi devs, <snip>
This can be easily done by changing this in makechrootpkg:
mount -t aufs none -o "dirs=$chrootdir/$LAYER=rw:$chrootdir/root=ro" "$uniondir"
to
rsync -a --delete --progress -h -c -W "$chrootdir/root/" "$uniondir"
<snip>
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified.
I'm fine with this approach. aufs was used originally (well, unionfs) because it was a cheap way to say "don't break this chroot". Copying is fine.
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
How many people are building clean packages on a size-restricted system? I'm guessing this is an edge case where you might be one of the few exceptions.
Tis true, I was just pointing out that it COULD cause issues. For instance, if you home partition is small and you're bittorrenting 8gigs of Friends episodes (like Dan does all the time), you might end up running out and not being able to build until the torrent is done and you move them to some other location.
It's definitely not a show stopper, but using rsync isn't without its own problems
When this change happens, shall we drop the aufs patch too from kernel? I don't use it at all.
Not just yet, the ISOs use aufs, and there's no real good system to replace that
Am Sonntag 24 Januar 2010 schrieb Aaron Griffin:
On Sun, Jan 24, 2010 at 1:38 AM, Tobias Powalowski <t.powa@gmx.de> wrote:
Am Sonntag 24 Januar 2010 schrieb Aaron Griffin:
On Sat, Jan 23, 2010 at 11:39 PM, Dan McGee <dpmcgee@gmail.com> wrote:
On Sat, Jan 23, 2010 at 9:59 PM, Aaron Griffin <aaronmgriffin@gmail.com>
wrote:
On Sat, Jan 23, 2010 at 7:29 PM, Allan McRae <allan@archlinux.org> wrote:
On 24/01/10 09:08, Pierre Schmitz wrote: > Hi devs, > <snip> > > This can be easily done by changing this in makechrootpkg: > > mount -t aufs none -o > "dirs=$chrootdir/$LAYER=rw:$chrootdir/root=ro" "$uniondir" > > to > > rsync -a --delete --progress -h -c -W "$chrootdir/root/" > "$uniondir" > > <snip>
You get a +1 from me for the idea. But looking at the rsync line, I think that this will not work if I use makechrootpkg with "-- -i" to install a package then use it without "-c" to build on top of that? I guess we should not do the rsync if "-c" is not specified.
I'm fine with this approach. aufs was used originally (well, unionfs) because it was a cheap way to say "don't break this chroot". Copying is fine.
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
How many people are building clean packages on a size-restricted system? I'm guessing this is an edge case where you might be one of the few exceptions.
Tis true, I was just pointing out that it COULD cause issues. For instance, if you home partition is small and you're bittorrenting 8gigs of Friends episodes (like Dan does all the time), you might end up running out and not being able to build until the torrent is done and you move them to some other location.
It's definitely not a show stopper, but using rsync isn't without its own problems
When this change happens, shall we drop the aufs patch too from kernel? I don't use it at all.
Not just yet, the ISOs use aufs, and there's no real good system to replace that
Ok, just an other thing, shall i add aufs patch to the lts kernel if we bump it to .32? Just in case to be able to provide lts install isos. greetings tpowa -- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa@archlinux.org
On Sat, 23 Jan 2010 21:59:31 -0600, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
The size overhead is about 500MB in its worst case. If you are that low on disk space you should not think about building packages. -- Pierre Schmitz, https://users.archlinux.de/~pierre
On 24/01/10 22:00, Pierre Schmitz wrote:
On Sat, 23 Jan 2010 21:59:31 -0600, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
The size overhead is about 500MB in its worst case. If you are that low on disk space you should not think about building packages.
Well, the worst case is 2GB as I have four chroots. (testing/stable-i686/x86_64). Allan
On Sun, 24 Jan 2010 22:05:34 +1000, Allan McRae <allan@archlinux.org> wrote:
On 24/01/10 22:00, Pierre Schmitz wrote:
On Sat, 23 Jan 2010 21:59:31 -0600, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
The size overhead is about 500MB in its worst case. If you are that low on disk space you should not think about building packages.
Well, the worst case is 2GB as I have four chroots. (testing/stable-i686/x86_64).
Allan
Of course this overhead was per chroot. And what if one has a thousand chroots? I think we are arguing about a noon-issue here. The point is that aufs just does not work (and I have tried different snapshots and kernels). I also doubt that we have a majority of devs with four chroots but less than 2GB free disk space. -- Pierre Schmitz, https://users.archlinux.de/~pierre
On Sun, Jan 24, 2010 at 6:15 AM, Pierre Schmitz <pierre@archlinux.de> wrote:
On Sun, 24 Jan 2010 22:05:34 +1000, Allan McRae <allan@archlinux.org> wrote:
On 24/01/10 22:00, Pierre Schmitz wrote:
On Sat, 23 Jan 2010 21:59:31 -0600, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
BUT (there's always a but), some systems are size restricted. For instance, slicehost slices have very limited disk space, so copying the chroot to do this has potential to cause issues. I don't like special case code for shitty systems, but perhaps we could keep both aufs and rsync code in there, enabled by a command line switch? (-U use a union (aufs) to keep chroot clean [DEPRECATED])
The size overhead is about 500MB in its worst case. If you are that low on disk space you should not think about building packages.
Well, the worst case is 2GB as I have four chroots. (testing/stable-i686/x86_64).
Allan
Of course this overhead was per chroot. And what if one has a thousand chroots? I think we are arguing about a noon-issue here. The point is that aufs just does not work (and I have tried different snapshots and kernels). I also doubt that we have a majority of devs with four chroots but less than 2GB free disk space.
I was actually thinking of a case where one is building multiple packages at once. Still, as most people (including myself) have pointed out, it's really a moot issue. It's a specific problem that I simply wanted to air out, in the off chance that it wasn't thought of. Carry on :)
Am Sonntag, 24. Januar 2010 00:08:27 schrieb Pierre Schmitz:
PS: We should probably get rid of aufs and merge this change into our devtools.
I have commited these changes. See http://projects.archlinux.org/devtools.git/commit/?id=f8ab1fb7a7a506406ed6f2... I'll probably make some other changes and put a package into testing soon. -- Pierre Schmitz, https://users.archlinux.de/~pierre
participants (5)
-
Aaron Griffin
-
Allan McRae
-
Dan McGee
-
Pierre Schmitz
-
Tobias Powalowski