[arch-general] Btrfs snapshots for upgrade operations
Hello all, Firstly can a TU or developer few it to the dev mailing list, as I don't have access. Archlinux is supporting btrfs for root filesystem some time now. Have any work or thoughts been done for supporting snapshots before update packages? This way you can keep record of what's happening to your system and easily rollback if something breaks. I know Fedora and Suse (if not mistaken) supports that with their package managers. Of course I am talking about an optional feature enabled only from the user, not by default. Thoughts, ideas, discussion are welcome. Regards Leonidas
Am 25.10.2012 15:50, schrieb Leonidas Spyropoulos:
Hello all,
Firstly can a TU or developer few it to the dev mailing list, as I don't have access.
Archlinux is supporting btrfs for root filesystem some time now. Have any work or thoughts been done for supporting snapshots before update packages? This way you can keep record of what's happening to your system and easily rollback if something breaks. I know Fedora and Suse (if not mistaken) supports that with their package managers.
Of course I am talking about an optional feature enabled only from the user, not by default.
Thoughts, ideas, discussion are welcome.
This is surely a great idea when you have filesystem-level snapshots (as you have in btrfs). This sounds like something that can be easily written as a proof-of-concept implementation in a shell script. Once you work out the exact mechanics, we can consider either adding this to pacman or as an external wrapper to pacman, whatever makes sense. (This would be a discussion for the pacman-dev mailing list.) What I'm saying is: If you have a more detailed description of how this scheme should work, preferably together with a simple implementation, then a much more fruitful discussion will emerge.
On Thu, Oct 25, 2012 at 4:23 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Am 25.10.2012 15:50, schrieb Leonidas Spyropoulos:
Hello all,
Firstly can a TU or developer few it to the dev mailing list, as I don't have access.
Archlinux is supporting btrfs for root filesystem some time now. Have any work or thoughts been done for supporting snapshots before update packages? This way you can keep record of what's happening to your system and easily rollback if something breaks. I know Fedora and Suse (if not mistaken) supports that with their package managers.
Of course I am talking about an optional feature enabled only from the user, not by default.
Thoughts, ideas, discussion are welcome.
This is surely a great idea when you have filesystem-level snapshots (as you have in btrfs).
This sounds like something that can be easily written as a proof-of-concept implementation in a shell script. Once you work out the exact mechanics, we can consider either adding this to pacman or as an external wrapper to pacman, whatever makes sense. (This would be a discussion for the pacman-dev mailing list.)
What I'm saying is: If you have a more detailed description of how this scheme should work, preferably together with a simple implementation, then a much more fruitful discussion will emerge.
Basically you tell the FS to subvol / to a location like /.pacmansnaps/$datetime. In the initramfs, you need an emergency facility to let the user list the available subvols (btrfs su list /) and choose one to mount (which would require adding subvol=/foo/bar or subvolid=12345 to the rootflags). Another thing to think about would be garbage collection of old snapshots. btrfs also has a way of specifying the default mounted subvolume of a filesystem. I suppose a shell script would use the btrfs command to manipulate the FS, while pacman would probably use IOCTLs?
Am 25.10.2012 17:03, schrieb Jan Steffens:
I suppose a shell script would use the btrfs command to manipulate the FS, while pacman would probably use IOCTLs?
I don't know the topic too well, but this would make sense.
On Thu, Oct 25, 2012 at 3:50 PM, Leonidas Spyropoulos <artafinde@gmail.com> wrote:
Archlinux is supporting btrfs for root filesystem some time now. Have any work or thoughts been done for supporting snapshots before update packages? This way you can keep record of what's happening to your system and easily rollback if something breaks. I know Fedora and Suse (if not mistaken) supports that with their package managers.
I put together a proof-of-concept a while back for doing something like this. It "worked", but was nowhere near ready for public consumption, so I never posted it anywhere. What I thought would be nice was the ability for an upgrade to take effect on the next reboot without touching the running system. No need to integrate with pacman, a wrapper in bash would do. It went something like this (assuming / is on btrfs): create a new subvolume as a snapshot of / mount the new subvolume on /mnt/ pacman -Syu --root=/mnt mark the new subvolume as the default one This means the current rootfs is not touched at all during the upgrade. However, next time the rootfs is remounted (i.e., on the next reboot) you will get the upgraded system. Moreover, if something went pear shaped, the subvolume of your old system should still be around which you can boot into with the right kernel parameter. Obviously any changes you do to the rootfs after creating the snapshot will be lost on the next reboot, so some care must be taken to make sure you know what you are doing :-) (it might be reasonable to only allow this if the rootfs is mounte read-only). Cheers, Tom
Hey Tom On Thu, Oct 25, 2012 at 4:02 PM, Tom Gundersen <teg@jklm.no> wrote:
On Thu, Oct 25, 2012 at 3:50 PM, Leonidas Spyropoulos <artafinde@gmail.com> wrote:
Archlinux is supporting btrfs for root filesystem some time now. Have any work or thoughts been done for supporting snapshots before update packages? This way you can keep record of what's happening to your system and easily rollback if something breaks. I know Fedora and Suse (if not mistaken) supports that with their package managers.
I put together a proof-of-concept a while back for doing something like this. It "worked", but was nowhere near ready for public consumption, so I never posted it anywhere.
Would you be able to share it?
What I thought would be nice was the ability for an upgrade to take effect on the next reboot without touching the running system. No need to integrate with pacman, a wrapper in bash would do.
It went something like this (assuming / is on btrfs):
create a new subvolume as a snapshot of / mount the new subvolume on /mnt/ pacman -Syu --root=/mnt mark the new subvolume as the default one
This means the current rootfs is not touched at all during the upgrade. However, next time the rootfs is remounted (i.e., on the next reboot) you will get the upgraded system. Moreover, if something went pear shaped, the subvolume of your old system should still be around which you can boot into with the right kernel parameter.
Obviously any changes you do to the rootfs after creating the snapshot will be lost on the next reboot, so some care must be taken to make sure you know what you are doing :-) (it might be reasonable to only allow this if the rootfs is mounte read-only).
Cheers,
Tom
There is an interesting conversation in btrfs maling list happening now : http://comments.gmane.org/gmane.comp.file-systems.btrfs/20738 -- Caution: breathing may be hazardous to your health. #include <stdio.h> int main(){printf("%s","\x4c\x65\x6f\x6e\x69\x64\x61\x73");}
On Thu, Oct 25, 2012 at 12:02 PM, Tom Gundersen <teg@jklm.no> wrote:
On Thu, Oct 25, 2012 at 3:50 PM, Leonidas Spyropoulos What I thought would be nice was the ability for an upgrade to take effect on the next reboot without touching the running system. No need to integrate with pacman, a wrapper in bash would do. Tom
Hi! What files would be upgraded on the next reboot? One thing I love about GNU/Linux is it's hability to be hot-upgraded without the need to reboot.
create a new subvolume as a snapshot of / mount the new subvolume on /mnt/ pacman -Syu --root=/mnt mark the new subvolume as the default one
In the initramfs, you need an emergency facility to let the user list the available subvols (btrfs su list /) and choose one to mount (which would require adding subvol=/foo/bar or subvolid=12345 to the rootflags).
And don't forget about /boot, or any other relevant btrfs mountpoint.
How much space does a subvolume/snapshot require? Will you snapshot all btrfs volumes, or just system-upgrade related one. For example, /home and /tmp vs /var and /etc. Also this necessitates a subvol flag per partition - could be messy. Or is it a bad idea to allow to mix &match upgrade subvolumes - which would require keeping all the subvolume IDs synchronized? It seems logical to swap the order - create a snapshot of the current system which will then be upgraded - especially for security upgrades which need to be effective immediately, unless it is possible to remount a volume without rebooting. It seems that *pacnew and *pacsave files will no longer be required. Just some thoughts, Yclept Nemo
On Sun, Oct 28, 2012 at 3:50 PM, Yclept Nemo <orbisvicis@gmail.com> wrote:
create a new subvolume as a snapshot of / mount the new subvolume on /mnt/ pacman -Syu --root=/mnt mark the new subvolume as the default one
How much space does a subvolume/snapshot require?
It is copy-on-write, so at first it takes no space at all, but as the two subvolumes start to diverge the space required would increase.
Will you snapshot all btrfs volumes, or just system-upgrade related one.
Only the ones where pacman writes information. In an ideal world we would get away with only snapshotting /usr. At the moment we certainly need to snapshot /etc and /var too, but there might be ways to get around that in the future.
It seems logical to swap the order - create a snapshot of the current system which will then be upgraded - especially for security upgrades which need to be effective immediately, unless it is possible to remount a volume without rebooting.
Sure, that would be the standard thing to do. However, I think it would be much more interesting to use btrfs to allow us to do "offline" upgrades without actually having to go offline. The reason is that online upgrades (i.e., what we do now) never really worked. It is all to easy to cause crashes by upgrading libs/modules.
It seems that *pacnew and *pacsave files will no longer be required.
I think those would still be required, at least for the time being. As long as pacman owns files in /etc, there must be a way to merge them on upgrade... Cheers, Tom
participants (6)
-
Jan Steffens
-
Leonidas Spyropoulos
-
Martín Cigorraga
-
Thomas Bächler
-
Tom Gundersen
-
Yclept Nemo