Re: [arch-general] Updating iputils over NFS
On Monday 01 Oct 2012 14:22:41 Thomas Bächler wrote:
IMO, root file systems on NFS are a failure by design anyway - I worked in such a scenario for years and it is a bad bad bad idea. While we should fix easy problems such as this one, we should not spend too much time on making this work.
I'd be really interested in your reasons for this. I'm running a network- booted cluster, and things seem much easier using NFS for this than it would be to have a hard disk in each node and use puppet (or similar) for administration. Paul
Am 01.10.2012 15:21, schrieb Paul Gideon Dann:
On Monday 01 Oct 2012 14:22:41 Thomas Bächler wrote:
IMO, root file systems on NFS are a failure by design anyway - I worked in such a scenario for years and it is a bad bad bad idea. While we should fix easy problems such as this one, we should not spend too much time on making this work.
I'd be really interested in your reasons for this. I'm running a network- booted cluster, and things seem much easier using NFS for this than it would be to have a hard disk in each node and use puppet (or similar) for administration.
We have a 100MBit/s network, which may or may not be the bottleneck Loading a modern desktop environment like KDE over NFS just takes too long. A user needs to wait up to 5 minutes after login, starting applications isn't instant, everything feels like 10 years ago speed-wise. The same goes for boot times. The worst part however is that booting a Linux system over NFS never works quite right out of the box, you need too many tricks to make it work smoothly. As lot of the OS is shared, upgrading parts of the system while it is running leads to problems. There is no way to share the entire system EXCEPT a number of config files. All my machines now have a small hard drive or SSD and have an option in the boot menu to sync an updated operating system on reboot - this uses rsync and --exclude to omit certain config files that are per-machine. Administration of this scheme has proven much easier and more robust than the old NFS-based one.
On Monday 01 Oct 2012 15:34:41 Thomas Bächler wrote:
We have a 100MBit/s network, which may or may not be the bottleneck Loading a modern desktop environment like KDE over NFS just takes too long. A user needs to wait up to 5 minutes after login, starting applications isn't instant, everything feels like 10 years ago speed-wise. The same goes for boot times.
The worst part however is that booting a Linux system over NFS never works quite right out of the box, you need too many tricks to make it work smoothly.
As lot of the OS is shared, upgrading parts of the system while it is running leads to problems. There is no way to share the entire system EXCEPT a number of config files.
All my machines now have a small hard drive or SSD and have an option in the boot menu to sync an updated operating system on reboot - this uses rsync and --exclude to omit certain config files that are per-machine. Administration of this scheme has proven much easier and more robust than the old NFS-based one.
Makes sense. I would certainly think twice for something as complex as a full interactive desktop setup. I hold by NFS for this cluster, though. I really like your sync-on-boot solution. I'd like to store this information away for future use. How did you implement it? Something in an initrd hook? Paul
Am 01.10.2012 15:50, schrieb Paul Gideon Dann:
Makes sense. I would certainly think twice for something as complex as a full interactive desktop setup. I hold by NFS for this cluster, though.
If you don't run a desktop, then you may be fine.
I really like your sync-on-boot solution. I'd like to store this information away for future use. How did you implement it? Something in an initrd hook?
sysupgrade install hook (run without autodetect): build() { add_binary rsync add_binary findmnt add_binary "/usr/lib/initcpio/ipconfig" "/bin/ipconfig" add_runscript add_checked_modules "/drivers/net" add_module nfs add_file "/root/sync/sync-include-file" "/sync-include-file" } help() { cat <<HELPEOF Custom Arch Linux Update on Boot HELPEOF } sysupgrade run hook: run_hook() { ipconfig ip=dhcp mount_handler="sysupgrade_mount_handler" } sysupgrade_mount_handler() { default_mount_handler "$@" # mount /boot if it exists realtab=/new_root/etc/fstab if [ -f "$realtab" ]; then if boot_source=$(findmnt -snero source --tab-file="$realtab" /boot); then mountopts=$(findmnt -snero options --tab-file="$realtab" /boot) fsck_device "$boot_source" msg ":: mounting '$boot_source' on /boot" mount "$boot_source" /new_root/boot -o "$mountopts" fi fi rsync --delete -ahvuHPAXE \ --include-from /sync-include-file \ 192.168.0.1::arch_sysupgrade/ /new_root/ msg ":: umounting '$boot_source' from /boot" umount /new_root/boot } my sync_include_file looks like this: # System files - /boot/syslinux/ldlinux.sys - /boot/lost+found/ + /dev/ - /dev/* + /home/ - /home/* - /lost+found/ + /media/ - /media/* + /mnt/ - /mnt/* + /proc/ - /proc/* + /run/ - /run/* - /root/.* + /sys/ - /sys/* + /tmp/ - /tmp/* + /usr/local/ - /usr/local/* - /var/tmp/**/ + /var/**/ + /var/lock + /var/run + /var/mail + /var/lib/pacman/** + /var/lib/texmf/** - /var/** # Site-specific config files - /boot/syslinux/syslinux.cfg - /etc/hostname - /etc/fstab - /etc/resolv.conf - /etc/ssh/ssh_host_* - /etc/udev/rules.d/01-site-input.rules - /etc/X11/xorg.conf_* - /etc/X11/xorg.conf.d/05-local-keyboard.conf - /usr/share/config/kdm/kdmrc Note that this syncs the whole content of /var/lib/{pacman,texmf}, but otherwise only recreates the directory structure of /var and does not sync any files. Also note that you must load the kernel and initramfs from the network when running sysupgrade, so you can boot directly into the updated system when finished without a further reboot.
On Monday 01 Oct 2012 17:19:40 Thomas Bächler wrote:
Am 01.10.2012 15:50, schrieb Paul Gideon Dann:
Makes sense. I would certainly think twice for something as complex as a full interactive desktop setup. I hold by NFS for this cluster, though. If you don't run a desktop, then you may be fine.
I really like your sync-on-boot solution. I'd like to store this information away for future use. How did you implement it? Something in an initrd hook? sysupgrade install hook (run without autodetect):
[...]
sysupgrade run hook:
[...]
my sync_include_file looks like this:
[...]
# Site-specific config files [...]
Note that this syncs the whole content of /var/lib/{pacman,texmf}, but otherwise only recreates the directory structure of /var and does not sync any files.
Also note that you must load the kernel and initramfs from the network when running sysupgrade, so you can boot directly into the updated system when finished without a further reboot.
Thanks. I'm not sure what sysupgrade you're referring to? Sysupgrade seems to be a BSD thing. It seems like this stuff needs to happen in the initrd, but I'm a little confused that you haven't mentioned it. What is it that runs these hooks? Cheers, Paul
Am 02.10.2012 12:00, schrieb Paul Gideon Dann:
Thanks. I'm not sure what sysupgrade you're referring to? Sysupgrade seems to be a BSD thing. It seems like this stuff needs to happen in the initrd, but I'm a little confused that you haven't mentioned it. What is it that runs these hooks?
Yes, those are mkinitcpio hooks.
On Tuesday 02 Oct 2012 13:15:21 Thomas Bächler wrote:
Am 02.10.2012 12:00, schrieb Paul Gideon Dann:
Thanks. I'm not sure what sysupgrade you're referring to? Sysupgrade seems to be a BSD thing. It seems like this stuff needs to happen in the initrd, but I'm a little confused that you haven't mentioned it. What is it that runs these hooks?
Yes, those are mkinitcpio hooks.
Thanks, that could be really helpful in the future. Paul
participants (2)
-
Paul Gideon Dann
-
Thomas Bächler