Has anyone done any research on stateless ArchLinux instances. A stateless Arch would be one where the root filesystem is mounted read-only and nothing changes there. Thus it can mounted over network (using NFS, NBD and similar) by several, diskless, PCs at the same time. I plan to have per user HOME directories on a server (again NFS or similar), and users credentials in LDAP. /var/run beeing a link to a tmpfs /run, and by using systemd-journal without /var/log/journal (it will store logs in memory) a lot of things avoid hitting the disk already. I'd use connman for handling the net connection and it seems to require a writable /var/lib/connman/ Anyone with any experience with this? -- дамјан
On Mon, Jul 9, 2012 at 10:10 AM, Damjan <gdamjan@gmail.com> wrote:
I'd use connman for handling the net connection and it seems to require a writable /var/lib/connman/
No experiece at all, but I'd say that /var must be writeable too. Think of it some like the /home of the system, so you should have one per machine in the NFS server. Just my two cents... -- Rodrigo
I'd use connman for handling the net connection and it seems to require a writable /var/lib/connman/
No experiece at all, but I'd say that /var must be writeable too. Think of it some like the /home of the system, so you should have one per machine in the NFS server.
Just my two cents...
And /tmp. Obviously you need to remount,rw to update and you may have problems forcing it to remount,ro without a reboot. You will probably be rebooting after running pacman, most of the time though anyway. Much simpler on OpenBSD, mount -urf works well and every time and there are far less reboots required too. A grsecurity kernel can send you logs whenever a remount takes place and can't miss any like when using monit or something. A benefit over OpenBSD. -- ________________________________________________________ Why not do something good every day and install BOINC. ________________________________________________________
On Monday 09 Jul 2012 10:11:43 Kevin Chadwick wrote:
No experiece at all, but I'd say that /var must be writeable too. Think of it some like the /home of the system, so you should have one per machine in the NFS server. And /tmp.
/tmp is a tmpfs for a default Arch install, so you don't need to worry about that. Paul
/tmp is a tmpfs for a default Arch install, so you don't need to worry about that.
So it is, likely on the clients too and so may not have applied here anyway. -- ________________________________________________________ Why not do something good every day and install BOINC. ________________________________________________________
On Mon, Jul 9, 2012 at 10:10 AM, Damjan <gdamjan@gmail.com> wrote:
Has anyone done any research on stateless ArchLinux instances.
A stateless Arch would be one where the root filesystem is mounted read-only and nothing changes there. Thus it can mounted over network (using NFS, NBD and similar) by several, diskless, PCs at the same time.
I plan to have per user HOME directories on a server (again NFS or similar), and users credentials in LDAP.
/var/run beeing a link to a tmpfs /run, and by using systemd-journal without /var/log/journal (it will store logs in memory) a lot of things avoid hitting the disk already.
I'd use connman for handling the net connection and it seems to require a writable /var/lib/connman/
Anyone with any experience with this?
I have been working towards initscripts allowing this, and various upstreams (such as util-linux) should also support this setup. That said, I have not actually tested this to any great extent, so don't know how well it will work (feedback very welcome!). What should work (but might not!): /etc and /usr (and /lib, /sbin, /bin) should be able to be mounted read-only. I expect you'll have to figure out how to deal with /etc/resolv.conf, I wonder if NetworkManager has learnt how to deal with this gracefully since I last checked... What will not work: as Rodrigo said, you'll still need /var to be mounted read-write, the point of /var is for applications to be able to write to it. Moreover, /var must be unique to each installation, and cannot be shared (you can put it on an NFS share though, just make sure you have one for each machine). Moreover, even if /etc/ is mounted read-only, you probably want one per machine. You might get away with sharing it, but then all your hostnames will be the same for instance. Importantly: you don't want /etc/machine-id to be shared by different machines (as it needs to be unique). If you do decide to share /etc, you can replace /etc/machine-id by an empty file and systemd will create a random one at every boot (in /run) and use that instead, so you should be fine in this respect. HTH, Tom
On Mon, Jul 9, 2012 at 10:51 AM, Tom Gundersen <teg@jklm.no> wrote:
On Mon, Jul 9, 2012 at 10:10 AM, Damjan <gdamjan@gmail.com> wrote:
Has anyone done any research on stateless ArchLinux instances.
A stateless Arch would be one where the root filesystem is mounted read-only and nothing changes there. Thus it can mounted over network (using NFS, NBD and similar) by several, diskless, PCs at the same time.
I plan to have per user HOME directories on a server (again NFS or similar), and users credentials in LDAP.
/var/run beeing a link to a tmpfs /run, and by using systemd-journal without /var/log/journal (it will store logs in memory) a lot of things avoid hitting the disk already.
I'd use connman for handling the net connection and it seems to require a writable /var/lib/connman/
Anyone with any experience with this?
I have been working towards initscripts allowing this, and various upstreams (such as util-linux) should also support this setup. That said, I have not actually tested this to any great extent, so don't know how well it will work (feedback very welcome!).
What should work (but might not!): /etc and /usr (and /lib, /sbin, /bin) should be able to be mounted read-only. I expect you'll have to figure out how to deal with /etc/resolv.conf, I wonder if NetworkManager has learnt how to deal with this gracefully since I last checked...
What will not work: as Rodrigo said, you'll still need /var to be mounted read-write, the point of /var is for applications to be able to write to it. Moreover, /var must be unique to each installation, and cannot be shared (you can put it on an NFS share though, just make sure you have one for each machine). Moreover, even if /etc/ is mounted read-only, you probably want one per machine. You might get away with sharing it, but then all your hostnames will be the same for instance. Importantly: you don't want /etc/machine-id to be shared by different machines (as it needs to be unique). If you do decide to share /etc, you can replace /etc/machine-id by an empty file and systemd will create a random one at every boot (in /run) and use that instead, so you should be fine in this respect.
HTH,
Tom
Reference for the machine-id stuff: <http://0pointer.de/public/systemd-man/machine-id.html> or machine-id(5) if you have access to a systemd system. -t
On Mon, 9 Jul 2012 10:51:11 +0200 Tom Gundersen <teg@jklm.no> wrote:
[...]
What should work (but might not!): /etc and /usr (and /lib, /sbin, /bin) should be able to be mounted read-only. I expect you'll have to figure out how to deal with /etc/resolv.conf, I wonder if NetworkManager has learnt how to deal with this gracefully since I last checked...
This has been working for quite some time on all my machines. The only real problem is cups which wants to write to /etc/cups and upstream refuses to fix this. Debian has some patches which offer only a partial solution. I solved it by recompilation with --sysconfig=/var/lib/cups. Assuming DHCP, the resolv.conf file can be protected in two ways: (i) For dhcpcd, use "nohook resolv.conf" in dhcpcd.conf and use a predefined DNS server (like 192.168.1.1 or any public dns provider); also works with netcfg. (ii) For other DHCP clients (dhclient perhaps) one can replace /etc/resolv.conf with a symlink to /run/resolv.conf. This was a discussion on gnome dev ML sometime ago, and I don't know whether this fix was accepted "officially" anywhere or remained a folk story. AFAIK, but this can be wrong, the real problem with NM is not having read-only resolv.conf, but protecting /etc/hosts... However, having NM on a serevr sounds like a bad idea to start with.
What will not work: as Rodrigo said, you'll still need /var to be mounted read-write, the point of /var is for applications to be able to write to it. Moreover, /var must be unique to each installation, and cannot be shared (you can put it on an NFS share though, just make sure you have one for each machine). Moreover, even if /etc/ is mounted read-only, you probably want one per machine. You might get away with sharing it, but then all your hostnames will be the same for instance. Importantly: you don't want /etc/machine-id to be shared by different machines (as it needs to be unique). If you do decide to share /etc, you can replace /etc/machine-id by an empty file and systemd will create a random one at every boot (in /run) and use that instead, so you should be fine in this respect.
HTH,
Tom
-- Leonid Isaev GnuPG key: 0x164B5A6D Fingerprint: C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
On Mon, Jul 9, 2012 at 6:20 PM, Leonid Isaev <lisaev@umail.iu.edu> wrote:
AFAIK, but this can be wrong, the real problem with NM is not having read-only resolv.conf, but protecting /etc/hosts...
I don't see a problem with read-only /etc/hosts. It shouldn't contain anything other than "::1 localhost" anyway. Use nss-myhostname. -- Mantas Mikulėnas
On Monday 09 Jul 2012 10:10:07 Damjan wrote:
Has anyone done any research on stateless ArchLinux instances.
A stateless Arch would be one where the root filesystem is mounted read-only and nothing changes there. Thus it can mounted over network (using NFS, NBD and similar) by several, diskless, PCs at the same time.
I plan to have per user HOME directories on a server (again NFS or similar), and users credentials in LDAP.
/var/run beeing a link to a tmpfs /run, and by using systemd-journal without /var/log/journal (it will store logs in memory) a lot of things avoid hitting the disk already.
I'd use connman for handling the net connection and it seems to require a writable /var/lib/connman/
Anyone with any experience with this?
I run an ArchLinux processing cluster for work. The nodes are diskless, so I can certainly offer some advice for getting this setup to work. Most important tip: don't try to use NFSv4. You can't boot Arch over NFSv4 yet. My setup has the nodes mounting root rw, but in practice they never touch it except for when I run an upgrade or do some manual configuration, which I usually do from a node (because it's easier). Each node has a separate /var directory stored on the server. I added a script that uses the initscripts hooks to mount the correct /var for each node based on its hostname (which is assigned via DHCP). If you don't care about retaining state, you could probably mount /var as a tmpfs and copy a template into it, which would avoid needing to create a directory on the server for each node. I'd be happy to provide further details if you'd like some pointers, but I got most of the important information from the Wiki, so pretty much all you need is there. Paul
On Monday 09 Jul 2012 10:08:51 you wrote:
My setup has the nodes mounting root rw, but in practice they never touch it except for when I run an upgrade or do some manual configuration, which I usually do from a node (because it's easier).
Oh, also, my setup has a separate root shared by the nodes; they don't share the same root as the server. I don't think there's a good way to avoid that, at least not for my usecase. Paul
On 09.07.2012 10:10, Damjan wrote:
Has anyone done any research on stateless ArchLinux instances.
A stateless Arch would be one where the root filesystem is mounted read-only and nothing changes there. Thus it can mounted over network (using NFS, NBD and similar) by several, diskless, PCs at the same time.
Thomas told me that he set up an initramfs that allows to fetch the rootfs using rsync. That way you save quite some bandwidth, gain speed and you can keep the "old" system if you don't want newer software. You also won't have to worry about breaking software running on a client when updating the base system on the server (changes in config files/libraries that are (re)loaded during runtime, resource files, ...). This will however require (albeit small) disks in the clients. -- Florian Pritz
participants (8)
-
Damjan
-
Florian Pritz
-
Kevin Chadwick
-
Leonid Isaev
-
Mantas Mikulėnas
-
Paul Gideon Dann
-
Rodrigo Rivas
-
Tom Gundersen