[arch-general] Improve the filesystem package
Hi I see some possible improvements to the filesystem package to use some of the recent improvements to systemd. I don't know if they are already planned. All the empty directories and the links could be replaced by a tmpfiles.d file, like /usr/lib/tmpfiles.d/archlinux.conf. /etc/{group,gshadow,passwd,shadow} could be removed as the base users and groups are already created by the upstream sysusers.d files. The arch-specific ones (like bin or daemon) could be created by /usr/lib/sysusers.d/archlinux.conf. The others /etc/ files could be moved to /usr/share/etc/ and copied as needed to /etc using tmpfiles.d files. A filesystem.install file could launch systemd-tmpfiles and systemd-sysusers to update the local configuration when the package is updated. In the end, filesystem would contain only tmpfiles.d and sysusers.d files + /usr/share/etc. I can make a draft of this package if you want. More infos : http://0pointer.de/blog/projects/stateless.html
On Fri, Aug 1, 2014 at 12:14 PM, Yamakaky <yamakaky@yamaworld.fr> wrote:
Hi
/etc/{group,gshadow,passwd,shadow} could be removed as the base users and groups are already created by the upstream sysusers.d files. The arch-specific ones (like bin or daemon) could be created by /usr/lib/sysusers.d/archlinux.conf.
I don't think it would work. On a newly installed system, sure, all is well. But then, imagine that the system administrator, me, creates a few uses: they are inserted into /etc/passwd and friends. Then, a filesystem update wants to add a new system user, so it updates the file /usr/lib/sysusers.d/archlinux.conf. The .install script runs systemd-sysusers and... nothing happens, because this program only creates the file when there is not there in the first place. With the current situation, pacman will create a /etc/passwd.pacnew file. I am expected to notice that and do the merge manually.
The others /etc/ files could be moved to /usr/share/etc/ and copied as needed to /etc using tmpfiles.d files.
The problem is similar. Once I have modified an existing file in /etc, what will happen when a new version of the file comes from the package. Will it not be installed? Or will it overwrite my own file? Maybe you can convice systemd-tmpfiles to copy the file to /etc/watever.pacsave, I don't know. But then, where is the advantage? It looks to me like a lot of work and risk to break things for no advantage. -- Rodrigo
Then, a filesystem update wants to add a new system user, so it updates the file /usr/lib/sysusers.d/archlinux.conf. The .install script runs systemd-sysusers and... nothing happens, because this program only creates the file when there is not there in the first place.
In fact no, it add the new user/group (I just tested). Try running `systemd-sysusers`, add a user in /usr/lib/sysusers.d/base.conf then run `systemd-sysusers`.
The problem is similar. Once I have modified an existing file in /etc, what will happen when a new version of the file comes from the package. Will it not be installed? Or will it overwrite my own file? Maybe you can convice systemd-tmpfiles to copy the file to /etc/watever.pacsave, I don't know. But then, where is the advantage?
Yes, I didn't thought about pacnew, dummy I am... I wrote about /usr/share/etc for the stateless boot stuff (cf link from my previous mail), which is pretty cool. Maybe for this time we could let them in /etc.
On 08/01/2014 02:03 PM, Rodrigo Rivas wrote:
On Fri, Aug 1, 2014 at 12:14 PM, Yamakaky <yamakaky@yamaworld.fr> wrote:
Hi
/etc/{group,gshadow,passwd,shadow} could be removed as the base users and groups are already created by the upstream sysusers.d files. The arch-specific ones (like bin or daemon) could be created by /usr/lib/sysusers.d/archlinux.conf.
I don't think it would work. On a newly installed system, sure, all is well. But then, imagine that the system administrator, me, creates a few uses: they are inserted into /etc/passwd and friends. Then, a filesystem update wants to add a new system user, so it updates the file /usr/lib/sysusers.d/archlinux.conf. The .install script runs systemd-sysusers and... nothing happens, because this program only creates the file when there is not there in the first place.
Just out of curiosity, why does the filesystem package have all these systemd-related users and groups in /etc/passwd, /etc/group (and their shadow counterparts) in the first place? Updates to these were the prime cause of problems people were experiencing. Also, why ship the /etc/shadow, /etc/gshadow files at all? AFAIK, nothing is supposed to mess with the shadow files anyway, except pwconv and grpconv (for initially converting a freshly installed, non-shadow system into one using shadow files), after which these files should be managed by the shadow system itself, in response to adding/removing/changing users and groups using the designated tools. Isn't the filesystem package supposed to be a kind of stable, hardly ever changing scaffold onto which other packages are supposed to attach their own changes? Why for example doesn't the systemd package add its users and groups using pre_install function in its install script? -- Alain
On 08/20/2014 02:04 PM, Alain Kalker wrote:
Also, why ship the /etc/shadow, /etc/gshadow files at all? AFAIK, nothing is supposed to mess with the shadow files anyway, except pwconv and grpconv (for initially converting a freshly installed, non-shadow system into one using shadow files), after which these files should be managed by the shadow system itself, in response to adding/removing/changing users and groups using the designated tools.
From `man pwconv`:
Each program acquires the necessary locks before conversion.
I'm quite certain that pacman doesn't do this when installing these files, and if it did, it would mean it has low-level knowledge about the shadow system which it isn't supposed to have. Not properly locking these files can lead to some very surprising, unwanted behaviour, which I experienced first-hand when upgrading the filesystem package,
Isn't the filesystem package supposed to be a kind of stable, hardly ever changing scaffold onto which other packages are supposed to attach their own changes? Why for example doesn't the systemd package add its users and groups using pre_install function in its install script?
-- Alain
On Wed, Aug 20, 2014 at 02:42:27PM +0200, Alain Kalker wrote:
On 08/20/2014 02:04 PM, Alain Kalker wrote:
Also, why ship the /etc/shadow, /etc/gshadow files at all? AFAIK, nothing is supposed to mess with the shadow files anyway, except pwconv and grpconv (for initially converting a freshly installed, non-shadow system into one using shadow files), after which these files should be managed by the shadow system itself, in response to adding/removing/changing users and groups using the designated tools.
From `man pwconv`:
Each program acquires the necessary locks before conversion.
Except that sometimes a package installs files owned by a _new_ user. So one needs some "basic" groups to exist _before_ high-level packages are unpacked. Shipping these users/groups only in un-shadowed files will lead to pwck/grck complaints... HTH, -- Leonid Isaev GPG fingerprints: DA92 034D B4A8 EC51 7EA6 20DF 9291 EE8A 043C B8C4 C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
On Fri, Aug 01, 2014 at 12:14:21PM +0200, Yamakaky wrote:
Hi
I see some possible improvements to the filesystem package to use some of the recent improvements to systemd. I don't know if they are already planned.
All the empty directories and the links could be replaced by a tmpfiles.d file, like /usr/lib/tmpfiles.d/archlinux.conf.
And what happens if my root filesystem is mounted read-only? So, please let's not allow tmpfiles to touch /etc...
/etc/{group,gshadow,passwd,shadow} could be removed as the base users and groups are already created by the upstream sysusers.d files. The arch-specific ones (like bin or daemon) could be created by /usr/lib/sysusers.d/archlinux.conf.
The others /etc/ files could be moved to /usr/share/etc/ and copied as needed to /etc using tmpfiles.d files.
A filesystem.install file could launch systemd-tmpfiles and systemd-sysusers to update the local configuration when the package is updated.
In the end, filesystem would contain only tmpfiles.d and sysusers.d files + /usr/share/etc.
I can make a draft of this package if you want.
And all that assumes that you can execute systemd-sysusers, no? Cheers, -- Leonid Isaev GPG fingerprints: DA92 034D B4A8 EC51 7EA6 20DF 9291 EE8A 043C B8C4 C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
And what happens if my root filesystem is mounted read-only? So, please let's not allow tmpfiles to touch /etc...
That's an interesting question, I don't know the answer. I think it will fail because ignoring the error would be worse for most of the cases.
And all that assumes that you can execute systemd-sysusers, no?
systemd is the only init officially supported and partial updates are not supported so I think we can assume it will be available with this new version of the package. After thinking, maybe removing the dirs from the package is not a so good idea. It reminds me the /usr/bin merge, which doesn't seem to be possible with tmpfiles.d. However, systemd-sysusers seems to be a good idea, especially for package creation. And stateless boot is cool.
On 2014-08-01 12:09, Yamakaky wrote:
And what happens if my root filesystem is mounted read-only? So, please let's not allow tmpfiles to touch /etc...
That's an interesting question, I don't know the answer. I think it will fail because ignoring the error would be worse for most of the cases.
And all that assumes that you can execute systemd-sysusers, no?
systemd is the only init officially supported and partial updates are not supported so I think we can assume it will be available with this new version of the package.
After thinking, maybe removing the dirs from the package is not a so good idea. It reminds me the /usr/bin merge, which doesn't seem to be possible with tmpfiles.d. However, systemd-sysusers seems to be a good idea, especially for package creation. And stateless boot is cool.
Stateless boot is cool for embedded appliances, not so much for desktop/server installations.
And all that assumes that you can execute systemd-sysusers, no?
Oh, I see, you are talking about emergency boot ?
On Fri, Aug 01, 2014 at 07:10:10PM +0200, Yamakaky wrote:
And all that assumes that you can execute systemd-sysusers, no?
Oh, I see, you are talking about emergency boot ?
Yes. What you suggest is OK for an update on a working system, but what about a fresh install or rescue? You call pacstrap to install into a directory, but systemd is installed _after_ filesystem and a bunch of other packages which expect all those directories. IMHO, the idea of a stateless system applies to a narrow usecase like coreos, when you magically obtained a working system (and systemd), and now can do all the cool stuff. However, coreos is not a general-purpose distro like Arch. On top of that, I remember that quite recently it was considered a no-no for a daemon to write to /etc... Cheers, -- Leonid Isaev GPG fingerprints: DA92 034D B4A8 EC51 7EA6 20DF 9291 EE8A 043C B8C4 C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
participants (5)
-
Alain Kalker
-
Doug Newgard
-
Leonid Isaev
-
Rodrigo Rivas
-
Yamakaky