[arch-projects] [initscripts][udev][RFC] first step towards removing load-modules.sh
Hi guys, This is in regards to FS#18421. I am looking at ways to move our udev package closer to upstream. The biggest difference is currently our load-modules.sh script which filters out what modules to load. This adds significant overhead to boot, so I think it is the first thing that should go. The below patch generate a blacklist file at boot from rc.conf, which should be linked to by a softlink in /etc/modprobe.d/. This allows udev to figure out blacklisting by itself. Once this is in initscripts and a similar patch is in mkinitcpio, then we can ditch load-modules.sh. One thing to note: the attached patch does "blacklist <module>" (which allows modules to be pulled in as dependencies). We could also do "install <module> /bin/false" (which has the same effect as deleting the module altogether), if that is necessary. I chose to use blacklist as it looks to be the recommended way. Thoughts? Cheers, Tom
From 56e6e5684d40b66d810a3d7d424b5aabec6d660e Mon Sep 17 00:00:00 2001 From: Tom Gundersen <teg@jklm.no> Date: Mon, 9 May 2011 22:15:43 +0200 Subject: [PATCH] udev: generate blacklist on boot
/run/initscripts/modules-blacklist.conf +fi
udev will read /etc/modprobe.d/*.conf and blacklist all modules that are listed as blacklist <module> We parse rc.conf at boot and generate such a .conf file. It cannot be written to /etc this early, so we save it to /run. A symlink exists in /etc to get the desired functionality. While we are at it we also write the list of autoloaded. This is useful in case we ever want to move to using /etc/modules-load.d/*.conf for loading modules. With this patch load-modules.sh can be removed from the udev package without loss of functionality. Original-idea-by: Benjamen Richer <br@waldteufel-online.net> Based-on-patch-by: David Reisner <d@falconindy.com> Signed-off-by: Tom Gundersen <teg@jklm.no> --- Makefile | 4 +++- rc.gen-module-lists | 32 ++++++++++++++++++++++++++++++++ rc.sysinit | 3 +++ 3 files changed, 38 insertions(+), 1 deletions(-) create mode 100755 rc.gen-module-lists diff --git a/Makefile b/Makefile index d3a1824..dfb2ff0 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ VER := $(shell git describe) -DIRS := /etc/rc.d /etc/conf.d /etc/rc.d/functions.d /etc/cron.hourly /sbin +DIRS := /etc/rc.d /etc/conf.d /etc/rc.d/functions.d /etc/cron.hourly /sbin /etc/modprobe.d /etc/module-load.d minilogd: minilogd.o @@ -13,6 +13,8 @@ install: minilogd installdirs install -m755 -t $(DESTDIR)/etc/cron.hourly adjtime install -m755 -t $(DESTDIR)/etc/rc.d functions hwclock network netfs install -m755 -t $(DESTDIR)/sbin minilogd rc + ln -s /run/initscripts/modules-blacklist.conf $(DESTDIR)/etc/modprobe.d/arch-blacklist.conf + ln -s /run/initscripts/modules-load.conf $(DESTDIR)/etc/modules-load.d/arch-modules.conf clean: rm -f minilogd minilogd.o diff --git a/rc.gen-module-lists b/rc.gen-module-lists new file mode 100755 index 0000000..14f9619 --- /dev/null +++ b/rc.gen-module-lists @@ -0,0 +1,32 @@ +#!/bin/bash +# +# /etc/rc.gen-modules-list +# + +. /etc/rc.conf + +declare -a modules blacklist +for mod in "${MODULES[@]}"; do + case $mod in + !*) blacklist+=("${mod:1}") ;; + *) modules+=("$mod") ;; + esac +done + +# create new module blacklist in /run/initscripts, there should be a symlink in /etc/modprobe.d/ pointing here +if [[ $blacklist ]]; then + /bin/mkdir -p /run/initscripts + echo "# Autogenerated from rc.conf at boot, do not edit" > /run/initscripts/modules-blacklist.conf + (( ${#blacklist[@]} )) && printf 'blacklist %s\n' "${blacklist[@]}" + +# create new module list in /run/initscripts, there should be a symlink in /etc/modules-load.d/ pointing here +if [[ $modules ]]; then + /bin/mkdir -p /run/initscripts + echo "# Autogenerated from rc.conf at boot, do not edit" > /run/initscripts/modules-load.conf + printf '%s\n' "${modules[@]}" >> /run/initscripts/modules-load.conf +fi + +unset blacklist modules + +# vim: set noet ts=2 sw=2: diff --git a/rc.sysinit b/rc.sysinit index 1521299..4e45d7c 100755 --- a/rc.sysinit +++ b/rc.sysinit @@ -83,6 +83,9 @@ if [[ $HWCLOCK_PARAMS ]]; then fi fi +# parse rc.conf and create the blacklist file for use by udev +status "Creating UDev blacklist" /etc/rc.gen-module-lists + status "Starting UDev Daemon" /sbin/udevd --daemon run_hook sysinit_udevlaunched -- 1.7.5.1
On Tue, May 10, 2011 at 10:02:07AM +0200, Tom Gundersen wrote:
Hi guys,
This is in regards to FS#18421.
I am looking at ways to move our udev package closer to upstream. The biggest difference is currently our load-modules.sh script which filters out what modules to load. This adds significant overhead to boot, so I think it is the first thing that should go.
The below patch generate a blacklist file at boot from rc.conf, which should be linked to by a softlink in /etc/modprobe.d/. This allows udev to figure out blacklisting by itself.
Once this is in initscripts and a similar patch is in mkinitcpio, then we can ditch load-modules.sh.
One thing to note: the attached patch does "blacklist <module>" (which allows modules to be pulled in as dependencies). We could also do "install <module> /bin/false" (which has the same effect as deleting the module altogether), if that is necessary. I chose to use blacklist as it looks to be the recommended way.
Thoughts?
Cheers,
Tom
I'm going to argue that we should be going the install /bin/false route, as it more accurately mimics what we currently do with load-modules.sh. This does have the drawback that if a user really does in fact want to install a given module, they need to run modprobe with -i. I don't recall if load-modules.sh allows explicit installation of a "blacklisted" module. Important: for your patchwork to be effective, we need to alter the udev rules that reference load-modules.sh. If this isn't done for the initcpio, you will almost assuredly be dumped to a rescue shell. Other than alteration of the rules, this should be an easy win in the initcpio, as we have a writeable filesystem from the start. And (correct me if im wrong), the only reason we ever need to write a blacklist file is if disablemodules= is specified on the kernel cmdline. /etc/rc.conf is verboten in early userspace because it uses arrays. As I mentioned to you on your GH pull request, I'm still a bit weary of starting up a symlink farming business in /run. I would love to see module-init-tools support for reading from /run/modprobe.d, but perhaps that's a bit of an exotic request. With the FHS revived and gearing up for a release, it seems likely that /run will be standardized, but I'm not sure if that has any clout here. semi-random aside: does anyone other than systemd support /etc/modules-load.d? d (uggh, way longer than i wanted this to be)
From 56e6e5684d40b66d810a3d7d424b5aabec6d660e Mon Sep 17 00:00:00 2001 From: Tom Gundersen <teg@jklm.no> Date: Mon, 9 May 2011 22:15:43 +0200 Subject: [PATCH] udev: generate blacklist on boot
udev will read /etc/modprobe.d/*.conf and blacklist all modules that are listed as
blacklist <module>
We parse rc.conf at boot and generate such a .conf file. It cannot be written to /etc this early, so we save it to /run. A symlink exists in /etc to get the desired functionality.
While we are at it we also write the list of autoloaded. This is useful in case we ever want to move to using /etc/modules-load.d/*.conf for loading modules.
With this patch load-modules.sh can be removed from the udev package without loss of functionality.
Original-idea-by: Benjamen Richer <br@waldteufel-online.net> Based-on-patch-by: David Reisner <d@falconindy.com> Signed-off-by: Tom Gundersen <teg@jklm.no> --- Makefile | 4 +++- rc.gen-module-lists | 32 ++++++++++++++++++++++++++++++++ rc.sysinit | 3 +++ 3 files changed, 38 insertions(+), 1 deletions(-) create mode 100755 rc.gen-module-lists
diff --git a/Makefile b/Makefile index d3a1824..dfb2ff0 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ VER := $(shell git describe) -DIRS := /etc/rc.d /etc/conf.d /etc/rc.d/functions.d /etc/cron.hourly /sbin +DIRS := /etc/rc.d /etc/conf.d /etc/rc.d/functions.d /etc/cron.hourly /sbin /etc/modprobe.d /etc/module-load.d
minilogd: minilogd.o
@@ -13,6 +13,8 @@ install: minilogd installdirs install -m755 -t $(DESTDIR)/etc/cron.hourly adjtime install -m755 -t $(DESTDIR)/etc/rc.d functions hwclock network netfs install -m755 -t $(DESTDIR)/sbin minilogd rc + ln -s /run/initscripts/modules-blacklist.conf $(DESTDIR)/etc/modprobe.d/arch-blacklist.conf + ln -s /run/initscripts/modules-load.conf $(DESTDIR)/etc/modules-load.d/arch-modules.conf
/run/initscripts/modules-blacklist.conf +fi
clean: rm -f minilogd minilogd.o diff --git a/rc.gen-module-lists b/rc.gen-module-lists new file mode 100755 index 0000000..14f9619 --- /dev/null +++ b/rc.gen-module-lists @@ -0,0 +1,32 @@ +#!/bin/bash +# +# /etc/rc.gen-modules-list +# + +. /etc/rc.conf + +declare -a modules blacklist +for mod in "${MODULES[@]}"; do + case $mod in + !*) blacklist+=("${mod:1}") ;; + *) modules+=("$mod") ;; + esac +done + +# create new module blacklist in /run/initscripts, there should be a symlink in /etc/modprobe.d/ pointing here +if [[ $blacklist ]]; then + /bin/mkdir -p /run/initscripts + echo "# Autogenerated from rc.conf at boot, do not edit" > /run/initscripts/modules-blacklist.conf + (( ${#blacklist[@]} )) && printf 'blacklist %s\n' "${blacklist[@]}" + +# create new module list in /run/initscripts, there should be a symlink in /etc/modules-load.d/ pointing here +if [[ $modules ]]; then + /bin/mkdir -p /run/initscripts + echo "# Autogenerated from rc.conf at boot, do not edit" > /run/initscripts/modules-load.conf + printf '%s\n' "${modules[@]}" >> /run/initscripts/modules-load.conf +fi + +unset blacklist modules + +# vim: set noet ts=2 sw=2: diff --git a/rc.sysinit b/rc.sysinit index 1521299..4e45d7c 100755 --- a/rc.sysinit +++ b/rc.sysinit @@ -83,6 +83,9 @@ if [[ $HWCLOCK_PARAMS ]]; then fi fi
+# parse rc.conf and create the blacklist file for use by udev +status "Creating UDev blacklist" /etc/rc.gen-module-lists + status "Starting UDev Daemon" /sbin/udevd --daemon
run_hook sysinit_udevlaunched -- 1.7.5.1
Am 10.05.2011 15:02, schrieb Dave Reisner:
I'm going to argue that we should be going the install /bin/false route, as it more accurately mimics what we currently do with load-modules.sh. This does have the drawback that if a user really does in fact want to install a given module, they need to run modprobe with -i. I don't recall if load-modules.sh allows explicit installation of a "blacklisted" module.
'blacklist' is fine. It usually only blacklists aliases of a module, but udev uses 'modprobe -b', which blacklists the module name as well - this is the behaviour I was trying to mimic with load-modules.sh.
As I mentioned to you on your GH pull request, I'm still a bit weary of starting up a symlink farming business in /run. I would love to see module-init-tools support for reading from /run/modprobe.d, but perhaps that's a bit of an exotic request. With the FHS revived and gearing up for a release, it seems likely that /run will be standardized, but I'm not sure if that has any clout here.
If we could just have a statement like include /run/modprobe.d/ in a modprobe config file. At least nothing like this is documented.
semi-random aside: does anyone other than systemd support /etc/modules-load.d?
What is that supposed to do?
On Tue, May 10, 2011 at 3:08 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Am 10.05.2011 15:02, schrieb Dave Reisner:
I'm going to argue that we should be going the install /bin/false route, as it more accurately mimics what we currently do with load-modules.sh. This does have the drawback that if a user really does in fact want to install a given module, they need to run modprobe with -i. I don't recall if load-modules.sh allows explicit installation of a "blacklisted" module.
'blacklist' is fine. It usually only blacklists aliases of a module, but udev uses 'modprobe -b', which blacklists the module name as well - this is the behaviour I was trying to mimic with load-modules.sh.
I prefer blacklist, especially if it is what was intended with load-modules.sh. However, I am ready to jump to `install /bin/false´ if things break in horrible ways.
As I mentioned to you on your GH pull request, I'm still a bit weary of starting up a symlink farming business in /run. I would love to see module-init-tools support for reading from /run/modprobe.d, but perhaps that's a bit of an exotic request.
Not at all. It makes perfect sense to make modules-init-tools use the same /run, /etc, /usr/lib hierarchy as everything else in systemd does. I am sure upstream patches would be eagerly accepted (and I would be the first one to simplify our implementation) ;-).
If we could just have a statement like include /run/modprobe.d/ in a modprobe config file. At least nothing like this is documented.
That would certainly also work for our purposes. However, I think there is a benefit in allowing packages to install their own modprobe.conf files (as described below for modules-load.d). I guess care must be taken as clashes are possible though, and this would only be interesting if it is adopted upstream so packages can rely on it across all distros.
semi-random aside: does anyone other than systemd support /etc/modules-load.d?
Not to my knowledge. However, I got the impression that Upstart is looking to implement the same standard config files as systemd is doing, so I think there is great benefit in being compatible with `everybody else´. Regardless of that, it is a neat idea. See below.
What is that supposed to do?
I wanted to suggest this for the future (at the moment this functionality is not hooked up in initscripts): /etc/modules-load.d/*.conf are just newline separated lists of modules to load at boot (so modules-load.d + modprobe.d can do the same as our MODULES array). Looking at the man pages this can be done much more nicely than what I did in my patch (actually, it works the way Dave wanted modprobe.d to work): <http://0pointer.de/public/systemd-man/modules-load.d.html>. I will remove the modprobe.d stuff from my patch, as it is not hooked up yet. I will revisit this once the next release is out. A quick idea of what I have in mind: The benefit of modprobe.d compared with the MODULES array is that packages can install lists of modules they need loaded in /usr/lib/modules-load.d/ (rather than advice the user to add them to their MODULES array). How we should do this: Generate /run/modules-load.d/rc.conf at boot based on the MODULES array. The user can add their own lists of modules to /etc/modules-load.d/*.conf and it will be the same as adding it to rc.conf. Packages can add lists of modules to /usr/lib/modules-load.d/*.conf. Then we process all these sources in the same way as we do with the MODULES array now, with the added benefit that it will be compatible with other distros (and upstream packages/tools can rely on this also on Arch). Cheers, Tom
Am 10.05.2011 15:30, schrieb Tom Gundersen:
On Tue, May 10, 2011 at 3:08 PM, Thomas Bächler <thomas@archlinux.org> wrote:
If we could just have a statement like include /run/modprobe.d/ in a modprobe config file. At least nothing like this is documented.
That would certainly also work for our purposes.
This exists, but it is deprecated for whatever reason. You are supposed to put all configuration files into /etc/modprobe.d/. Let's go with the symlink.
On Tue, May 10, 2011 at 5:53 PM, Thomas Bächler <thomas@archlinux.org> wrote:
This exists, but it is deprecated for whatever reason. You are supposed to put all configuration files into /etc/modprobe.d/.
Ah, I c. That explains the lack of documentation. -t
On Tue, May 10, 2011 at 3:02 PM, Dave Reisner <d@falconindy.com> wrote:
Important: for your patchwork to be effective, we need to alter the udev rules that reference load-modules.sh. If this isn't done for the initcpio, you will almost assuredly be dumped to a rescue shell.
Not sure I understand what you mean. When we remove load-modules.sh from udev, we must of course also delete 80-drivers.rules and adjust 81-arch.rules. However, we do not need to remove load-modules.sh at the same time as introducing this blacklisting magic. Right?
Other than alteration of the rules, this should be an easy win in the initcpio, as we have a writeable filesystem from the start. And (correct me if im wrong), the only reason we ever need to write a blacklist file is if disablemodules= is specified on the kernel cmdline. /etc/rc.conf is verboten in early userspace because it uses arrays.
Sounds like it should be easy then (I have not looked at the code much yet). Unless I am missing something (see above) we can do the initscript/mkinitcpio independently and when both are in [core] start patching udev. Cheers, Tom
On Tue, May 10, 2011 at 03:49:38PM +0200, Tom Gundersen wrote:
On Tue, May 10, 2011 at 3:02 PM, Dave Reisner <d@falconindy.com> wrote:
Important: for your patchwork to be effective, we need to alter the udev rules that reference load-modules.sh. If this isn't done for the initcpio, you will almost assuredly be dumped to a rescue shell.
Not sure I understand what you mean. When we remove load-modules.sh from udev, we must of course also delete 80-drivers.rules and adjust 81-arch.rules. However, we do not need to remove load-modules.sh at the same time as introducing this blacklisting magic. Right?
You are correct. I'm thinking two steps ahead of where you are. Just call me eager to see load-modules.sh go away. =P
Other than alteration of the rules, this should be an easy win in the initcpio, as we have a writeable filesystem from the start. And (correct me if im wrong), the only reason we ever need to write a blacklist file is if disablemodules= is specified on the kernel cmdline. /etc/rc.conf is verboten in early userspace because it uses arrays.
Sounds like it should be easy then (I have not looked at the code much yet). Unless I am missing something (see above) we can do the initscript/mkinitcpio independently and when both are in [core] start patching udev.
Yep, there should be no problems coordinating these efforts independently, as the two environments are also, for all intents and purposes, independent. d
participants (3)
-
Dave Reisner
-
Thomas Bächler
-
Tom Gundersen