On Fri, Feb 22, 2008 at 6:03 PM, Aaron Griffin wrote:

On Fri, Feb 22, 2008 at 5:56 PM, eliott wrote:

...

On 2/22/08, Aaron Griffin wrote:

...

Ok, guys - honest question. Because udev is being a big pain in the ass with the way we do module blacklisting, we might want to reevaluate it.

Right now we support blacklisting of modules in rc.conf, in addition to a kernel param disablemodules=x,y,z

Udev autoloading is controlled by MOD_AUTOLOAD and the load_modules kernel param.

We *can* use modprobe based blacklisting here, but we lose the above items. blacklisting will be controlled only by /etc/modprobe.conf (and modprobe.d/*) and we lose the ability to shut it off via rc.conf.

What do we gain? Speed and simplicity. No extraneous scripts to handle this stuff, and all that jazz.

This is how fast this script was when I originally wrote it: http://img.phraktured.net/other/udev_modules_boot.png Now it apparently takes 3 times as long due to all the added blacklisting cruft

What do we lose? Robustness. See below for an explanation of the blacklist changes

I thin modprobe.conf would probably be 'cleaner' and 'closer to home'. It would probably be a little less 'classic arch', as it moves things away from rc.conf instead of into it. It might be the right thing to do though..not sure.

That's about where I am right now. It feels cleaner, but begins moving things out of rc.conf, which is un-Archy. I dunno, I'm hoping to hear other people's opinions on this, as I'm holding off pushing out a udev with start_udev still in it until we decide what to do here... anyone have a strong opinion?

I like it in rc.conf. I've always loved the single point of config for most startup things, and I know I see this as a great selling point of Arch in our own forums and elsewhere. I'd really rather not move it outside of there. This is a point someone made earlier in the thread that I wanted to address, and it went something like this: "why don't we just do the blacklist variable stuff in the initscripts" Aaron covered it most of the way, but I just wanted to make it clear that every time you plug in a USB device or make any other hardware chage, udev triggers. If I've added a blacklisted module since the last time I booted (which may have been 50 days ago), then I want it to not load, and any processing of udev-related stuff outside of the udev framework would mean the module I added would not be blacklisted.

...

I do have one concern though. How would this effect mkinitcpio, and excluding modules via grub at boot time (whether from an install cd or from a real boot and with an initrd)?

That's harder. But we have two options. Firstly, the initramfs udev can have different sets of rules, there's nothing wrong with that. Secondly, before starting udev in the initramfs, we can do something like:

for x in $disablemodules; do echo "blacklist $x" >> /etc/modprobe.conf done

/me shrugs

Still doesn't take into account the whole "load all dependencies" thing though. Bummer. -Dan

On Sat, Feb 23, 2008 at 11:12:51AM +0100, Xavier wrote:

On Fri, Feb 22, 2008 at 07:06:04PM -0600, Dan McGee wrote:

...

Aaron covered it most of the way, but I just wanted to make it clear that every time you plug in a USB device or make any other hardware chage, udev triggers. If I've added a blacklisted module since the last time I booted (which may have been 50 days ago), then I want it to not load, and any processing of udev-related stuff outside of the udev framework would mean the module I added would not be blacklisted.

I was also thinking about this. I am afraid Aaron's ideas #2 and #3 don't take care of that, since they build the blacklist in rc.sysinit, right?

Do you suggest reverting to the previous load-modules.sh then, which built the blacklist every time? It doesn't sound very efficient, but how is it possible to get the behavior you are describing otherwise?

http://cvs.archlinux.org/cgi-bin/viewcvs.cgi/base/udev/load-modules.sh.diff?r1=1.9&r2=1.10&cvsroot=Core

Hmmm... is it possible to implement some form of caching to solve this? If load-modules.sh took a quick md5sum or the MODULES array every time it runs, we could maintain current behavior, including blacklisting modules on a running system, at little or no cost, and quite transparently. This would at least make the dependency resolution only occur when the MODULES array happens to change. -S

On Sat, Feb 23, 2008 at 8:20 AM, Dan McGee wrote:

On Sat, Feb 23, 2008 at 6:18 AM, Simo Leone wrote:

...

On Sat, Feb 23, 2008 at 11:12:51AM +0100, Xavier wrote:

...

On Fri, Feb 22, 2008 at 07:06:04PM -0600, Dan McGee wrote:

...

Aaron covered it most of the way, but I just wanted to make it clear that every time you plug in a USB device or make any other hardware chage, udev triggers. If I've added a blacklisted module since the last time I booted (which may have been 50 days ago), then I want it to not load, and any processing of udev-related stuff outside of the udev framework would mean the module I added would not be blacklisted.

I was also thinking about this. I am afraid Aaron's ideas #2 and #3 don't take care of that, since they build the blacklist in rc.sysinit, right?

Do you suggest reverting to the previous load-modules.sh then, which built the blacklist every time? It doesn't sound very efficient, but how is it possible to get the behavior you are describing otherwise?

http://cvs.archlinux.org/cgi-bin/viewcvs.cgi/base/udev/load-modules.sh.diff?r1=1.9&r2=1.10&cvsroot=Core

Hmmm... is it possible to implement some form of caching to solve this? If load-modules.sh took a quick md5sum or the MODULES array every time it runs, we could maintain current behavior, including blacklisting modules on a running system, at little or no cost, and quite transparently. This would at least make the dependency resolution only occur when the MODULES array happens to change.

Wowzers batman! Now we might as well do it the old way if we get this complex and have to md5sum the damn file anyway. The point of this is to keep multiple parses/reads of the file to a minimum, and md5sum is another fork that we don't want.

There HAS to be a way to get udev to do something similar to what we want, doesn't there?

This is exactly my point. It's getting retardedly complex. To answer Roman's question as to why 116 and 118 differ in speeds: 116 rejiggered the module blacklist (poorly, I might add) on every module load. 118 *tried* to do it once when udev started, but I was unaware that the environment is flushed _for each device_ so my "solution" actually made things slower (the blacklist was rebuild for every device, not just those that load modules). We can go back to the 116 way of loading modules and building the blacklist everytime, but we can make a few changes: a) framebuffer modules blacklisted in there is a very bad idea - move those to a udev rule similar to: KERNEL=="fb[0-9], GOTO="skip_load_modules" b) optimize the nested loops better. For right now, for each module, we loop over each dependent module, and compare that to each blacklist module... it's gross. I modified it to use \ based grep at some point, to convert it to one loop, but it's still a tad slow. c) Maybe even write some small C app that outputs our full blacklist, to optimize that messy part.

On Tue, Feb 26, 2008 at 11:30 AM, Aaron Griffin wrote:

On Tue, Feb 26, 2008 at 11:15 AM, Daniel Isenmann wrote:

...

I'm a little bit confused right now. How can I blacklist modules. I have test both, MOD_BLACKLIST=(...) and MODULES=(!...), but both don't work. udev loads every module which can be loaded. For example I blacklist nvidiafb, but udev loads it. Packages are up2date with testing repo.

Can someone please explain it or the status is on this topic?

The status is that I haven't gotten a concise answer. On this. For the time being, I think we should do the following:

a) Rebuild udev 118 with start_udev in there, for the people who keep their systems in some goofy limbo state by only updating singular packages at a time

And add a big old echo at the top saying "You are using start_dev. This script will be removed in a future release!"

b) Switch to the udev 116 way of module loading For now, yes. We know it works, and we can do some more investigation into this.

c) Remove framebuffer module loading from the load-modules script (it should never have been there in the first place). Agreed. What are the hotpoints in this script as well? Something like this seems inefficient: i="$(/sbin/modprobe -i --show-depends $1 | sed "s#^insmod /lib.*/\(.*\)\.ko.*#\1#g" | sed 's|-|_|g')"

We invoke 3 subprocesses here (modprobe, sed, and sed). Surely the two seds can be combined. k="$(echo $BLACKLIST ${MOD_BLACKLIST[@]} | sed 's|-|_|g')" j="$(echo ${MODULES[@]} | sed 's|-|_|g')" Hmm, two more seds. # add disablemodules= from commandline to blacklist k="${k} $(echo ${disablemodules} | sed 's|-|_|g' | sed 's|,| |g')" See a trend here? So we have the following in one run of load-modules.sh (if we look back at the version packaged with 116): cat: 1 invocation eval: invocation count depends on results of cat sed: 6 calls, probably not cheap echo: 5+ calls, but probably a shell builtin grep: 1 call This all means we spawn *at least* 8 processes per module passed to load-modules.sh. I can help clean this up and test if anyone else is willing to help. -Dan

On Tue, Feb 26, 2008 at 11:59 AM, Aaron Griffin wrote:

On Tue, Feb 26, 2008 at 11:54 AM, Dan McGee wrote:

...

On Tue, Feb 26, 2008 at 11:30 AM, Aaron Griffin wrote:

...

On Tue, Feb 26, 2008 at 11:15 AM, Daniel Isenmann wrote:

...

I'm a little bit confused right now. How can I blacklist modules. I have test both, MOD_BLACKLIST=(...) and MODULES=(!...), but both don't work. udev loads every module which can be loaded. For example I blacklist nvidiafb, but udev loads it. Packages are up2date with testing repo.

Can someone please explain it or the status is on this topic?

The status is that I haven't gotten a concise answer. On this. For the time being, I think we should do the following:

a) Rebuild udev 118 with start_udev in there, for the people who keep their systems in some goofy limbo state by only updating singular packages at a time

And add a big old echo at the top saying "You are using start_dev. This script will be removed in a future release!"

Doesn't work. It's run with 2>&1 >/dev/null, which is very good at keeping users uninformed of what's going on.

Well that is dumb. Shoot.

...

...

c) Remove framebuffer module loading from the load-modules script (it should never have been there in the first place). Agreed. What are the hotpoints in this script as well? Something like this seems inefficient: i="$(/sbin/modprobe -i --show-depends $1 | sed "s#^insmod /lib.*/\(.*\)\.ko.*#\1#g" | sed 's|-|_|g')"

We invoke 3 subprocesses here (modprobe, sed, and sed). Surely the two seds can be combined.

already done locally.

...

See a trend here? So we have the following in one run of load-modules.sh (if we look back at the version packaged with 116):

Good catch with all the seds... but ummm.... how about we NOT worry about optimizing a script we'd like to get rid of? I'm sure start_udev could use some cleanup, want to do that too?

start_udev isn't even used by the curent initscripts though, so thats a different point than this one. Yes, I feel like maybe a short and sweet C program would be better, but for now cleaning some things up makes sense and will probably have an effect on the loading events time, thats all. If I can spend a half hour making it a little better and/or more efficient, I think that is time well spent, even if it is going to eventually be scrapped. -Dan