[arch-projects] [mkinitcpio][RFC] a better fallback image?

Tom Gundersen

25 Jul 2012 25 Jul '12

9:21 p.m.

Dave, Tobias, Thomas, I just managed to hose one of my machines. It was mainly due to an unfortunate series of events, but I have a suggestion which would have avoided this situation and might beneficial in general. The way things are now, I think the fallback image mkinitcpio generates by default is not very useful. In my experience the fallback image is broken exactly when the regular image is broken. I expect the reason for this is that the autodetection of modules is very good these days, so _that_ is never the problem. When there is a problem, which happens every once in a while, it is either due to user error (i.e. configuration errors), or mkinitcpio not being ran at all (and hence having the modules of the previous kernel (which is what happened to me today). It seems to me that a more useful fallback image, would be one that is generated at compile-time, rather than at install-time, and shipped as a part of the kernel package. This would avoid user errors, and mkinitcpio run-time problems. This fallback image should contain the widest sets of hooks and modules so that it should work on any hardware and any setup (at least to the extent possible with our current hooks). Any thoughts? Cheers, Tom

Show replies by date

Dave Reisner

25 Jul 25 Jul

9:43 p.m.

On Wed, Jul 25, 2012 at 11:21:10PM +0200, Tom Gundersen wrote:

...

Dave, Tobias, Thomas,

I just managed to hose one of my machines. It was mainly due to an unfortunate series of events, but I have a suggestion which would have avoided this situation and might beneficial in general.

Oops. ;P What did you break?

...

The way things are now, I think the fallback image mkinitcpio generates by default is not very useful. In my experience the fallback image is broken exactly when the regular image is broken. I expect the reason for this is that the autodetection of modules is very good these days, so _that_ is never the problem.

When there is a problem, which happens every once in a while, it is either due to user error (i.e. configuration errors), or mkinitcpio not being ran at all (and hence having the modules of the previous kernel (which is what happened to me today).

It seems to me that a more useful fallback image, would be one that is generated at compile-time, rather than at install-time, and shipped as a part of the kernel package. This would avoid user errors, and mkinitcpio run-time problems.

This fallback image should contain the widest sets of hooks and modules so that it should work on any hardware and any setup (at least to the extent possible with our current hooks).

Any thoughts?

Cheers,

Tom

You've mentioned this idea before, and more recently, so has Jan. I've slowly and quietly been adding support to mkinitcpio to make this less painful to do. An xz-compressed image with the usb and fw hooks still weighs in at under 10MiB, so I have no real concerns about this being problematic for users with separate /boot partitions. That said, I'm wondering how it would be possible to generate this at compile time. I'm not really sure how this would be possible. Is there any objection to just adding another preset (maybe related to how you fubar'd your install this time?). As an alternative/addition, which has also been brought up before, why don't we build in the most basic of modules? I'll bet we can cover at least 50% of the use cases by picking some choice pata/sata modules (e.g. ahci, ata_piix, pata_jmicron, sd_mod, ext4) and compiling them in staticly. It, of course, doesn't cover folks with non-trivial setups, but it provides a bulletproof bootstrap for a lot of people. d

Tom Gundersen

10:01 p.m.

On Wed, Jul 25, 2012 at 11:43 PM, Dave Reisner wrote:

...

On Wed, Jul 25, 2012 at 11:21:10PM +0200, Tom Gundersen wrote:

...
Dave, Tobias, Thomas,

I just managed to hose one of my machines. It was mainly due to an unfortunate series of events, but I have a suggestion which would have avoided this situation and might beneficial in general.

Oops. ;P

What did you break?

Had a hard crash during a pacman. Trying to fix things from a rescue shell I installed the kernel and as I didn't have udev running, mkinitcpio claimed /dev was not mounted (at least I think that's why that happened) and refused to run. I then started udevd manually, and the machine hung again. Result: on next restart I had a 3.5 kernel, but the modules in both my initramfs' were 3.2 (or something like that). So without an ext4 I was sad.

...

You've mentioned this idea before, and more recently, so has Jan. I've slowly and quietly been adding support to mkinitcpio to make this less painful to do. An xz-compressed image with the usb and fw hooks still weighs in at under 10MiB, so I have no real concerns about this being problematic for users with separate /boot partitions.

Nice!

...

That said, I'm wondering how it would be possible to generate this at compile time. I'm not really sure how this would be possible. Is there any objection to just adding another preset (maybe related to how you fubar'd your install this time?).

The underlying problem is that in principle you might (as I did) upgrade your kernel without running mkinitcpio. We have seen pacman issues causing this, simple crashes as what I had might cause it, or a broken mkinitcpio.conf might do it (I guess the last one could be worked around by adding a special preset that does not care about user config).

...

As an alternative/addition, which has also been brought up before, why don't we build in the most basic of modules? I'll bet we can cover at least 50% of the use cases by picking some choice pata/sata modules (e.g. ahci, ata_piix, pata_jmicron, sd_mod, ext4) and compiling them in staticly. It, of course, doesn't cover folks with non-trivial setups, but it provides a bulletproof bootstrap for a lot of people.

I really think this would be a good idea. I wanted to make some additions to pierres pkgstats stuff so we could have an idea of how large percentage of our users would be covered by the modules you propose. I expect the vast majority would. Cheers, Tom

Jan Steffens

9:46 p.m.

On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen wrote:

...

Dave, Tobias, Thomas,

I just managed to hose one of my machines. It was mainly due to an unfortunate series of events, but I have a suggestion which would have avoided this situation and might beneficial in general.

The way things are now, I think the fallback image mkinitcpio generates by default is not very useful. In my experience the fallback image is broken exactly when the regular image is broken. I expect the reason for this is that the autodetection of modules is very good these days, so _that_ is never the problem.

Right. AFAIK the only remaining use is booting after hardware changes.

...

When there is a problem, which happens every once in a while, it is either due to user error (i.e. configuration errors), or mkinitcpio not being ran at all (and hence having the modules of the previous kernel (which is what happened to me today).

It seems to me that a more useful fallback image, would be one that is generated at compile-time, rather than at install-time, and shipped as a part of the kernel package. This would avoid user errors, and mkinitcpio run-time problems.

This fallback image should contain the widest sets of hooks and modules so that it should work on any hardware and any setup (at least to the extent possible with our current hooks).

Any thoughts?

Cheers,

Tom

I think there are a few corner cases it wouldn't boot, but it sounds generally useful. Wouldn't this add quite a lot of weight to our kernel packages? Making the modules built-in instead of shipping them twice would probably help with this. I'm still thinking about that "rescue" initcpio with a bigger toolset (pacman, rsync, et cetera), but that's another issue.

Tom Gundersen

10:06 p.m.

On Wed, Jul 25, 2012 at 11:46 PM, Jan Steffens wrote:

...

I think there are a few corner cases it wouldn't boot, but it sounds generally useful.

Probably, not covering 100% would be ok, but maybe we could do 95% ;-)

...

Wouldn't this add quite a lot of weight to our kernel packages? Making the modules built-in instead of shipping them twice would probably help with this.

Hm, I think some extra space would be worth it for increased reliability. We could easily make this a split package though, so people don't have to install it (btw, the rescue image should (imo) contain all the (relevant) modules, whereas what Dave suggested was to compile in a few selected ones to cover most cases).

...

I'm still thinking about that "rescue" initcpio with a bigger toolset (pacman, rsync, et cetera), but that's another issue.

By adding mkinitcpio hooks to the respective packages over time, the fallback image could be improved gradually to allow more advanced rescuing. -t

Tom Gundersen

29 Jul 29 Jul

10:18 p.m.

On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen wrote:

...

It seems to me that a more useful fallback image, would be one that is generated at compile-time, rather than at install-time, and shipped as a part of the kernel package. This would avoid user errors, and mkinitcpio run-time problems.

This fallback image should contain the widest sets of hooks and modules so that it should work on any hardware and any setup (at least to the extent possible with our current hooks).

I put a PKGBUILD up on AUR[0]. Rather than making it part of the kernel package, I made it separate (at Thomas' request). At least this should make it easy to test without having to rebuild kernels. At the moment it simply includes all the hooks I could think of. If, in the future, we would like to add more features to it (e.g. pacman, arch-install-scripts or better networking tools), we should add mkinitcpio hooks to the relevant pacakges. Any thoughts? Cheers, Tom [0]: https://aur.archlinux.org/packages.php?ID=61313

Gerardo Exequiel Pozzi

11:08 p.m.

On 07/29/2012 07:18 PM, Tom Gundersen wrote:

...

On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen wrote:

...
It seems to me that a more useful fallback image, would be one that is generated at compile-time, rather than at install-time, and shipped as a part of the kernel package. This would avoid user errors, and mkinitcpio run-time problems.

This fallback image should contain the widest sets of hooks and modules so that it should work on any hardware and any setup (at least to the extent possible with our current hooks). I put a PKGBUILD up on AUR[0]. Rather than making it part of the kernel package, I made it separate (at Thomas' request). At least this should make it easy to test without having to rebuild kernels.

At the moment it simply includes all the hooks I could think of. If, in the future, we would like to add more features to it (e.g. pacman, arch-install-scripts or better networking tools), we should add mkinitcpio hooks to the relevant pacakges.

Any thoughts?

Cheers,

Tom

[0]: https://aur.archlinux.org/packages.php?ID=61313

missing shutdown hook? -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Tom Gundersen

11:20 p.m.

On Mon, Jul 30, 2012 at 1:08 AM, Gerardo Exequiel Pozzi wrote:

...

On 07/29/2012 07:18 PM, Tom Gundersen wrote:

...
On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen wrote:

...
It seems to me that a more useful fallback image, would be one that is generated at compile-time, rather than at install-time, and shipped as a part of the kernel package. This would avoid user errors, and mkinitcpio run-time problems.

This fallback image should contain the widest sets of hooks and modules so that it should work on any hardware and any setup (at least to the extent possible with our current hooks).

I put a PKGBUILD up on AUR[0]. Rather than making it part of the kernel package, I made it separate (at Thomas' request). At least this should make it easy to test without having to rebuild kernels.

At the moment it simply includes all the hooks I could think of. If, in the future, we would like to add more features to it (e.g. pacman, arch-install-scripts or better networking tools), we should add mkinitcpio hooks to the relevant pacakges.

Any thoughts?

Cheers,

Tom

[0]: https://aur.archlinux.org/packages.php?ID=61313

missing shutdown hook?

Yeah, good point. -t

4289

Age (days ago)

4293

Last active (days ago)

List overview

Download

7 comments

4 participants

participants (4)

Dave Reisner
Gerardo Exequiel Pozzi
Jan Steffens
Tom Gundersen