[arch-dev-public] New kernel packages and mkinitcpio hooks
Hi All, As most of you have noticed already, the kernel packages had some recent changes where they do not install the kernel to /boot, as well as they do not have any kmod [0] nor mkinitcpio hooks anymore. Neither they do install mkinitcpio presets [1]. These changes were made after a lot of discussions between myself, Jen and the maintainers of the other kernels as well. We have spent a lot of time to arrive at this current format. [2] Kernels and presets are not installed and generated by mkinitcpio alpm hooks and scripts. All kernels that have a pkgbase entry file [3] will be processed by these scripts. We have changed all our officials kernels. Also, kernel removals are handled by mkinitcpio as well. We want to make the boot process more flexible, while also keeping it backwards compatible by default. There are a few changes I have planned for the next mkinitcpio release [4] that will help even further to achieve this. I have been working on hooks for dracut to install the kernels to /boot, as mkinitcpio does, and also create a simple configuration file for each kernel. Right now dracut does not conflict with mkinitcpio, and it is possible to have both installed at the same time on a system. The hooks I'm writing for dracut will, however, check if mkinitcpio is installed and won't run if it is. We might eventually conflict them if we decide dracut is mature enough for usage with Arch Linux. As a side note, I have been using it for the past 8 months without any issues. Regards, Giancarlo Razzolini [0] https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/kmod&id=9e2016961cc34830957ceee0593a80bfa9e05ba9 [1] https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=43c5745a17e2ebe413a7140d4ef9326e26d6cb20 [2] https://github.com/archlinux/mkinitcpio/pull/7 [3] https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=f1c97a49a09b0fd8d7d1c3f6e8e635ef45974373 [4] https://github.com/archlinux/mkinitcpio/issues
On Sun, 2019-11-10 at 18:41 -0300, Giancarlo Razzolini via arch-dev-public wrote:
These changes were made after a lot of discussions between myself, Jen and the maintainers of the other kernels as well. We have spent a lot of time to arrive at this current format. [2]
Hello, Nice to see that moving forward. It is a smart to move pacman installed kernel out of /boot. Why do you rely on mkinitcpio (or later dracut) to install them in /boot instead of the systemd kernel-install logic? Regards, Sébastien "Seblu" Luttringer
Em novembro 10, 2019 23:54 Sébastien Luttringer escreveu:
Hello,
Nice to see that moving forward. It is a smart to move pacman installed kernel out of /boot. Why do you rely on mkinitcpio (or later dracut) to install them in /boot instead of the systemd kernel-install logic?
Hi Seblu, This has been discussed a bit on the dracut thread, as well on some other threads over time. I *personally* don't like the complexity of kernel-install that much. I'm not saying we will never use it, just that I don't see the case for using it *right now*. Things are a bit more flexible now, and anyone can override the hooks/scripts to install the kernel whatever way they want. Also, I plan more changes to mkinitcpio to make it even more flexible. Also, dracut is not set in stone and right now it's an experimental/secondary thing. mkinitcpio remains the default initramfs generator for Arch. I'll try to make sure it gets the changes, but there are some that will depend on upstream taking them, as well. Regards, Giancarlo Razzolini
On 11/10/19 9:54 PM, Sébastien Luttringer via arch-dev-public wrote:
Nice to see that moving forward. It is a smart to move pacman installed kernel out of /boot. Why do you rely on mkinitcpio (or later dracut) to install them in /boot instead of the systemd kernel-install logic?
As I said above, the documentation for kernel-install is pretty clear on its expected usage. It's a one-stop shop, it executes mkinitcpio or whatever other plugin you've installed for generating an initramfs, and it's totally 100% independent of the mkinitcpio hook. Interested parties who would like to see kernel-install in use should mask the *standalone* mkinitcpio hook, not extend it to also call kernel-install. You'd be replacing the current infrastructure from scratch, not adding kernel-install to the mix. On compatibility: kernel-install documentation mandates that your boot partition will be autodetected based on filesystem structure, with the candidate options being: - /efi - /boot/efi - /boot and mandates that the subdirectory $MACHINE_ID/$KERNEL_VERSION/ be used for all installation of relevant files. kernel-install will additionally autogenerate a systemd-boot loader/entries/*.conf with a boot menu option it has determined is in the best interest of the user. (Think grub-mkconfig, but for systemd-boot.) Also, yes, kernel-install mandates the use of systemd-boot. Your kernel is installed into a filesystem directory structure which is not supported by grub-mkconfig's autodetection, and which uses continually changing paths which encode the kernel version. There is no unversioned kernel installed, so you cannot write a bootloader config once and then let that keep on being used. Therefore you must autogenerate a bootloader configuration *somehow*, and it is either use the systemd-boot one which kernel-install installs, or write your own generator script. Meanwhile, mkinitcpio's presets and the kernel file which was historically installed by the linux package install directly to: /boot/vmlinuz-linux /boot/initramfs-linux.img Under no circumstances can we unilaterally remove mkinitcpio presets and switch to kernel-install without a mandatory manual intervention for all (or most) archlinux users. It seems more polite to keep the status quo and work on making this a user-configurable choice (it really seems like they can coexist in the wider archlinux community just fine), and perhaps giving preferential status for ${bootloader_of_the_day} in the installation guide. -- Eli Schwartz Bug Wrangler and Trusted User
On Mon, 2019-11-11 at 15:41 -0500, Eli Schwartz via arch-dev-public wrote:
On 11/10/19 9:54 PM, Sébastien Luttringer via arch-dev-public wrote: As I said above, the documentation for kernel-install is pretty clear on its expected usage. It's a one-stop shop, it executes mkinitcpio or whatever other plugin you've installed for generating an initramfs, and it's totally 100% independent of the mkinitcpio hook.
Hi Eli, Let me first agree on the clarity of kernel-install documentation. Its goal is well summed in the man page NAME section: "Add and remove kernel and initramfs images to and from /boot.
...
Also, yes, kernel-install mandates the use of systemd-boot. No, kernel-install doesn't mandate use of systemd-boot. It's a modular framework to make your machine bootable. The default plugin, install kernel for systemd-boot which is different.
Since 2014, I use kernel-install[1] to manage kernel upgrades on over than 300 Arch servers. There is regular Arch kernels, custom versioned kernels, some are booted with grub (uefi or not) others with systemd-boot.
Meanwhile, mkinitcpio's presets and the kernel file which was historically installed by the linux package install directly to:
/boot/vmlinuz-linux /boot/initramfs-linux.img An Arch plugin of few lines could easily put the files in our historical location. This are just examples [2][3].
Under no circumstances can we unilaterally remove mkinitcpio presets and switch to kernel-install without a mandatory manual intervention for all (or most) archlinux users. We can easily switch to kernel-install, without user intervention, as it was done with pacman hooks. Switching from mkinitcpio is another topic.
kernel-install runs a collection of scripts (hooks), with the systemd overriding logic. There is no huge difference with the pacman hooks, except the logic is cross distro, provided by systemd, and not tied to pacman. Improvements can be shared. What it makes me prefer systemd kernel-install than pacman hooks logic. Regards, [1] https://git.seblu.net/archlinux/kernel-install-poc/ [2] https://git.seblu.net/archlinux/kernel-install-poc/blob/master/90-loaderentr... [3] https://git.seblu.net/archlinux/kernel-install-poc/blob/master/50-mkinitcpio... -- Sébastien Luttringer
Em novembro 12, 2019 15:24 Sébastien Luttringer via arch-dev-public escreveu:
kernel-install runs a collection of scripts (hooks), with the systemd overriding logic. There is no huge difference with the pacman hooks, except the logic is cross distro, provided by systemd, and not tied to pacman. Improvements can be shared. What it makes me prefer systemd kernel-install than pacman hooks logic.
I just disagree with the part where it somehow implies that improvements can't be made to the current hooks. The main reason I started mirroring the mkinitcpio on github is so we can receive contributions more easily. Using kernel-install, on Arch, is also pacman dependent, because you trigger it using pacman hooks. The only difference is what do you call. As I've said, even though I *personally* don't like it too much, nothing prevents us from using it in the future. Now that the kernel package doesn't install the kernel anymore, we can even provide more than one option for the user to install it wherever they want. So far we have only mkinitcpio hooks for doing that, soon dracut hooks as well. We can have a single hook in the future that reads from a configuration file which method the user wants. Regards, Giancarlo Razzolini
On 11/12/19 1:24 PM, Sébastien Luttringer wrote:
On Mon, 2019-11-11 at 15:41 -0500, Eli Schwartz via arch-dev-public wrote:
On 11/10/19 9:54 PM, Sébastien Luttringer via arch-dev-public wrote: As I said above, the documentation for kernel-install is pretty clear on its expected usage. It's a one-stop shop, it executes mkinitcpio or whatever other plugin you've installed for generating an initramfs, and it's totally 100% independent of the mkinitcpio hook.
Hi Eli,
Let me first agree on the clarity of kernel-install documentation. Its goal is well summed in the man page NAME section: "Add and remove kernel and initramfs images to and from /boot.
...
Also, yes, kernel-install mandates the use of systemd-boot. No, kernel-install doesn't mandate use of systemd-boot. It's a modular framework to make your machine bootable. The default plugin, install kernel for systemd-boot which is different.
The script /usr/bin/kernel-install mandates systemd-boot, because it detects the directory to install the kernel to by looking for a systemd-boot config. I guess if you have both then it will still work. If /boot/efi is your EFI partition mountpoint, but it is a 2mb partition (it is on my system, because it contains exactly one file, that being grubx64.efi), then this will be detected as the root install location of your kernels. My kernels are installed to a btrfs partition that does not conform to systemd-boot's logic. This is supported by both grub and rEFInd.
Since 2014, I use kernel-install[1] to manage kernel upgrades on over than 300 Arch servers. There is regular Arch kernels, custom versioned kernels, some are booted with grub (uefi or not) others with systemd-boot.
Meanwhile, mkinitcpio's presets and the kernel file which was historically installed by the linux package install directly to:
/boot/vmlinuz-linux /boot/initramfs-linux.img An Arch plugin of few lines could easily put the files in our historical location. This are just examples [2][3].
Under no circumstances can we unilaterally remove mkinitcpio presets and switch to kernel-install without a mandatory manual intervention for all (or most) archlinux users. We can easily switch to kernel-install, without user intervention, as it was done with pacman hooks. Switching from mkinitcpio is another topic.
The kernel-install hook you linked to seems to create two copies of each installed kernel, in a filesystem that is very commonly quite small in size. This is fine if you opt into it, but I don't consider this to be "no manual intervention" grade quality. It also runs mkinitcpio manually, without respecting mkinitcpio.d presets, so if anyone relies on something configured into those presets, then automatically switching from using mkinitcpio via the current pacman hook (which thus far has only moved from one package to another, but still does the same thing once you've migrated to the latest mkinitcpio package), to using mkinitcpio via kernel-install, represents a *behavior change*. Having kernel-install do the potential upgrade path of: - move over the kernel - execute the mkinitcpio preset - copy /boot/initramfs-linux.img to its special directory - run grub-mkconfig to configure a one-to-one copy of the non kernel-install initramfs/kernel for booting, when the originals still work flawlessly despite not having been cp'ed into a kernel-install specific directory means that you're moving lots of stuff into kernel-install, then not actually using it. It means using kernel-install for the sake of using kernel-install, rather than because it fundamentally handles performing a single 'cp' command better than a custom hook. It's a lot of effort to go to if you're not actually going to buy into kernel-install's intended use case, which is "we hardcode systemd-boot into the non-configurable part of the tool". It also depends on running grub-mkconfig. I will not install or update to any version of any package that automatically modifies my grub.cfg without my consent, as it absolutely will break my boot. I never used grub-mkconfig, because I consider it conceptually broken by design (it tries to be way too clever for its own good, doesn't handle corner cases properly, and is a nightmare to debug), haven't configured its obtuse configuration file to work with my system, etc. I've written quite a bit of wiki documentation for using grub, from scratch, because none of the current documentation is any good, and all of it tells you to use grub-mkconfig (which as you may have noticed at this point, I have very biased and emotional feelings about): https://wiki.archlinux.org/index.php/User:Eschwartz/Grub#Configuration Moving beyond grub. Should we also provide kernel-install plugins for refind, syslinux, clover... do we want to support bootloaders that might need to be generated differently on Redesigning the boot stage of archlinux to regularly autogenerate boot configs is something best left to user choice, for example, people choosing to use something like your kernel-install hooks. :) This gives people the option to choose the kernel-autoinstaller provider that integrates with their chosen setup. Just because it's been proven to work in one place that it was set up to work with, doesn't mean it is 100% guaranteed to work absolutely everywhere without any configuration at all, across many different bootloaders and the multiple different "standard" efi directories and infinite number of arbitrary efi directories that people have chosen "just because I can".
kernel-install runs a collection of scripts (hooks), with the systemd overriding logic. There is no huge difference with the pacman hooks, except the logic is cross distro, provided by systemd, and not tied to pacman. Improvements can be shared. What it makes me prefer systemd kernel-install than pacman hooks logic.
The pacman version works today, and /usr/share/libalpm/scripts/mkinitcpio-install is just as cross distro and not tied to pacman as kernel-install is. The pacman-specific part is /usr/share/libalpm/hooks/90-mkinitcpio-install.hook, which simply ties to pacman in order to execute mkinitcpio-install and give it a list of kernel files on stdin. There's nothing stopping people from using it anywhere they want. It's a dead-simple system, which lots of people have historically used and been happy with. There's no reason it cannot coexist with kernel-install, and there's no reason people cannot continue to work on it, if they like the way it works. ... tl;dr I think it's perfectly reasonable to consider kernel-install to be an alternative to the traditional generic method, and warmly encourage people who like kernel-install to finally write documentation and wiki guides for using it, rather than proposing to declare it the sole supported boot method. It looks like you have a lot of great ideas for doing that! Maybe you could help get us there? Do you think your kernel-install plugins would make for a good package to add to the official repos in order to implement your example boot configuration? This could help us get ready for a day in the future when Giancarlo's suggestion "We can have a single hook in the future that reads from a configuration file which method the user wants" can be implemented. -- Eli Schwartz Bug Wrangler and Trusted User
participants (3)
-
Eli Schwartz
-
Giancarlo Razzolini
-
Sébastien Luttringer