Early Microcode and initrd.
I notice that the AMD microcode is [1] in the initrd while the intel microcode is not.
Booting with option: initrd=/boot/intel-ucode.img to get the microcode installed early works fine with dracut as it always has.
Is this something that will remain the same as now or does this potentially change under dracut?
thanks
gene
[1] lsinitrd shows:
Early CPIO image ======================================================================== drwxr-xr-x 3 root root 0 May 21 11:48 . -rw-r--r-- 1 root root 2 May 21 11:48 early_cpio drwxr-xr-x 3 root root 0 May 21 11:48 kernel drwxr-xr-x 3 root root 0 May 21 11:48 kernel/x86 drwxr-xr-x 2 root root 0 May 21 11:48 kernel/x86/microcode -rw-r--r-- 1 root root 30546 May 21 11:48 kernel/x86/microcode/AuthenticAMD.bin
Em maio 21, 2019 15:42 Genes Lists via arch-general escreveu:
Early Microcode and initrd.
I notice that the AMD microcode is [1] in the initrd while the intel microcode is not.
Booting with option: initrd=/boot/intel-ucode.img to get the microcode installed early works fine with dracut as it always has.
Is this something that will remain the same as now or does this potentially change under dracut?
This probably needs to be changed. No point in having AMD microcode into the initramfs if you have an intel processor. It's wasted space.
Loading the microcode before will probably always work and should continue to be the desirable solution.
Regards, Giancarlo Razzolini
W dniu 21.05.2019 o 20:48, Giancarlo Razzolini via arch-general pisze:
Em maio 21, 2019 15:42 Genes Lists via arch-general escreveu:
Early Microcode and initrd.
I notice that the AMD microcode is [1] in the initrd while the intel microcode is not.
Booting with option: initrd=/boot/intel-ucode.img to get the microcode installed early works fine with dracut as it always has.
Is this something that will remain the same as now or does this potentially change under dracut?
This probably needs to be changed. No point in having AMD microcode into the initramfs if you have an intel processor. It's wasted space.
Loading the microcode before will probably always work and should continue to be the desirable solution.
Regards, Giancarlo Razzolini
But I believe dracut may handle microcode itself too, at least in fedora it is packed with the core initramfs somehow.
On 5/21/19 2:48 PM, Giancarlo Razzolini wrote:
Em maio 21, 2019 15:42 Genes Lists via arch-general escreveu:
...
Loading the microcode before will probably always work and should continue to be the desirable solution.
..
Funnily enough, while I do have intel-ucode package installed I don't have amd-ucode. The dracut script seems to have found it somewhere I suppose - not sure where yet.
thank you.
gene
On 21/05/2019 20:55, Genes Lists via arch-general wrote:
Funnily enough, while I do have intel-ucode package installed I don't have amd-ucode. The dracut script seems to have found it somewhere I suppose - not sure where yet.
AMD's microcode is in linux-firmware: /usr/lib/firmware/amd-ucode/
On 2019-05-22 14:00, Giovanni Harting via arch-general wrote:
AMD's microcode is in linux-firmware: /usr/lib/firmware/amd-ucode/
Dracut indeed looks in /usr/lib/firmware/{amd,intel}-ucode/ for microcode updates to include in the initramfs. When using a host-only initramfs, only the relevant files are included, otherwise the image is supposed to contain the updates for both AMD and Intel.
Unfortunately this process currently only works for AMD, but not for Intel, since Arch Linux doesn't package the necessary files standalone: they are downloaded by intel-ucode and used to produce a CPIO image, but not added to /usr/lib/firmware/intel-ucode/ separately. If dracut becomes the default initramfs generator, this should be changed so that microcode updates work out of the box with dracut.
Cheers, Jonas
[1] https://github.com/dracutdevs/dracut/blob/8365177644bee107dcb66a71c8a7509900... [2] https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packag...
Em maio 22, 2019 9:25 Jonas Witschel escreveu:
On 2019-05-22 14:00, Giovanni Harting via arch-general wrote:
Unfortunately this process currently only works for AMD, but not for Intel, since Arch Linux doesn't package the necessary files standalone: they are downloaded by intel-ucode and used to produce a CPIO image, but not added to /usr/lib/firmware/intel-ucode/ separately. If dracut becomes the default initramfs generator, this should be changed so that microcode updates work out of the box with dracut.
This is not a huge issue, if the microcode is passed alongside the initramfs on the cmdline, like we have been doing.
We might also change the intel-ucode package, but it is not a requirement and will not prevent the move to dracut.
Regards, Giancarlo Razzolini
On 2019-05-22 17:25, Giancarlo Razzolini via arch-general wrote:
Unfortunately this process currently only works for AMD, but not for Intel, since Arch Linux doesn't package the necessary files standalone: they are downloaded by intel-ucode and used to produce a CPIO image, but not added to /usr/lib/firmware/intel-ucode/ separately. If dracut becomes the default initramfs generator, this should be changed so that microcode updates work out of the box with dracut.
This is not a huge issue, if the microcode is passed alongside the initramfs on the cmdline, like we have been doing.
We might also change the intel-ucode package, but it is not a requirement and will not prevent the move to dracut.
This is certainly not a blocker, no, but would be very nice to have: dracut can bundle kernel, kernel command line, microcode updates and initramfs in a single EFI executable using the "--uefi" parameter. This means setting up booting becomes as simple as
dracut --uefi bootctl install
No further configuration is needed at all because dracut and bootctl implement the Boot Loader Specification [1] (slight caveat: there is currently a small bug if the EFI system partition is mounted to /efi [2]).
Regards, Jonas
[1] https://systemd.io/BOOT_LOADER_SPECIFICATION#type-2-efi-unified-kernel-image... [2] https://github.com/dracutdevs/dracut/pull/574
On 5/22/19 12:22 PM, Jonas Witschel wrote:
On 2019-05-22 17:25, Giancarlo Razzolini via arch-general wrote:
Unfortunately this process currently only works for AMD, but not for Intel, since Arch Linux doesn't package the necessary files standalone: they are downloaded by intel-ucode and used to produce a CPIO image, but not added to /usr/lib/firmware/intel-ucode/ separately. If dracut becomes the default initramfs generator, this should be changed so that microcode updates work out of the box with dracut.
This is not a huge issue, if the microcode is passed alongside the initramfs on the cmdline, like we have been doing.
We might also change the intel-ucode package, but it is not a requirement and will not prevent the move to dracut.
This is certainly not a blocker, no, but would be very nice to have: dracut can bundle kernel, kernel command line, microcode updates and initramfs in a single EFI executable using the "--uefi" parameter. This means setting up booting becomes as simple as
dracut --uefi bootctl install
No further configuration is needed at all because dracut and bootctl implement the Boot Loader Specification [1] (slight caveat: there is currently a small bug if the EFI system partition is mounted to /efi [2]).
Regards, Jonas
[1] https://systemd.io/BOOT_LOADER_SPECIFICATION#type-2-efi-unified-kernel-image... [2] https://github.com/dracutdevs/dracut/pull/574
True, but on the other hand bootctl sucks and you can implement a 2MB efi partition that works everywhere and supports a single unified kernel/microcode/initramfs setup stored on an ext4/btrfs/whatever /boot subdirectory inside your rootfs, full disk encryption optional and supported, no worries about your booted kernel image getting into a mismatch with your /lib/modules directory because you have multiple copies of your kernel flying around and most of them are just staged data that is never used, and so on.
All those benefits just because you did the *simple* thing and didn't use byzantine fstab layouts which uefi+bootctl makes mandatory. :)
(Yes, I am bitter that supporting systemd-boot as a target bootloader means we cannot trust /boot to be a sane filesystem.)
I would rather avoid such bundles at all costs, it sounds like someone saw a problem named bootctl and decided to create even more problems in the name of a solution.
On 2019-05-22 18:41, Eli Schwartz via arch-general wrote:
True, but on the other hand bootctl sucks and you can implement a 2MB efi partition that works everywhere and supports a single unified kernel/microcode/initramfs setup stored on an ext4/btrfs/whatever /boot subdirectory inside your rootfs, full disk encryption optional and supported, no worries about your booted kernel image getting into a mismatch with your /lib/modules directory because you have multiple copies of your kernel flying around and most of them are just staged data that is never used, and so on.
systemd-boot was just one example where such a setup might be helpful, of course you don't have to use it: something like
dracut --uefi $ESP/EFI/Arch/arch-linux.efi efibootmgr --create --disk ... --label "Arch Linux" --loader "EFI\Arch\arch-linux.efi" --verbose
works equally well.
I would rather avoid such bundles at all costs, it sounds like someone saw a problem named bootctl and decided to create even more problems in the name of a solution.
IMHO such bundles are more or less the only way to go if you want to have an encrypted system in combination with Secure Boot: you can bundle everything in a single application, sign it with your keys and put it on the EFI system partition. Currently I use sbupdate-git for this purpose, if we switch to dracut anyway, I could remove sbupdate and replace it by a simple hook that signs the EFI executable generated by dracut.
The only alternative I am aware of for such a configuration is using GRUB for its encrypted /boot support, but that has some drawbacks as well (is LUKS2 supported by now?) and is much harder to customise: for example, I store the disk encryption key in a Trusted Platform Module (TPM). Retrieving it from the TPM is easy using a custom script run as a mkinitcpio hook/dracut module, writing a GRUB module is much less fun...
Cheers, Jonas
On 05/22/2019 11:41 AM, Eli Schwartz via arch-general wrote:
I would rather avoid such bundles at all costs, it sounds like someone saw a problem named bootctl and decided to create even more problems in the name of a solution.
I've always respected the Arch KISS approach. The further we stay into the ... what was the word? ... "byzantine", the further away from KISS Arch migrates.
But, somehow, those charged with the stewardship of Arch seem to always make solid choices for the distro going forward.
dracut --uefi
This seems to fail for me: $ sudo dracut --no-early-microcode --uefi /boot/EFI/Linux/arch-linux.efi dracut: Executing: /usr/bin/dracut --no-early-microcode --uefi /boot/EFI/Linux/arch-linux.efi /usr/bin/dracut: line 1063: arch: command not found /usr/bin/dracut: line 1069: arch: command not found dracut: Architecture '' not supported to create a UEFI executable
any ideas why??
dracut 049-3 on an Arch [testing] VM
On 2019-05-27 16:18, Damjan Georgievski via arch-general wrote:> This seems to fail for me:
$ sudo dracut --no-early-microcode --uefi /boot/EFI/Linux/arch-linux.efi dracut: Executing: /usr/bin/dracut --no-early-microcode --uefi /boot/EFI/Linux/arch-linux.efi /usr/bin/dracut: line 1063: arch: command not found /usr/bin/dracut: line 1069: arch: command not found dracut: Architecture '' not supported to create a UEFI executable
This is a bug in dracut, see
https://github.com/dracutdevs/dracut/pull/573
for a fix.
Regards, Jonas
Em maio 27, 2019 11:18 Damjan Georgievski via arch-general escreveu:
dracut --uefi
This seems to fail for me: $ sudo dracut --no-early-microcode --uefi /boot/EFI/Linux/arch-linux.efi dracut: Executing: /usr/bin/dracut --no-early-microcode --uefi /boot/EFI/Linux/arch-linux.efi /usr/bin/dracut: line 1063: arch: command not found /usr/bin/dracut: line 1069: arch: command not found dracut: Architecture '' not supported to create a UEFI executable
any ideas why??
dracut 049-3 on an Arch [testing] VM
There are a few more instances where arch must be replaced with uname -m.
I'll deploy a version of dracut with that patch later:
https://github.com/dracutdevs/dracut/pull/573
Regards, Giancarlo Razzolini
dracut 049-3 on an Arch [testing] VM
There are a few more instances where arch must be replaced with uname -m.
I'll deploy a version of dracut with that patch later:
Thanks, I've also noticed another issue about the uefi stub and sent a PR: https://github.com/dracutdevs/dracut/pull/575
Em maio 27, 2019 13:45 Damjan Georgievski escreveu:
Thanks, I've also noticed another issue about the uefi stub and sent a PR: https://github.com/dracutdevs/dracut/pull/575
Thank you all for testing and also submitting patches to make it work better not just on Arch, but any other distro too.
Regards, Giancarlo Razzolini
Hi
I've come across a possible issue which related to usb keyboard/mouse device.
I notice that booting using dracut initramfs with kernel 5.2-rc2 that as soon as it boots it shows on the console: stopping job udev Kernel Device Manager with red spinner.
This times out and boot continues fine - once the system has booted I check the boot log and included interesting part of logs below. Keyboard and mouse are both working and the relevant hid modules are loaded fine - and xinput shows all looking normal.
Interesingly, using mkinitcpio initrd with same 5.2 kernel- the system comes up with no keyboard or mouse (it uses logitech wireless kbd/mouse). The hid_logitech modules seem not to have been loaded at all. So something in the newer kernel is leading to this issue but dracut manages to pull through anyway while mkinitcpio does not.
Obviously anyone experiencing this might be quite taken aback.
Also, everythign works fine with 5.1.5 kernel with both dracut and mkinitcpio. the hid modules are all fine for that test case.
udev is a bit of a mystery to me - but thought I'd best mention the problem.
gene
--- boot journal with 5.2 with dracut --------
saplinux systemd[1]: systemd-udev-trigger.service: Succeeded. saplinux systemd[1]: Stopped udev Coldplug all Devices. saplinux systemd[1]: dracut-pre-trigger.service: Succeeded. saplinux systemd[1]: Stopped dracut pre-trigger hook. saplinux systemd[1]: Stopping udev Kernel Device Manager...
...
saplinux systemd-udevd[386]: Giving up waiting for workers to finish. saplinux systemd-udevd[386]: Event loop failed: Connection timed out saplinux systemd[1]: systemd-udevd.service: Main process exited, code=exited, status=1/F> saplinux systemd[1]: systemd-udevd.service: Killing process 459 (systemd-udevd) with sig> saplinux systemd[1]: systemd-udevd.service: Killing process 480 (systemd-udevd) with sig> saplinux systemd[1]: systemd-udevd.service: Killing process 482 (systemd-udevd) with sig>
saplinux kernel: input: Logitech USB Receiver as /devices/pci0000:00/0000:00:14.0/usb1/1> saplinux kernel: hid-generic 0003:046D:C52B.0001: input,hidraw0: USB HID v1.11 Keyboard > saplinux kernel: input: Logitech USB Receiver Mouse as /devices/pci0000:00/0000:00:14.0/> saplinux kernel: input: Logitech USB Receiver Consumer Control as /devices/pci0000:00/00> saplinux kernel: input: Logitech USB Receiver System Control as /devices/pci0000:00/0000> saplinux kernel: hid-generic 0003:046D:C52B.0002: input,hiddev0,hidraw1: USB HID v1.11 M> saplinux kernel: hid-generic 0003:046D:C52B.0003: hiddev1,hidraw2: USB HID v1.11 Device > saplinux kernel: usbcore: registered new interface driver usbhid saplinux kernel: usbhid: USB HID core driver saplinux systemd[1]: systemd-udevd.service: Failed with result 'exit-code'.
saplinux systemd[1]: Stopped udev Kernel Device Manager. saplinux systemd[1]: systemd-tmpfiles-setup-dev.service: Succeeded.
On 5/27/19 2:07 PM, Genes Lists via arch-general wrote:
Hi
I've come across a possible issue which related to usb keyboard/mouse device.
Quick follow up. I have this issue on 2 machines I've tested.
(i) vendor_id : GenuineIntel cpu family : 6 model : 78 model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz stepping : 3
(ii) vendor_id : GenuineIntel cpu family : 6 model : 158 model name : Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz stepping : 9
gene
On 5/27/19 2:07 PM, Genes Lists via arch-general wrote:
Likely the same issue https://lkml.org/lkml/2019/5/28/812
Thanks,
gene
FYI - I have switched my custom kernel builds to use dracut by default now - tested on 8 machines with no issues. Very supportive of switching to dracut.
thank you.
gene
On 6/4/19 12:07 PM, Genes Lists via arch-general wrote:
FYI - I have switched my custom kernel builds to use dracut by default
Ug take it back - i have one problem - a big problem.
On a computer using md RAID-6 the raid disks are not assembled at boot - and the boot fails. Works fine switching back to mkinitcpio.
Is there anything special we need to do with dracut to make md raid work at boot? The raid array is for data only not root which is a separate disk.
dracut docs suggest nothing is needed to assemble raid arrays at boot. Perhaps this is incorrect?
Thanks.
On 6/4/19 4:53 PM, Genes Lists via arch-general wrote:
Forgot to post the boot the boot log which shows the problem occurs after switching the pivot root:
... saplinux kernel: md/raid:md127: raid level 6 active with 6 out of 6 devices, algorithm 2 saplinux kernel: md127: detected capacity change from 0 to 12001828929536 saplinux systemd[1]: Started dracut initqueue hook. ... saplinux systemd[1]: Switching root. ... kernel: EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: data=ordered systemd[1]: dev-md0.device: Job dev-md0.device/start timed out. systemd[1]: Timed out waiting for device /dev/md0. systemd[1]: Dependency failed for File System Check on /dev/md0. systemd[1]: Dependency failed for /mnt/md0. systemd[1]: Dependency failed for /srv/nfs4/install. systemd[1]: Dependency failed for Local File Systems. systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'. systemd[1]: local-fs.target: Triggering OnFailure= dependencies. ...
Speculation - this may be related to the kernel config which has CONFIG_DEFAULT_HOSTNAME="archlinux" for arch kernels and i use a different name (saplinux)
In either case, in the boot logs the hostame shown before the pivot root is that default as compiled into thekernel - but after the switch root the actual hostname for the server is used.
Since raid uses the real hostname (cat /proc/mdstat for example shows the real hostname on the raid server) - that since the hostnames doesn't match. Before the switchroot the raid is assembled but at md127 instead of md0 - after the pivot root all hell breaks loose as there is no md0 only md127.
Clearly dracut must be used for raid - so whats the right way to do this?
thanks.
Em junho 4, 2019 18:08 Genes Lists via arch-general escreveu:
On 6/4/19 4:53 PM, Genes Lists via arch-general wrote:
Forgot to post the boot the boot log which shows the problem occurs after switching the pivot root:
... saplinux kernel: md/raid:md127: raid level 6 active with 6 out of 6 devices, algorithm 2 saplinux kernel: md127: detected capacity change from 0 to 12001828929536 saplinux systemd[1]: Started dracut initqueue hook. ... saplinux systemd[1]: Switching root. ... kernel: EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: data=ordered systemd[1]: dev-md0.device: Job dev-md0.device/start timed out. systemd[1]: Timed out waiting for device /dev/md0. systemd[1]: Dependency failed for File System Check on /dev/md0. systemd[1]: Dependency failed for /mnt/md0. systemd[1]: Dependency failed for /srv/nfs4/install. systemd[1]: Dependency failed for Local File Systems. systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'. systemd[1]: local-fs.target: Triggering OnFailure= dependencies. ...
Please, report all issues on bugs.archlinux.org with as much information as possible. As far as I know dracut supports raid. Perhaps this might even be an upstream issue.
Regards, Giancarlo Razzolini
Giancarlo Razzolini
Found this (very) old thread [1] - and tried the suggesion - removed name= from mdadm.conf, rebuilt the initrd and rebooted but had same problem. Booted from mkinitrd and still works. May be i need to learn more about raid
Will need to get back to this.
I now understand why this works with mkinitrd - it's because this approach does not assemble the array until -after- the pivot root - and the hostname is now the real hostname and this matches what's in the raid superblock.
dracut attempts to assemble the array(s) before - in this case the hostname is the fake hostname.
In neither case is /etc/mdadm.conf included - tho nkinitrd approach its not needed.
I need to do more work ... perhaps test with the --madadmconf flag when building the initrd is next step.
gene
On 6/15/19 5:12 PM, Genes Lists via arch-general wrote: ...
I need to do more work ... perhaps test with the --madadmconf flag when building the initrd is next step.
I confirm that building the initrd with --mdadmconf fixes the problem assembling the raid disks using dracut - this happens fine now.
In my case the /etc/mdadm.conf has RRAY /dev/md0 metadata=1.2 name=xxx ...
where xxx is the hostname which is also in the RAID superblock.
I would therefore recommend that the pacman hook file which is used to build the initrd use the --mdadmconf option.
I will note that the file is not included even with -H or --include /etc/mdadm.conf.
best
gene
arch-general@lists.archlinux.org