[arch-general] Updating linux-lts initcpios... error: command failed to execute correctly
Hi, This morning's `pacman -Su' had a problem building the fallback initcpio image for 4.14.79-1-lts. Yesterday's /var/log/pacman.log shows [2018-11-06 10:45] [ALPM] upgraded linux-lts (4.14.78-1 -> 4.14.79-1) [2018-11-06 10:45] [ALPM] running '60-linux-lts.hook'... [2018-11-06 10:45] [ALPM] running '90-linux-lts.hook'... that all went well. Today, package `linux-lts' wasn't upgraded but 90-linux-lts.hook was triggered. 60-linux-lts.hook wasn't. pacman's TTY output complained `error: command failed to execute correctly' and the list of `Running' build hooks didn't complete. The same steps for package `linux' immediately after didn't have the same problem. :: Running post-transaction hooks... ( 1/12) Updating linux-lts initcpios... ==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'default' -> -k /boot/vmlinuz-linux-lts -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-lts.img ==> Starting build: 4.14.79-1-lts -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [autodetect] -> Running build hook: [modconf] -> Running build hook: [block] -> Running build hook: [filesystems] -> Running build hook: [keyboard] -> Running build hook: [fsck] ==> Generating module dependencies ==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-lts.img ==> Image generation successful ==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'fallback' -> -k /boot/vmlinuz-linux-lts -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-lts-fallback.img -S autodetect ==> Starting build: 4.14.79-1-lts -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [modconf] -> Running build hook: [block] ==> WARNING: Possibly missing firmware for module: wd719x ==> WARNING: Possibly missing firmware for module: aic94xx → error: command failed to execute correctly ( 2/12) Updating linux initcpios... ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default' -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img ==> Starting build: 4.18.16-arch1-1-ARCH -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [autodetect] -> Running build hook: [modconf] -> Running build hook: [block] -> Running build hook: [filesystems] -> Running build hook: [keyboard] -> Running build hook: [fsck] ==> Generating module dependencies ==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img ==> Image generation successful ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback' -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect ==> Starting build: 4.18.16-arch1-1-ARCH -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [modconf] -> Running build hook: [block] ==> WARNING: Possibly missing firmware for module: wd719x ==> WARNING: Possibly missing firmware for module: aic94xx -> Running build hook: [filesystems] -> Running build hook: [keyboard] -> Running build hook: [fsck] ==> Generating module dependencies ==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-fallback.img ==> Image generation successful ( 3/12) Updating icon theme caches... I was hoping pacman.log might have more detail, but it doesn't even record that there was an error. I think that's a bug. [2018-11-07 11:18] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'fallback' [2018-11-07 11:18] [ALPM-SCRIPTLET] -> -k /boot/vmlinuz-linux-lts -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-lts-fallback.img -S autodetect [2018-11-07 11:18] [ALPM-SCRIPTLET] ==> Starting build: 4.14.79-1-lts [2018-11-07 11:18] [ALPM-SCRIPTLET] -> Running build hook: [base] [2018-11-07 11:18] [ALPM-SCRIPTLET] -> Running build hook: [udev] [2018-11-07 11:18] [ALPM-SCRIPTLET] -> Running build hook: [modconf] [2018-11-07 11:18] [ALPM-SCRIPTLET] -> Running build hook: [block] [2018-11-07 11:18] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: wd719x [2018-11-07 11:18] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: aic94xx [2018-11-07 11:19] [ALPM] running '90-linux.hook'... [2018-11-07 11:19] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default' Where should I look for clues why the command failed? And after that, is attempting another build of the image as simple as `/usr/bin/mkinitcpio -p linux-lts' as root? -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy
On 11/7/18 7:27 AM, Ralph Corderoy wrote:
Hi,
This morning's `pacman -Su' had a problem building the fallback initcpio image for 4.14.79-1-lts. Yesterday's /var/log/pacman.log shows
[2018-11-06 10:45] [ALPM] upgraded linux-lts (4.14.78-1 -> 4.14.79-1) [2018-11-06 10:45] [ALPM] running '60-linux-lts.hook'... [2018-11-06 10:45] [ALPM] running '90-linux-lts.hook'...
that all went well.
Today, package `linux-lts' wasn't upgraded but 90-linux-lts.hook was triggered. 60-linux-lts.hook wasn't.
The 60- hook is: Description = Updating linux-lts module dependencies... and its purpose is to run depmod whenever you update modules. The 90- hook is: Description = Updating linux-lts initcpios... and its purpose is to run mkinitcpio whenever you update either the kernel, or else any file which is part of the mkinitcpio configuration. $ pacman -Qo /usr/lib/initcpio/ /usr/lib/initcpio/ is owned by archiso 39-1 /usr/lib/initcpio/ is owned by btrfs-progs 4.19-1 /usr/lib/initcpio/ is owned by cryptsetup 2.0.5-1 /usr/lib/initcpio/ is owned by device-mapper 2.02.182-1 /usr/lib/initcpio/ is owned by dmraid 1.0.0.rc16.3-11 /usr/lib/initcpio/ is owned by lvm2 2.02.182-1 /usr/lib/initcpio/ is owned by mdadm 4.0-1 /usr/lib/initcpio/ is owned by mkinitcpio 25-1 /usr/lib/initcpio/ is owned by mkinitcpio-busybox 1.29.3-1 /usr/lib/initcpio/ is owned by systemd 239.300-1 As you can see, there's lots of things which can do so! This explains why one ran and the other did not.
pacman's TTY output complained `error: command failed to execute correctly' and the list of `Running' build hooks didn't complete. The same steps for package `linux' immediately after didn't have the same problem.
This means mkinitcpio itself got borked, but rerunning it with a different kernel preset works okay... why, remains to be seen.
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-lts.img ==> Image generation successful ==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'fallback' -> -k /boot/vmlinuz-linux-lts -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-lts-fallback.img -S autodetect ==> Starting build: 4.14.79-1-lts -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [modconf] -> Running build hook: [block] ==> WARNING: Possibly missing firmware for module: wd719x ==> WARNING: Possibly missing firmware for module: aic94xx → error: command failed to execute correctly
So, the fallback image for linux-lts seems to have simply died halfway through (but the main image generated okay).
I was hoping pacman.log might have more detail, but it doesn't even record that there was an error. I think that's a bug.
[2018-11-07 11:18] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'fallback' [2018-11-07 11:18] [ALPM-SCRIPTLET] -> -k /boot/vmlinuz-linux-lts -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-lts-fallback.img -S autodetect [2018-11-07 11:18] [ALPM-SCRIPTLET] ==> Starting build: 4.14.79-1-lts [2018-11-07 11:18] [ALPM-SCRIPTLET] -> Running build hook: [base] [2018-11-07 11:18] [ALPM-SCRIPTLET] -> Running build hook: [udev] [2018-11-07 11:18] [ALPM-SCRIPTLET] -> Running build hook: [modconf] [2018-11-07 11:18] [ALPM-SCRIPTLET] -> Running build hook: [block] [2018-11-07 11:18] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: wd719x [2018-11-07 11:18] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: aic94xx [2018-11-07 11:19] [ALPM] running '90-linux.hook'... [2018-11-07 11:19] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
Well, the log only contains a copy of the output from the hook which you can look up after the fact, so it definitely won't have *more* input. As for logging the failure of the hook to complete, I guess you could open a ticket on the bugtracker for this (and/or post to the pacman-dev mailing list if that floats your boat).
Where should I look for clues why the command failed? And after that, is attempting another build of the image as simple as `/usr/bin/mkinitcpio -p linux-lts' as root?
This looks like it is another occurrence of https://bugs.archlinux.org/task/60396 To test, you could try reinstalling a package that provides a mkinitcpio configuration, and test resizing the terminal to see if you get the same error. Then, try installing pacman-git from the AUR or from my unofficial repo hosted at https://wiki.archlinux.org/index.php/Unofficial_user_repositories#eschwartz The fix should be in pacman-git, and this should no longer crash. -- Eli Schwartz Bug Wrangler and Trusted User
Hi Eli,
$ pacman -Qo /usr/lib/initcpio/ /usr/lib/initcpio/ is owned by archiso 39-1 ...
Thanks, it hadn't occurred to me to use -Qo on a directory before.
Well, the log only contains a copy of the output from the hook which you can look up after the fact, so it definitely won't have *more* input.
Understood. I was thinking along the lines of how a ./configure writes to an agreed file descriptor other than stdout/stderr for more verbose detail, that kind of thing, but there weren't extra messages to record anyway given below suggests it's a SIGPIPE. Would the exit status of the hook have reflected that signal by being 128+13?
As for logging the failure of the hook to complete, I guess you could open a ticket on the bugtracker for this
Okay. https://bugs.archlinux.org/task/60735
This looks like it is another occurrence of https://bugs.archlinux.org/task/60396
It's very likely. I know I did maximise the X terminal part-way through pacman's run to unwrap lines.
To test...
I'll skip that, sorry. Way behind with life as it is at the moment. I've looked at the bug report and the patch and it seems sound.
And after that, is attempting another build of the image as simple as `/usr/bin/mkinitcpio -p linux-lts' as root?
I did this and it seems to have gone well. Thanks for the prompt answers. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy
On 11/7/18 10:06 AM, Ralph Corderoy wrote:
Hi Eli,
$ pacman -Qo /usr/lib/initcpio/ /usr/lib/initcpio/ is owned by archiso 39-1 ...
Thanks, it hadn't occurred to me to use -Qo on a directory before.
It's pretty nifty, yeah. Manpage and --help output does say "file(s)", though -- maybe we should change that.
Well, the log only contains a copy of the output from the hook which you can look up after the fact, so it definitely won't have *more* input.
Understood. I was thinking along the lines of how a ./configure writes to an agreed file descriptor other than stdout/stderr for more verbose detail, that kind of thing, but there weren't extra messages to record anyway given below suggests it's a SIGPIPE. Would the exit status of the hook have reflected that signal by being 128+13?
Yes, pacman just checks if the exit status of the hook was something indicating an error.
As for logging the failure of the hook to complete, I guess you could open a ticket on the bugtracker for this
Thanks. <3
This looks like it is another occurrence of https://bugs.archlinux.org/task/60396
It's very likely. I know I did maximise the X terminal part-way through pacman's run to unwrap lines.
To test...
I'll skip that, sorry. Way behind with life as it is at the moment. I've looked at the bug report and the patch and it seems sound.
Sure, I wasn't expecting problems -- it's been tested already. :) But of course you are free to use my git repo either way -- it's verified by my packaging key already, it contains cool new features, and I am good about keeping it up to date. Anyway, it's planned to be added to a maintenance release for pacman 5.1.2, hopefully soon.
And after that, is attempting another build of the image as simple as `/usr/bin/mkinitcpio -p linux-lts' as root?
I did this and it seems to have gone well.
Forgot to mention, but yeah, running the command that the hook was supposed to run, will usually work. :D If it doesn't work, it would be because the hook specifies NeedsTargets and therefore expects to receive a list of files on standard input, listing the files which triggered the hook.
Thanks for the prompt answers.
No problem! :) -- Eli Schwartz Bug Wrangler and Trusted User
On 11/7/18 1:27 PM, Ralph Corderoy wrote:
Hi,
This morning's `pacman -Su' had a problem building the fallback initcpio
Right command is pacman -Syu
image for 4.14.79-1-lts. Yesterday's /var/log/pacman.log shows
[2018-11-06 10:45] [ALPM] upgraded linux-lts (4.14.78-1 -> 4.14.79-1) [2018-11-06 10:45] [ALPM] running '60-linux-lts.hook'... [2018-11-06 10:45] [ALPM] running '90-linux-lts.hook'...
Hi leoutation,
This morning's `pacman -Su' had a problem building the fallback initcpio
Right command is pacman -Syu
Not in my case as I've previously done a `pacman -Syuw' during `free bandwidth' hours, thanks to cron. Later on, when I'm at the keyboard and it's a convenient time for breakage, I upgrade to what's already been downloaded manually and definitely don't want `-y'. :-) -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy
On 11/7/18 5:46 PM, Ralph Corderoy wrote:
Hi leoutation,
This morning's `pacman -Su' had a problem building the fallback initcpio
Right command is pacman -Syu
Not in my case as I've previously done a `pacman -Syuw' during `free bandwidth' hours, thanks to cron. Later on, when I'm at the keyboard and it's a convenient time for breakage, I upgrade to what's already been downloaded manually and definitely don't want `-y'. :-)
That's dangerous because you could forget and run pacman -S pkgname. Not saying you would, and in fact I've done -Sywu all the time myself... But, just because I do it doesn't mean I should. ;) Instead, I eventually did some fiddling with the checkupdates script, which can be seen here: https://git.archlinux.org/users/eschwartz/pacman-contrib.git/commit/?id=80be... This will eventually make its way to pacman-contrib upstream, but it's pending some pacman changes which are still in git and only very recently landed. In the meantime you can also use https://github.com/eli-schwartz/dotfiles/blob/master/bin/cacheupdates Just run cacheupdates the way you'd run checkupdates, but if you specify the argument "-d" (or "--download") then it will also try to download any pending updates into the cache, still using the temporary copy of the syncdbs. -- Eli Schwartz Bug Wrangler and Trusted User
Hi Eli,
Not in my case as I've previously done a `pacman -Syuw' during `free bandwidth' hours, thanks to cron. Later on, when I'm at the keyboard and it's a convenient time for breakage, I upgrade to what's already been downloaded manually and definitely don't want `-y'. :-)
That's dangerous because you could forget and run pacman -S pkgname.
AIUI, if I did then the version of pkgname it would try to download to install would be that snap-shotted when cron ran `pacman -Syuw'. And that's a problem because that version might not be available for download any more if a later one has been made available? Similarly, with packages pkgname depends on. Is there another problem I don't know? I actually tend to: sudo -i at 1:23 <<<'pacman -Syw pkgname' as it's normally not urgent. Can an attempt to fetch an old, now missing, package still occur even with `pacman -Sy pkgname' because the remote database has been altered, and old versions removed, between the -y's refresh and fetching a long list of packages? -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy
On 11/8/18 12:25 PM, Ralph Corderoy wrote:
Hi Eli,
Not in my case as I've previously done a `pacman -Syuw' during `free bandwidth' hours, thanks to cron. Later on, when I'm at the keyboard and it's a convenient time for breakage, I upgrade to what's already been downloaded manually and definitely don't want `-y'. :-)
That's dangerous because you could forget and run pacman -S pkgname.
AIUI, if I did then the version of pkgname it would try to download to install would be that snap-shotted when cron ran `pacman -Syuw'. And that's a problem because that version might not be available for download any more if a later one has been made available? Similarly, with packages pkgname depends on. Is there another problem I don't know?
-_- No, the problem is that you'll accidentally do a partial update. The package itself might very well be cached already, but if not, you'll get a 404 "cannot find package, I will now error without doing anything". That's not "dangerous". Dangerous is when you break your system by installing incompatible packages via a "successful" partial update.
I actually tend to: sudo -i at 1:23 <<<'pacman -Syw pkgname' as it's normally not urgent.
Can an attempt to fetch an old, now missing, package still occur even with `pacman -Sy pkgname' because the remote database has been altered, and old versions removed, between the -y's refresh and fetching a long list of packages?
It won't be old and now missing, because you used -Sy and it will refresh *again*. But if you don't use -y, then trying to install a package first looks in the cache, where it may not need to be fetched... -- Eli Schwartz Bug Wrangler and Trusted User
Hi Eli,
Dangerous is when you break your system by installing incompatible packages via a "successful" partial update.
Ah, right. I `pacman -S foo'. foo depends on libbar. libbar 1.0 is already installed because xyzzy needs it. The earlier `pacman -Syuw' updated the local database to know libbar 1.1 is available thus installing foo upgrades libbar. xyzzy remains at the old version, incompatible with libbar 1.1.
Can an attempt to fetch an old, now missing, package still occur even with `pacman -Sy pkgname' because the remote database has been altered, and old versions removed, between the -y's refresh and fetching a long list of packages?
It won't be old and now missing, because you used -Sy and it will refresh *again*.
But isn't it multiple stages with a race condition? 1 Update /var/lib/pacman/sync/core.db 2 Update /var/lib/pacman/sync/extra.db 3 Update /var/lib/pacman/sync/community.db 4 Update /var/lib/pacman/sync/multilib.db 5 Create /var/cache/pacman/pkg/foo-1.0.pkg.tar.xz 6 Create /var/cache/pacman/pkg/bar-2.0.pkg.tar.xz 7 Create /var/cache/pacman/pkg/xyzzy-3.0.pkg.tar.xz Can the remote core.db be updated between 1 and 6 because bar 2.1 is available? Do bar-2.0.pkg.tar.xz and bar-2.1.pkg.tar.xz ever exist at the same time? Perhaps the fetch of bar-2.0.pkg.tar.xz might fail because it's been removed by the time it's attempted. This is nothing to do with my original question. :-) I guess I'm asking how is a consistent multi-file database of indexes and `records' presented? It might be that the rare times this happens, the user just runs `pacman -Sy pkgname' again. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy
On 11/8/18 1:05 PM, Ralph Corderoy wrote:
It won't be old and now missing, because you used -Sy and it will refresh *again*.
But isn't it multiple stages with a race condition?
1 Update /var/lib/pacman/sync/core.db 2 Update /var/lib/pacman/sync/extra.db 3 Update /var/lib/pacman/sync/community.db 4 Update /var/lib/pacman/sync/multilib.db 5 Create /var/cache/pacman/pkg/foo-1.0.pkg.tar.xz 6 Create /var/cache/pacman/pkg/bar-2.0.pkg.tar.xz 7 Create /var/cache/pacman/pkg/xyzzy-3.0.pkg.tar.xz
Can the remote core.db be updated between 1 and 6 because bar 2.1 is available? Do bar-2.0.pkg.tar.xz and bar-2.1.pkg.tar.xz ever exist at the same time? Perhaps the fetch of bar-2.0.pkg.tar.xz might fail because it's been removed by the time it's attempted.
This is nothing to do with my original question. :-) I guess I'm asking how is a consistent multi-file database of indexes and `records' presented? It might be that the rare times this happens, the user just runs `pacman -Sy pkgname' again.
Well, most users will have multiple mirrors in their pacman mirrorlist. So if the package is no longer available on the first mirror, pacman will keep trying until it finds it somewhere. Also, the job we use to delete old packages only runs at 3-hour intervals, which means there's a window of opportunity when multiple versions are still available, and most mirrors sync roughly hourly but not on the same minute of that hour. So it would be an odd coincidence if you managed to mistime your update such that 1) your mirror updates in the seconds between syncing the db and downloading the package 2) that mirror update is during its one-hour sync cycle that is also the one-third of the time when the cleanup job runs 3) every one of your fallback mirrors is newer than your first configured mirror If you do manage to win this game of roulette, you can just pacman -Syu again. :) pacman will simply abort the upgrade if it cannot find one of the package files, so consistency is maintained -- it's just a much more exotic case of "I haven't done pacman -Syu in a week and cannot install new packages that have been updated on the mirrors". ... Source: ansible repo containing our dbscripts role, and in particular, the job cleanup.{service,time} : https://git.archlinux.org/infrastructure.git/tree/roles/dbscripts/files The dbscripts cleanup script itself: https://git.archlinux.org/dbscripts.git/tree/cron-jobs/ftpdir-cleanup -- Eli Schwartz Bug Wrangler and Trusted User
participants (3)
-
Eli Schwartz
-
leoutation＠gmx.fr
-
Ralph Corderoy