[arch-general] Multi-threaded mkinitpcio
So every update of linux kernels and modules which need mkinitpcio to be executed, takes too much unnecessary time (maybe 2 min) which could be handled if it executes on multi threads of CPU. Actually this 2 minutes bothers me this much that I'm emailing Arch General Mailing List is because I'm afraid my Arch would be broken if my laptop shuts down in this process and I have to rescue it by live USB. I have read the mkinitpcio wiki page and I can't find anything regarding this matter: https://wiki.archlinux.org/index.php/Mkinitcpio I've checked and my cpu usage and it shows me that it uses maximum power of a core when mkinitpcio is running, so I assumed it could be helped if it runs on multi threads. So my question is why it doesn't execute over all threads or at least have the option to do that? Does it help if it gets executed over multi threads or the bottleneck is somewhere else?
Em abril 2, 2020 22:28 Amin Vakil escreveu:
So every update of linux kernels and modules which need mkinitpcio to be executed, takes too much unnecessary time (maybe 2 min) which could be handled if it executes on multi threads of CPU.
Which compression algorithm are you using?
Actually this 2 minutes bothers me this much that I'm emailing Arch General Mailing List is because I'm afraid my Arch would be broken if my laptop shuts down in this process and I have to rescue it by live USB.
As it's true for pretty much all update process for pretty much every OS in the world, things can break if the update is interrupted.
So my question is why it doesn't execute over all threads or at least have the option to do that?
mkinitcpio is a bunch of scripts copying files to a temporary dir and then compressing the result afterwards. I bet most of this time you're seeing is on compression's fault, not mkinicpio itself.
Does it help if it gets executed over multi threads or the bottleneck is somewhere else?
Depending on the compression algorithm, reproducibility might not work if it's using multiple threads. Also, I see multiple issues that could happen if the build hooks ran out of order. Regards, Giancarlo Razzolini
Actually this 2 minutes bothers me this much that I'm emailing Arch General Mailing List is because I'm afraid my Arch would be broken if my laptop shuts down in this process and I have to rescue it by live USB. For nearly a decade of using this particular Arch installation alone, I never had a single case of losing power while mkinitcpio was running. I wouldn’t be afraid of that scenario. And laptops are better than my PC in that manner, because you have backup power from the battery.
Does it help if it gets executed over multi threads or the bottleneck is somewhere else? For my particular setup, running `mkinitcpio -P` yields those times:
real 1m8.320s user 0m24.742s sys 0m12.806s ------------------------ Half of the time mkinitcpio was waiting for disk I/O. Most likely `find`, but that’s only mu guess. This is only a single computer. Unlike modern laptops it uses spinning rust for storage, but OTOH it is also running a CPU that remembers 2000s and DDR2-800 RAM. So both would affect the outcome.
On Fri, 3 Apr 2020 04:36:37 +0200, mpan wrote:
Half of the time mkinitcpio was waiting for disk I/O.
Even while it's unlikely that only mkinitcpio suffers from a damaged HDD and even while smartctl not necessarily does show issues of a already damaged aged HDD, I would run smartctl and take a look, if it shows something fishy.
hi, change the COMPRESSION variable to lz4 ou lzop in your /etc/mkinitcpio.conf : it will considerably reduce the compression time. grep COMP /etc/mkinitcpio.conf # COMPRESSION #COMPRESSION="gzip" #COMPRESSION="bzip2" #COMPRESSION="lzma" #COMPRESSION="xz" #COMPRESSION="lzop" COMPRESSION="lz4" # COMPRESSION_OPTIONS #COMPRESSION_OPTIONS=() see here <https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO> for quick benchmarks... regards, lacsaP. Le ven. 3 avr. 2020 à 08:08, Ralf Mardorf via arch-general < arch-general@archlinux.org> a écrit :
On Fri, 3 Apr 2020 04:36:37 +0200, mpan wrote:
Half of the time mkinitcpio was waiting for disk I/O.
Even while it's unlikely that only mkinitcpio suffers from a damaged HDD and even while smartctl not necessarily does show issues of a already damaged aged HDD, I would run smartctl and take a look, if it shows something fishy.
Em abril 3, 2020 3:17 Pascal via arch-general escreveu:
hi,
change the COMPRESSION variable to lz4 ou lzop in your /etc/mkinitcpio.conf : it will considerably reduce the compression time.
And break reproducibility. Also, I wouldn't say considerably. Might be faster, but not leaps and bounds faster. Regards, Giancarlo Razzolini
I don't quite understand where the reproducibility is broken ? Amin Vakil says that rebuilding his initrd (when updating kernel and/or modules) takes a lot of time and the final compression phase can be reduced by switching to lz4 or lzop. when you go from xz (which may be his case) to lzop, the difference is remarkable. Le ven. 3 avr. 2020 à 14:47, Giancarlo Razzolini <grazzolini@archlinux.org> a écrit :
Em abril 3, 2020 3:17 Pascal via arch-general escreveu:
hi,
change the COMPRESSION variable to lz4 ou lzop in your /etc/mkinitcpio.conf : it will considerably reduce the compression time.
And break reproducibility. Also, I wouldn't say considerably. Might be faster, but not leaps and bounds faster.
Regards, Giancarlo Razzolini
On 04/04/2020 12:11, Pascal via arch-general wrote:
I don't quite understand where the reproducibility is broken ?
multi-threaded compression does not create a predictable reproducible archive (for xz/zstd).
Amin Vakil says that rebuilding his initrd (when updating kernel and/or modules) takes a lot of time and the final compression phase can be reduced by switching to lz4 or lzop. when you go from xz (which may be his case) to lzop, the difference is remarkable.
Sure, has nothing to do with reproducibility however.
Le ven. 3 avr. 2020 à 14:47, Giancarlo Razzolini <grazzolini@archlinux.org> a écrit :
Em abril 3, 2020 3:17 Pascal via arch-general escreveu:
hi,
change the COMPRESSION variable to lz4 ou lzop in your /etc/mkinitcpio.conf : it will considerably reduce the compression time.
And break reproducibility. Also, I wouldn't say considerably. Might be faster, but not leaps and bounds faster.
Regards, Giancarlo Razzolini
Em abril 4, 2020 12:13 Jelle van der Waa escreveu:
multi-threaded compression does not create a predictable reproducible archive (for xz/zstd).
Zstd is multi-thread safe, as far as I know, xz is not. But the kernel does not support zstd.
Sure, has nothing to do with reproducibility however.
It has, because lzop is the only algorithm that produces unreproducible builds due to them adding their own timestamp, which there's no flag to remove. Of course lzop will be faster than xz, but mkinitcpio's default is gz, which should be comparable, at least in speed, with lzop. Regards, Giancarlo Razzolini
thank you for these details. regard, lacsaP. Le lun. 6 avr. 2020 à 05:39, Giancarlo Razzolini via arch-general < arch-general@archlinux.org> a écrit :
Em abril 4, 2020 12:13 Jelle van der Waa escreveu:
multi-threaded compression does not create a predictable reproducible archive (for xz/zstd).
Zstd is multi-thread safe, as far as I know, xz is not. But the kernel does not support zstd.
Sure, has nothing to do with reproducibility however.
It has, because lzop is the only algorithm that produces unreproducible builds due to them adding their own timestamp, which there's no flag to remove. Of course lzop will be faster than xz, but mkinitcpio's default is gz, which should be comparable, at least in speed, with lzop.
Regards, Giancarlo Razzolini
Hi Giancarlo,
Of course lzop will be faster than xz, but mkinitcpio's default is gz, which should be comparable, at least in speed, with lzop.
A slap-dash test. $ perf stat -e instructions gzip <initramfs-linux-lts-fallback >/dev/null Performance counter stats for 'gzip': 19,533,249,541 instructions:u 19.642066695 seconds time elapsed 19.540737000 seconds user 0.100003000 seconds sys $ perf stat -e instructions lzop <initramfs-linux-lts-fallback >/dev/null Performance counter stats for 'lzop': 1,233,520,108 instructions:u 1.129177816 seconds time elapsed 1.092271000 seconds user 0.036743000 seconds sys $ gzip is about twenty times slower. 1,233,520,108 / 19,533,249,541 = 0.0631497644777874596 1.129177816 / 19.642066695 = 0.05748772944994829119 Yes, my computer is probably slower than most, but then perhaps so is the OP's. -- Cheers, Ralph.
On 06-04-2020 11:47, Ralph Corderoy wrote:
Hi Giancarlo,
Of course lzop will be faster than xz, but mkinitcpio's default is gz, which should be comparable, at least in speed, with lzop. A slap-dash test.
$ perf stat -e instructions gzip <initramfs-linux-lts-fallback >/dev/null Performance counter stats for 'gzip': 19,533,249,541 instructions:u 19.642066695 seconds time elapsed 19.540737000 seconds user 0.100003000 seconds sys $ perf stat -e instructions lzop <initramfs-linux-lts-fallback >/dev/null Performance counter stats for 'lzop': 1,233,520,108 instructions:u 1.129177816 seconds time elapsed 1.092271000 seconds user 0.036743000 seconds sys $
gzip is about twenty times slower.
1,233,520,108 / 19,533,249,541 = 0.0631497644777874596 1.129177816 / 19.642066695 = 0.05748772944994829119
Yes, my computer is probably slower than most, but then perhaps so is the OP's.
Those outputs appear to be from unpacking initramfs. I do think OP and Giancarlo were talking about creating an initramfs . LW
Hi Lone Wolf,
$ perf stat -e instructions gzip <initramfs-linux-lts-fallback >/dev/null $ perf stat -e instructions lzop <initramfs-linux-lts-fallback >/dev/null
Those outputs appear to be from unpacking initramfs. I do think OP and Giancarlo were talking about creating an initramfs .
Oh, sorry for the noise if I'm wrong, but /boot/initramfs-linux-lts-fallback.img is gzip-compressed here. I uncompressed it to give the initramfs-linux-lts-fallback used above. I thought the compression of it was the step OP and Giancarlo were discussing, as you say. -- Cheers, Ralph.
On 06-04-2020 12:43, Ralph Corderoy wrote:
Hi Lone Wolf,
$ perf stat -e instructions gzip <initramfs-linux-lts-fallback >/dev/null $ perf stat -e instructions lzop <initramfs-linux-lts-fallback >/dev/null
Those outputs appear to be from unpacking initramfs. I do think OP and Giancarlo were talking about creating an initramfs . Oh, sorry for the noise if I'm wrong, but /boot/initramfs-linux-lts-fallback.img is gzip-compressed here. I uncompressed it to give the initramfs-linux-lts-fallback used above. I thought the compression of it was the step OP and Giancarlo were discussing, as you say.
Guess I misinterpretated the command, ralph . Below are some data from my own system (threadripper 1920x , NvME2 ssd) . lzop is about twice as fast as gzip , while xz is very slow. Lone_Wolf $ perf stat -e instructions mkinitcpio --generate testing --compress gzip > /dev/null Performance counter stats for 'mkinitcpio --generate testing --compress gzip': 16,585,047,999 instructions:u 4.706074377 seconds time elapsed 4.000137000 seconds user 1.187220000 seconds sys $ perf stat -e instructions mkinitcpio --generate testing --compress xz
/dev/null
Performance counter stats for 'mkinitcpio --generate testing --compress xz': 84,360,010,752 instructions:u 17.415348248 seconds time elapsed 16.756740000 seconds user 1.500338000 seconds sys $ perf stat -e instructions mkinitcpio --generate testing --compress lzop > /dev/null Performance counter stats for 'mkinitcpio --generate testing --compress lzop': 5,316,618,410 instructions:u 2.426944720 seconds time elapsed 1.695263000 seconds user 1.027651000 seconds sys $
participants (8)
-
Amin Vakil
-
Giancarlo Razzolini
-
Jelle van der Waa
-
Lone_Wolf
-
mpan
-
Pascal
-
Ralf Mardorf
-
Ralph Corderoy