[arch-general] xz compression in kernel 2.6.38
Hi all, Now that kernel 2.6.38 supports xz (lzma2, not lzma), how about using it as the default compression for the kernel and initramfs. mkinitcpio simply requires COMPRESSION="xz" to generate lzma2 compressed initramfs. But xz compressed kernel requires change in the config option and once module-init-toools supports xz compressed modules we can say good bye to gzip. Related commits :- For kernel and initramfs xz support - http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdif... http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdif... http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdif... Squashfs xz support http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdif... http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdif... I think this option should be added to mkinitcpio to make users aware of this option. Not sure of the reduction in kernel and initramfs size though (compared to gzip and lzma). Regards. Keshav PS: Seems like I am actually late in conveying this since 2.6.38 kernel is now in testing. But atleast from pkgrel 2 onwards?
[2011-03-16 19:58:23 +0530] KESHAV P.R.:
Now that kernel 2.6.38 supports xz (lzma2, not lzma), how about using it as the default compression for the kernel and initramfs. mkinitcpio simply requires COMPRESSION="xz" to generate lzma2 compressed initramfs. But xz compressed kernel requires change in the config option and once module-init-toools supports xz compressed modules we can say good bye to gzip.
Since XZ decompresses much more slowly than GZIP, this does not seem like a good idea unless it is backed up by figures showing that the space gain is really worth it. -- Gaetan
On Wed, Mar 16, 2011 at 20:09, Gaetan Bisson <bisson@archlinux.org> wrote:
[2011-03-16 19:58:23 +0530] KESHAV P.R.:
Now that kernel 2.6.38 supports xz (lzma2, not lzma), how about using it as the default compression for the kernel and initramfs. mkinitcpio simply requires COMPRESSION="xz" to generate lzma2 compressed initramfs. But xz compressed kernel requires change in the config option and once module-init-toools supports xz compressed modules we can say good bye to gzip.
Since XZ decompresses much more slowly than GZIP, this does not seem like a good idea unless it is backed up by figures showing that the space gain is really worth it.
I don't think with the present processors the decompression speed is a big deal. But anyway the testing kernel already has xz enabled and is the default now ( http://projects.archlinux.org/svntogit/packages.git/commit/kernel26/trunk/co... ) . Still i think this discussion is relevant when it comes to initramfs and compressed kernel modules. Damn, the devs are fast. - Keshav
On Wed, Mar 16, 2011 at 20:12, KESHAV P.R. <skodabenz@gmail.com> wrote:
On Wed, Mar 16, 2011 at 20:09, Gaetan Bisson <bisson@archlinux.org> wrote:
[2011-03-16 19:58:23 +0530] KESHAV P.R.:
Now that kernel 2.6.38 supports xz (lzma2, not lzma), how about using it as the default compression for the kernel and initramfs. mkinitcpio simply requires COMPRESSION="xz" to generate lzma2 compressed initramfs. But xz compressed kernel requires change in the config option and once module-init-toools supports xz compressed modules we can say good bye to gzip.
mkinitcpio.conf requires COMPRESSION="xz --check=crc32" , not simply "xz" since crc64 (default) checksum is not supported by kernel de-compressor. See http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Do... . - Keshav
Am 16.03.2011 17:12, schrieb KESHAV P.R.:
On Wed, Mar 16, 2011 at 20:12, KESHAV P.R. <skodabenz@gmail.com> wrote:
On Wed, Mar 16, 2011 at 20:09, Gaetan Bisson <bisson@archlinux.org> wrote:
[2011-03-16 19:58:23 +0530] KESHAV P.R.:
Now that kernel 2.6.38 supports xz (lzma2, not lzma), how about using it as the default compression for the kernel and initramfs. mkinitcpio simply requires COMPRESSION="xz" to generate lzma2 compressed initramfs. But xz compressed kernel requires change in the config option and once module-init-toools supports xz compressed modules we can say good bye to gzip.
mkinitcpio.conf requires COMPRESSION="xz --check=crc32" , not simply "xz" since crc64 (default) checksum is not supported by kernel de-compressor. See http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Do...
Thanks for this info. There is one request to remove the "-9" default compression ratio, so this is probably a good time to change how compression is handled for mkinitcpio. The advantage of xz over lzma shouldn't be that big in any case.
On Wed, Mar 16, 2011 at 22:42, Thomas Bächler <thomas@archlinux.org> wrote:
Am 16.03.2011 17:12, schrieb KESHAV P.R.:
On Wed, Mar 16, 2011 at 20:12, KESHAV P.R. <skodabenz@gmail.com> wrote:
On Wed, Mar 16, 2011 at 20:09, Gaetan Bisson <bisson@archlinux.org> wrote:
[2011-03-16 19:58:23 +0530] KESHAV P.R.:
Now that kernel 2.6.38 supports xz (lzma2, not lzma), how about using it as the default compression for the kernel and initramfs. mkinitcpio simply requires COMPRESSION="xz" to generate lzma2 compressed initramfs. But xz compressed kernel requires change in the config option and once module-init-toools supports xz compressed modules we can say good bye to gzip.
mkinitcpio.conf requires COMPRESSION="xz --check=crc32" , not simply "xz" since crc64 (default) checksum is not supported by kernel de-compressor. See http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Do...
Thanks for this info. There is one request to remove the "-9" default compression ratio, so this is probably a good time to change how compression is handled for mkinitcpio. The advantage of xz over lzma shouldn't be that big in any case.
Kernel documentation recommends using a 1MiB dictionary max (for the kernel, maybe more for initramfs), not 64MiB that is required for -9 option. Otherwise it is better to have specific compression level option for each format in mkinitcpio.conf instead of the actual mkinitcpio script. In case of xz, kernel docs recommends I am typing this from Arch x86_64 2.6.38 with xz --check=crc32 -9 compressed initramfs. I don't know about any problems with -9 option, except maaybe amount of memory required to decompress. - Keshav Offnote: What happened to Big Kernel Lock (in the config)
Am 16.03.2011 19:00, schrieb KESHAV P.R.:
Offnote: What happened to Big Kernel Lock (in the config)
There are still some minor things that require the BKL, so it is still enabled. If you don't use any of those features, there will be no BKL - it doesn't steal your performance or anything. There are patches that allow the BKL to be removed in 2.6.39 I think. We'll see.
On Wed, Mar 16, 2011 at 11:14 PM, Thomas Bächler <thomas@archlinux.org> wrote:
There are patches that allow the BKL to be removed in 2.6.39 I think. We'll see.
The BKL will no longer be there in .39: <http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=4ba8216cd90560bc402f52076f64d8546e8aefcb>. Cheers, Tom
participants (4)
-
Gaetan Bisson
-
KESHAV P.R.
-
Thomas Bächler
-
Tom Gundersen