[arch-dev-public] RFC: (devtools) Changing default compression method to zstd

[Baptiste Jonglez]

Hi,

On 24-03-19, Robin Broda via arch-dev-public wrote:
> So, TL;DR, the benefits of `zstd -c -T0 -18 -` over `xz -c -z -` are:
> - Massive speed gain in compression
> - Massive speed gain in decompression
> - Stable, reproducible multithreading
> The speed gain in decompression substantially increases pacman's package installation speed.

Interesting results, thanks!

Just one detail: your results for -19, -20 and -21 are identical because
apparently zstd needs an additional flag (--ultra) to "unlock" the higher
compression levels:

    zstd -c -T0 -20 -
    Warning : compression level higher than max, reduced to 19


Also, I see you did not test zstd with a small number of cores: can you
add e.g. -T1, -T2 and -T4 to the comparison?  It would give a more
realistic idea of what to expect when building on a typical machine, as
opposed to dragon ;)
In my tests, using less threads also decreased memory usage when
compressing (35% less memory when switching from -T2 to -T1).

For decompression, it seems that both xz and zstd run single-threaded, so
there's not much to think about (zstd is just incredibly fast).

In any case, I support this change!

Baptiste
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.archlinux.org/pipermail/arch-dev-public/attachments/20190324/c45d0155/attachment.sig>