[arch-dev-public] RFC: (devtools) Changing default compression method to zstd

[Robin Broda]

On 3/24/19 9:18 PM, Baptiste Jonglez wrote:
> Just one detail: your results for -19, -20 and -21 are identical because
> apparently zstd needs an additional flag (--ultra) to "unlock" the higher
> compression levels:
> 
>     zstd -c -T0 -20 -
>     Warning : compression level higher than max, reduced to 19
> 
> Also, I see you did not test zstd with a small number of cores: can you
> add e.g. -T1, -T2 and -T4 to the comparison?  It would give a more
> realistic idea of what to expect when building on a typical machine, as
> opposed to dragon ;)
> In my tests, using less threads also decreased memory usage when
> compressing (35% less memory when switching from -T2 to -T1).
> 

Damn, i knew i must've missed something - archange had already mentioned on IRC that these results look weird, but i shrugged it off.
Should've double-checked. I'll get you a new table with the higher levels fixed and a second set with -T2 for comparison later.

Regardless, IIRC preliminary testing showed that these gains are not worth it, as they were quite small in the tests we ran a while ago.


> For decompression, it seems that both xz and zstd run single-threaded, so
> there's not much to think about (zstd is just incredibly fast).
> 

Correct


Rob

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.archlinux.org/pipermail/arch-dev-public/attachments/20190325/1b57ee56/attachment-0001.sig>