On 3/25/19 1:38 AM, Evangelos Foutras via arch-dev-public wrote:
On Mon, 25 Mar 2019 at 01:22, Jan Alexander Steffens via arch-dev-public <arch-dev-public@archlinux.org> wrote:
When we implement this, I would say we go with "zstd -c -T0 -" in pacman's makepkg.conf and "zstd -C -T0 -18 -" in the configs shipped with devtools.
I think users that build their own local packages are more likely to benefit from fast compression. Anyone building with makechrootpkg for distribution gets the high compression level.
That's actually really smart! We can also leave out "-T0" since the default compression level is very fast anyway. Plus, it's already implemented in this way in pacman git so we don't have to touch anything there. (But, if it were to be multithreaded there as well, I would replace zstd with zstdmt; same for devtools.)
That's why the subject line is prefixed with '(devtools)' :) What's unclear to me is the difference between zstd -T0 and zstdmt, however.
As far as compression level goes, I believe we should select the highest one that doesn't have increased memory requirements during decompression. So that would be -19. Robin makes a good case about -18; looking at upstream's "Compression Speed vs Ratio" graph [1] I would say -18 is preferable to -19 if we are concerned about compression speed and memory usage (my totally unscientific measurements show a 25% memory increase and 20% speed decrease going from -18 to -19 when using -T4). That said, I might still opt for -19 due to the slightly higher compression ratio; memory usage isn't too big of an issue and the slower speed is mitigated by multithreading (i.e.: it will still be much faster than xz).
I do think that at -19+, memory usage becomes a bigger issue. The difference between -18 and -19 on cuda is almost a gigabyte! While not really a problem for our beefy build boxes, some 4- or even 8-GB developer machines could really suffer from such an incline in memory usage. Thus i stand by -18 being the more sensible choice. Rob