[arch-general] Suggestion: switch to zstd -19 for compressing packages over xz

[Adam Fontenot]

Hi,

It's now been about half a year since support for zstd landed in our
packaging tools. I've been quietly using it for all my locally built
packages since then with no issues. I think it would be worthwhile to
have a discussion about whether to use zstd for officially built
packages. Here is a brief summary of negatives and positives:

Negatives:
 * Changing things takes time and might break someone's workflow.
 * Zstd -19 results in slightly larger files than xz -6 (default).

Positives:
 * Change would be invisible to most users.
 * Would keep Arch Linux on top of the latest in packaging tech.
 * Updating would be much faster for most users.

To expand on that last point: for reasonably fast connections, the
additional time required to decompress xz compressed packages means
that updating can actually take more time than it would for packages
that are not compressed at all! Because zstd is designed to have very
fast decompression, for a wide range of modern broadband connections
zstd -19 is the fastest algorithm to download and decompress. For
example, check out this compression test (with the Mozilla dataset):
https://quixdb.github.io/squash-benchmark/unstable/

Or look at my local test with the most recent Firefox package:

Tool      Compression time  Size   Time to DL (100 Mbps) + decompress
xz -6     5m 53s            49 MB  0m 21s
zstd -19  6m 0s             53 MB  0m 6s

So while xz and zstd compress in about the same amount of time and
result in files of similar size, from the user's standpoint zstd
results in much faster updates. Multiply this by a few hundred packages
and you have a pretty substantial effect.

I look forward to discussing this with you all.

Cheers,
Adam