[arch-general] About Arch pkg compress format

[Jan de Groot]

On Wed, 2008-05-21 at 18:47 +0300, Dimitrios Apostolou wrote:
> Hi,
> 
> On Tue, 2008-05-20 at 10:14 +0200, Jan de Groot wrote:
> > Pacman itself is ready for .tar.bz2 package files. The whole issue
> > with .bz2 files is that compression and decompression times increase a
> > lot without giving the same amount of size reduction back. We've done
> > some recent tests with LZMA, which compresses just as good as bzip2 at
> > the lowest compression rate, but does it at the same speed as gzip. 
> 
> About LZMA I should add that when using higher (actually the default)
> compression rate, compression is much better than bzip2 but takes more
> time/memory. Decompression however, which is what really matters in a
> packaging format, is kept fast and lightweight.

What's the memory usage when unzipping an LZMA file? Is it much higher
than the needs of gzip? We already have problems supporting low-memory
systems with our installer, adding a compression algorithm that eats
more memory will cause even more problems for these systems.

> > The
> > downside is that LZMA is not supported by libarchive, and won't be
> > supported officially either, because libarchive is BSD licensed and LZMA
> > is GPL licensed.
> 
> Can't this problem be circumvented by spawning the lzma command line
> utility, and piping all data to it? I understand that this perhaps
> negates the purpose of libarchive, but the overhead should be small.

libarchive can't spawn external commands to unzip archive files. bsdtar
can, but as pacman is a binary with libarchive integrated, I don't see
this happening.