[arch-general] lzma compression tests

[Jan de Groot]

On Mon, 2008-03-24 at 14:43 -0500, Dan McGee wrote:
> On Mon, Mar 24, 2008 at 2:24 PM, Mark Constable <markc at renta.net> wrote:
> > FWIW, possibly worth considering adding lzma to pacman...
> >
> >   http://www.kdedevelopers.org/node/3326
> >
> >  --markc
> 
> Interesting. You may want ot take this to the pacman-dev ML. A good
> "real-world" look might be grabbing the openoffice package and
> uncompressing it to a folder, and then run the following tests:
> 
> bsdtar czf package (gzip)
> bsdtar cjf package (bzip2)
> <lzma creation of package>
> 
> and then the similar unzip costs, along with the size of the original
> folder structure (unzipped package) vs. their zipped counterparts.
> 
> -Dan
> 

I tested with OpenOffice.org, here are the results. Note that I had to
use a weird construction to compress lzma files, as tar has no native
support yet (1.19.90 has official lzma support I read on some debian
mailinglist).

Compression times take a huge increase in time, but in case of
openoffice, I think 30 seconds extra compression time will save on much
more upload time, as the resulting package would be 20MB smaller.
For the end user, it depends on their connection. Most users will have
<10Mb/s connections and will see a positive change in installation
speed: 10 seconds extra for extraction, 20 seconds less for downloading
the package.

[jan at server ~]$ time bsdtar czf oo.tar.gz oo/
real	0m24.244s
user	0m23.882s
sys	0m0.360s

[jan at server ~]$ time bsdtar cjf oo.tar.bz2 oo/
real	0m55.812s
user	0m55.310s
sys	0m0.377s

[jan at server ~]$ time tar c oo | lzma -2 > oo.tar.lzma
real	0m55.070s
user	0m54.583s
sys	0m0.740s

[jan at server ~]$ time tar c oo | lzma -1 > oo.tar.lzma1
real	0m38.995s
user	0m38.737s
sys	0m0.503s

[jan at server ~]$ du -h oo.tar.*
108M	oo.tar.bz2
115M	oo.tar.gz
95M	oo.tar.lzma
101M	oo.tar.lzma1

[jan at server u]$ time tar zxf ../oo.tar.gz
real	0m3.570s
user	0m3.383s
sys	0m0.967s

[jan at server u]$ time tar jxf ../oo.tar.bz2 
real	0m19.431s
user	0m17.826s
sys	0m1.367s

[jan at server u]$ time tar --use-compress-program=lzma -xf ../oo.tar.lzma
real	0m13.211s
user	0m12.306s
sys	0m1.117s
(the lzma1 archive gives equal timings)