CCing to pacman-dev. On Mon, Mar 24, 2008 at 3:25 PM, Jan de Groot wrote:

On Mon, 2008-03-24 at 14:43 -0500, Dan McGee wrote:

...

On Mon, Mar 24, 2008 at 2:24 PM, Mark Constable wrote:

...

FWIW, possibly worth considering adding lzma to pacman...

http://www.kdedevelopers.org/node/3326

--markc

Interesting. You may want ot take this to the pacman-dev ML. A good "real-world" look might be grabbing the openoffice package and uncompressing it to a folder, and then run the following tests:

bsdtar czf package (gzip) bsdtar cjf package (bzip2) <lzma creation of package>

and then the similar unzip costs, along with the size of the original folder structure (unzipped package) vs. their zipped counterparts.

-Dan

I tested with OpenOffice.org, here are the results. Note that I had to use a weird construction to compress lzma files, as tar has no native support yet (1.19.90 has official lzma support I read on some debian mailinglist).

Compression times take a huge increase in time, but in case of openoffice, I think 30 seconds extra compression time will save on much more upload time, as the resulting package would be 20MB smaller. For the end user, it depends on their connection. Most users will have <10Mb/s connections and will see a positive change in installation speed: 10 seconds extra for extraction, 20 seconds less for downloading the package.

[jan@server ~]$ time bsdtar czf oo.tar.gz oo/ real 0m24.244s user 0m23.882s sys 0m0.360s

[jan@server ~]$ time bsdtar cjf oo.tar.bz2 oo/ real 0m55.812s user 0m55.310s sys 0m0.377s

[jan@server ~]$ time tar c oo | lzma -2 > oo.tar.lzma real 0m55.070s user 0m54.583s sys 0m0.740s

[jan@server ~]$ time tar c oo | lzma -1 > oo.tar.lzma1 real 0m38.995s user 0m38.737s sys 0m0.503s

[jan@server ~]$ du -h oo.tar.* 108M oo.tar.bz2 115M oo.tar.gz 95M oo.tar.lzma 101M oo.tar.lzma1

[jan@server u]$ time tar zxf ../oo.tar.gz real 0m3.570s user 0m3.383s sys 0m0.967s

[jan@server u]$ time tar jxf ../oo.tar.bz2 real 0m19.431s user 0m17.826s sys 0m1.367s

[jan@server u]$ time tar --use-compress-program=lzma -xf ../oo.tar.lzma real 0m13.211s user 0m12.306s sys 0m1.117s (the lzma1 archive gives equal timings)

Did you try with bsdtar on the extraction as well? That may make a difference over (gnu)tar. Should be: time bsdtar xf ../oo.tar.gz time bsdtar xf ../oo.tar.bz2 time bsdtar --use-compress-program=lzma -xf ../oo.tar.lzma -Dan