On Mon, Nov 30, 2009 at 2:54 AM, Allan McRae
Xavier wrote:
At some point during this year, I was generating delta for each update that came in. I don't remember if I posted any results here, so just in case, before I throw all this stuff away, I will post the filesize I got here :) Looking quickly at it, it seems a gnome update would see an awesome benefit.
<snip>
Those numbers do indeed look quite nice.
Can you remind me what the status of the repo delta cleanup script is? And do deltas work nicely with xz files given xdelta3 is patched for it?
1) Repo delta cleanup script : I am more or less happy with the basic logic of my patch : http://code.toofishes.net/cgit/xavier/pacman.git/commit/?h=working&id=8a3c54bf3622a4af6e2b45945412fa3e02994e45 The problem is the interface : cleanupdelta [-b <pacman db>] core extra ... : check the listed sync databases Say you have core.db.tar.gz somewhere. You need to extract that to $FOO/sync/core/ then use cleanupdelta -b $FOO core Looks kinda ugly.. Ideally we would be able to run cleanupdelta directly on the tar.gz , but we are not there yet. 2) xz + xdelta3 I stole a script from bbs that I edited slightly and posted to this ML earlier this year. This script allowed to take all packages from the cache directory and generate all deltas it can from that. It was quite easy to take the cache of .gz packages, and convert that to two other caches of .xz and .bz2 packages, and then run this delta script on each cache. It also prints out the ratio delta size / package size for each delta. So iirc, ratio were worse for xz packages than gz, but still alright. But I don' t remember the numbers Anyway, the delta size between two xz packages or two gz packages should be exactly the same, since the delta is made between uncompressed data. I would just expect the meta-info inside the delta to be slightly different because of different filenames and compression. That means the better the package compression is, the less interesting the deltas are (because they automatically get a bigger ratio). Finally, xdelta3 was finally recently patched to support xz : http://code.google.com/p/xdelta/source/detail?r=298# I don't know in which upstream release this belongs. It seems the last release 3.0w was on Oct 25 and the patch on Oct 27 but it will probably be in the next one. Anyway the patch is exactly the same as the one we have in our xdelta3 package currently.