[pacman-dev] old delta stat

[Xavier]

On Mon, Nov 30, 2009 at 2:54 AM, Allan McRae <allan at archlinux.org> wrote:
> Xavier wrote:
>>
>> At some point during this year, I was generating delta for each update
>> that came in.
>> I don't remember if I posted any results here, so just in case, before
>> I throw all this stuff away, I will post the filesize I got here :)
>> Looking quickly at it, it seems a gnome update would see an awesome
>> benefit.
>>
> <snip>
>
> Those numbers do indeed look quite nice.
>
> Can you remind me what the status of the repo delta cleanup script is? And
> do deltas work nicely with xz files given xdelta3 is patched for it?
>

1) Repo delta cleanup script :
I am more or less happy with the basic logic of my patch :
http://code.toofishes.net/cgit/xavier/pacman.git/commit/?h=working&id=8a3c54bf3622a4af6e2b45945412fa3e02994e45

The problem is the interface :
cleanupdelta [-b <pacman db>] core extra ... : check the listed sync databases

Say you have core.db.tar.gz somewhere. You need to extract that to
$FOO/sync/core/
then use cleanupdelta -b $FOO core

Looks kinda ugly.. Ideally we would be able to run cleanupdelta
directly on the tar.gz , but we are not there yet.

2) xz + xdelta3

I stole a script from bbs that I edited slightly and posted to this ML
earlier this year. This script allowed to take all packages from the
cache directory and generate all deltas it can from that.
It was quite easy to take the cache of .gz packages, and convert that
to two other caches of .xz and .bz2 packages, and then run this delta
script on each cache. It also prints out the ratio delta size /
package size for each delta.
So iirc, ratio were worse for xz packages than gz, but still alright.
But I don' t remember the numbers

Anyway, the delta size between two xz packages or two gz packages
should be exactly the same, since the delta is made between
uncompressed data. I would just expect the meta-info inside the delta
to be slightly different because of different filenames and
compression.
That means the better the package compression is, the less interesting
the deltas are (because they automatically get a bigger ratio).

Finally, xdelta3 was finally recently patched to support xz :
http://code.google.com/p/xdelta/source/detail?r=298#
I don't know in which upstream release this belongs. It seems the last
release 3.0w was on Oct 25 and the patch on Oct 27 but it will
probably be in the next one.
Anyway the patch is exactly the same as the one we have in our xdelta3
package currently.