[pacman-dev] [PATCH] pkgdelta: use highest compression ratio when creating deltas with xdelta3

[Allan McRae]

On 15/03/14 13:12, Matthias Krüger wrote:
> On 03/13/2014 07:11 AM, Allan McRae wrote:
>> On 06/03/14 09:25, Matthias Krüger wrote:
>>> Side note: it might be even more advantageous to use bsdiff instead of
>>> xdelta3
>>> comparing the /usr/bin/blender binaries of the above versions
>>> (12:2.69.c7ac0e-1 and 13:2.69.13290d-1) :
>>>
>>> xdelta3     10.4M
>>> xdelta3 -9   9.9M
>>> bsdiff          4.7M
>>
>> I took a look, and changing from xdelta3 to bsdiff would be very simple.
>>   It looks like it is a five minute patch...
>>
>> But what I need is for someone to generate deltas (with and without -9
>> maybe) for a whole bunch of packages.  Then generate diffs using bsdiff
>> and compare the results.  The comparison will need to include:
>>
>> 1) size of deltas/diffs
>> 2) memory used when reconstructing package
>> 3) time taken to reconstruct package.
>>
>> Once we have that information, we can make an informed decision.
>>
>> Allan
>>
>>
> I got some numbers for xdelta (-9), see attached file.
> If someone provides some script or tool to run bsdiff on a package
> properly (so that it does not diff the archives themselves), I can offer
> to compute the numbers for bsdiff as well.

The numbers of -9 look like there is no significant change. A quick look
here showed also no change adding -S djw.  I'll accept a patch adding
both -S djw and -9 to the diff creation.

I ran some tests on my system.  bsdiff uses masses of memory when
reconstructing the file (needs ~16x the size of the file).  It can use
less, but the performance penalty is massive.  And it uses even more
when creating the diff.  Coupled with its lack of transparent
decompression, I don't think we should consider that further.

Allan