[pacman-dev] delta support in libalpm

[Brendan Hide]

Hi guys

I'm new here so I'm asking in advance that you forgive my ignorance. 
Even before having clicked send, I feel like I'm spamming... o.O

Xavier wrote:
> On Sat, Nov 8, 2008 at 12:47 AM, Henning Garus
> <henning.garus at googlemail.com> wrote:
>   
>> 2. create the package, but don't compress it with bsdtar, use gzip -n
>> instead. This means we have to use gzip again, in libalpm, when we
>> apply the delta
> That sounds alright, I just noticed that xdelta3 has an option to
> disable the external recompression : -R
> So we don't even have extra decompression/recompression steps, there is no loss.
>
> +                       snprintf(command, PATH_MAX, "xdelta3 -d -R -c
> -s %s %s | gzip -n > %s", from, delta, to);
>   
The way I understand xdelta3's -R and -D options:
   -D           disable external decompression (encode/decode)
When applying a delta, same behaviour as -R
When creating a delta, even when given 2 compressed files, do not 
discern if the file is compressed, ie, given 2 .tar.gz files, pretend 
they're .bin files

   -R           disable external recompression (decode)
When applying a delta, given a compressed file, decompress *if* the 
delta's metadata indicates the file was decompressed in the encode 
process, apply the delta and, if decompression occurred whilst applying 
the delta, do not bother to recompress. ie, when given a .tar.gz and a 
.xd3, create a .tar

Unless my understanding above is completely wrong, using -R is going to 
help but not without -D in the encoding process.

Also, since we're doing md5s of the .tar.gz instead of the .tar, we'd 
also need to change some of the housekeeping - perhaps doing md5s of the 
.tar as well as (or instead of) the .tar.gz.

There was also a bit of recent discussion on the Arch forum about this. 
Some statistics indicate that vanilla -D isn't really worth it. 
http://bbs.archlinux.org/viewtopic.php?pid=496539#p496539 shows a 10% 
bandwidth savings with -D versus 85% bw savings without. I mentioned a 
kluge workaround there, gzip --rsyncable, giving a 77% bw saving. The 
kluge probably isn't the right way to go anyway.

So, um... how does this change the way forward? Or is my understanding 
of the -R parameter completely wrong?

-- 
__________
Brendan Hide