[pacman-dev] Incremental package index support?

[Allan McRae]

On 25/03/13 04:39, Dave Reisner wrote:
> On Sun, Mar 24, 2013 at 07:28:15PM +0100, Lukas Fleischer wrote:
>> On Sun, Mar 24, 2013 at 11:26:04AM -0700, Anatol Pomozov wrote:
>>> Hi
>>>
>>> On Sun, Mar 24, 2013 at 10:57 AM, Anatol Pomozov
>>> <anatol.pomozov at gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I am an update junkie. I do "pacman -Suy" several times a day. One
>>>> thing I do not like with current pacman is that it redownloads whole
>>>> index file each time I update system.
>>>>
>>>> :: Synchronizing package databases...
>>>>  core                     104.4 KiB   104K/s 00:01 [######################] 100%
>>>>  extra                   1416.9 KiB   171K/s 00:08 [######################] 100%
>>>>  community               1916.6 KiB   204K/s 00:09 [######################] 100%
>>>>
>>>> It is 3.5M it total even if only just a few new packages were added.
>>>> Multiply 3.5M to number of users and you'll get a huge number. IMHO it
>>>> is waste of bandwidth and user time. From other side index is
>>>> incremental by it nature. Could pacman use index format that fits
>>>> incremental updates better?
>>>>
>>>> A naive proposal is to use compressed format for "full fetch" (e.g.
>>>> for users who did not update more than a week), and uncompressed
>>>> append-only index for incremental. On incremental update client sends
>>>> request to server "give me append file file starting from position XXX
>>>> till the end of file", so only delta from the last update will be
>>>> fetched. Then pacman applies the delta to client index.
>>>>
>>>> What do you think about this idea?
>>>
>>> Ok, found pacman.conf option "UseDelta" that seems does what I need.
>>> BTW why it is not enabled by default? Isn't saving bandwidth a good
>>> thing?
>>
>> I'm not sure whether "UseDelta" also affects package lists (rather than
>> packages only). Arch Linux does not support deltas on the server side
>> yet, see FS#18590 [1] for details.
>>
>> [1] https://bugs.archlinux.org/task/18590
> 
> UseDelta only affects packages. It's simply not feasible to create
> deltas for the sync DBs as they exist today, especially over HTTP/FTP.
> 

To clarify - 10s if not 100s of packages get updated on an average day.
 That is a lot of deltas...