[pacman-dev] Incremental package index support?

[William Giokas]

On Tue, Mar 26, 2013 at 01:56:35PM +1000, Allan McRae wrote:
> On 25/03/13 04:39, Dave Reisner wrote:
> > On Sun, Mar 24, 2013 at 07:28:15PM +0100, Lukas Fleischer wrote:
> >> On Sun, Mar 24, 2013 at 11:26:04AM -0700, Anatol Pomozov wrote:
> >>> Hi
> >>>
> >>> On Sun, Mar 24, 2013 at 10:57 AM, Anatol Pomozov
> >>> <anatol.pomozov at gmail.com> wrote:
> >>>> Hi,
> >>>>
> >>>> I am an update junkie. I do "pacman -Suy" several times a day. One
> >>>> thing I do not like with current pacman is that it redownloads whole
> >>>> index file each time I update system.
> >>>>
> >>>> :: Synchronizing package databases...
> >>>>  core                     104.4 KiB   104K/s 00:01 [######################] 100%
> >>>>  extra                   1416.9 KiB   171K/s 00:08 [######################] 100%
> >>>>  community               1916.6 KiB   204K/s 00:09 [######################] 100%
> >>>>
> >>>> It is 3.5M it total even if only just a few new packages were added.
> >>>> Multiply 3.5M to number of users and you'll get a huge number. IMHO it
> >>>> is waste of bandwidth and user time. From other side index is
> >>>> incremental by it nature. Could pacman use index format that fits
> >>>> incremental updates better?
> >>>>
> >>>> A naive proposal is to use compressed format for "full fetch" (e.g.
> >>>> for users who did not update more than a week), and uncompressed
> >>>> append-only index for incremental. On incremental update client sends
> >>>> request to server "give me append file file starting from position XXX
> >>>> till the end of file", so only delta from the last update will be
> >>>> fetched. Then pacman applies the delta to client index.
> >>>>
> >>>> What do you think about this idea?
> >>>
> >>> Ok, found pacman.conf option "UseDelta" that seems does what I need.
> >>> BTW why it is not enabled by default? Isn't saving bandwidth a good
> >>> thing?
> >>
> >> I'm not sure whether "UseDelta" also affects package lists (rather than
> >> packages only). Arch Linux does not support deltas on the server side
> >> yet, see FS#18590 [1] for details.
> >>
> >> [1] https://bugs.archlinux.org/task/18590
> > 
> > UseDelta only affects packages. It's simply not feasible to create
> > deltas for the sync DBs as they exist today, especially over HTTP/FTP.
> > 
> 
> To clarify - 10s if not 100s of packages get updated on an average day.
>  That is a lot of deltas...

If only libgit2 was still a thing...

-- 
William Giokas | KaiSforza
GnuPG Key: 0x73CD09CF
Fingerprint: F73F 50EF BBE2 9846 8306  E6B8 6902 06D8 73CD 09CF
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mailman.archlinux.org/pipermail/pacman-dev/attachments/20130325/86d9b828/attachment-0001.asc>