[pacman-dev] Incremental package index support?

[Dave Reisner]

On Sun, Mar 24, 2013 at 07:28:15PM +0100, Lukas Fleischer wrote:
> On Sun, Mar 24, 2013 at 11:26:04AM -0700, Anatol Pomozov wrote:
> > Hi
> > 
> > On Sun, Mar 24, 2013 at 10:57 AM, Anatol Pomozov
> > <anatol.pomozov at gmail.com> wrote:
> > > Hi,
> > >
> > > I am an update junkie. I do "pacman -Suy" several times a day. One
> > > thing I do not like with current pacman is that it redownloads whole
> > > index file each time I update system.
> > >
> > > :: Synchronizing package databases...
> > >  core                     104.4 KiB   104K/s 00:01 [######################] 100%
> > >  extra                   1416.9 KiB   171K/s 00:08 [######################] 100%
> > >  community               1916.6 KiB   204K/s 00:09 [######################] 100%
> > >
> > > It is 3.5M it total even if only just a few new packages were added.
> > > Multiply 3.5M to number of users and you'll get a huge number. IMHO it
> > > is waste of bandwidth and user time. From other side index is
> > > incremental by it nature. Could pacman use index format that fits
> > > incremental updates better?
> > >
> > > A naive proposal is to use compressed format for "full fetch" (e.g.
> > > for users who did not update more than a week), and uncompressed
> > > append-only index for incremental. On incremental update client sends
> > > request to server "give me append file file starting from position XXX
> > > till the end of file", so only delta from the last update will be
> > > fetched. Then pacman applies the delta to client index.
> > >
> > > What do you think about this idea?
> > 
> > Ok, found pacman.conf option "UseDelta" that seems does what I need.
> > BTW why it is not enabled by default? Isn't saving bandwidth a good
> > thing?
> 
> I'm not sure whether "UseDelta" also affects package lists (rather than
> packages only). Arch Linux does not support deltas on the server side
> yet, see FS#18590 [1] for details.
> 
> [1] https://bugs.archlinux.org/task/18590

UseDelta only affects packages. It's simply not feasible to create
deltas for the sync DBs as they exist today, especially over HTTP/FTP.