[pacman-dev] [PATCH 3/5] repo-add : rework of delta support

[Xavier]

On Thu, Feb 26, 2009 at 10:25 AM, Allan McRae <allan at archlinux.org> wrote:
>
> I think deciding when they should be removed is the easiest part here:
> 1) when you can not get from a delta to the current package, it should be
> removed
> 2) when getting from a delta to the current package requires more download
> than just downloading the current package (or whatever criterion pacman uses
> - 80%?), it should be deleted.  Perhaps this should be a separate script, or
> maybe a flag (repo-add --cleandelta)?
>

What is a bit disappointing is that this means reimplementing the
logic we already have in pacman, but well.
Btw, the delta sizes could not be more variable. The full range is
obtained, just using my not so big cache : 0% - 100% (of the newest
package)

I stole and hacked a script from the forums to generate the numbers I wanted :
http://bbs.archlinux.org/viewtopic.php?pid=433730#p433730
Thanks sabooky :)

0% : ghostscript-8.64-2_to_8.64-3-x86_64.delta
0% : ttf-bitstream-vera-1.10-5_to_1.10-6-x86_64.delta
0% : sound-theme-freedesktop-0.1-1_to_0.2-1-x86_64.delta
0% : libmad-0.15.1b-3_to_0.15.1b-4-x86_64.delta
0% : tdb-3.3.0-1_to_3.3.1-1-x86_64.delta
1% : libvorbis-1.2.1rc1-1_to_1.2.1rc1-2-x86_64.delta
2% : gnome-power-manager-2.24.2-2_to_2.24.4-1-x86_64.delta
2% : teeworlds-0.5.1-1_to_0.5.1-2-x86_64.delta
3% : ghostscript-8.64-1_to_8.64-2-x86_64.delta
3% : xextproto-7.0.4-1_to_7.0.5-1-x86_64.delta
4% : openjdk6-1.4-2_to_1.4.1-1-x86_64.delta
4% : xcb-proto-1.3-1_to_1.4-1-x86_64.delta
7% : tzdata-2009b-1_to_2009a-1-x86_64.delta
7% : alsa-utils-1.0.18-1_to_1.0.19-1-x86_64.delta
7% : groff-1.20.1-1_to_1.20.1-2-x86_64.delta
8% : xcb-util-0.3.2-1_to_0.3.3-1-x86_64.delta
8% : firefox-3.0.5-1_to_3.0.6-1-x86_64.delta
8% : pm-utils-1.2.3-4_to_1.2.4-1-x86_64.delta
9% : kernel26-2.6.28.3-1_to_2.6.28.4-1-x86_64.delta
9% : kernel26-2.6.28.5-1_to_2.6.28.6-1-x86_64.delta
9% : kernel26-2.6.28.6-1_to_2.6.28.7-1-x86_64.delta
10% : libxcb-1.1.93-1_to_1.2-1-x86_64.delta
10% : libxi-1.1.4-1_to_1.1.4-2-x86_64.delta
11% : kernel26-2.6.28.4-1_to_2.6.28.5-1-x86_64.delta
12% : xkeyboard-config-1.4-2_to_1.5-1-x86_64.delta
13% : xcb-proto-1.2-2_to_1.3-1-x86_64.delta
14% : libdrm-2.3.1-2_to_2.3.1-3-x86_64.delta
16% : gimp-2.6.4-2_to_2.6.5-1-x86_64.delta
16% : inputproto-1.4.4-1_to_1.5.0-1-x86_64.delta
18% : bluez-4.29-1_to_4.30-1-x86_64.delta
20% : evolution-data-server-2.24.3-1_to_2.24.4.1-1-x86_64.delta
23% : libxml2-2.7.3-1.1_to_2.7.2-1-x86_64.delta
24% : pciutils-3.0.3-1_to_3.1.2-1-x86_64.delta
27% : ntfs-3g-2009.1.1-1_to_2009.2.1-1-x86_64.delta
27% : flashplugin-10.0.d21.1-1_to_10.0.22.87-1-x86_64.delta
29% : libv4l-0.5.7-1_to_0.5.8-1-x86_64.delta
29% : gnutls-2.6.3-1_to_2.6.4-1-x86_64.delta
29% : man-pages-3.17-1_to_3.18-1-x86_64.delta
31% : ruby-1.8.7_p72-2_to_1.8.7_p72-3-x86_64.delta
33% : device-mapper-1.02.29-1_to_1.02.30-1-x86_64.delta
34% : libpng-1.2.34-1_to_1.2.35-1-x86_64.delta
37% : libx11-1.1.5-2_to_1.1.99.2-2-x86_64.delta
38% : gpm-1.20.5-2_to_1.20.6-1-x86_64.delta
40% : dhcpcd-4.0.7-1_to_4.0.10-1-x86_64.delta
41% : libx11-1.1.99.2-2_to_1.2-1-x86_64.delta
41% : xorg-xinit-1.1.0-1_to_1.1.1-1-x86_64.delta
42% : pixman-0.12.0-1_to_0.14.0-1-x86_64.delta
42% : hal-0.5.11-4_to_0.5.11-7-x86_64.delta
42% : tdb-3.2.7-1_to_3.3.0-1-x86_64.delta
46% : libxfont-1.3.4-1_to_1.4.0-1-x86_64.delta
46% : gsynaptics-0.9.14-2_to_0.9.15-1-x86_64.delta
49% : libmad-0.15.1b-2_to_0.15.1b-3-x86_64.delta
49% : xf86-input-evdev-2.1.0-1_to_2.1.2-1-x86_64.delta
49% : whois-4.7.28-2_to_4.7.30-1-x86_64.delta
49% : sqlite3-3.6.10-1_to_3.6.11-1-x86_64.delta
50% : gnome-mplayer-0.9.3-1_to_0.9.4-1-x86_64.delta
50% : perl-error-0.17011-1_to_0.17015-1-x86_64.delta
51% : xf86-input-synaptics-0.99.3-1_to_1.0.0-1-x86_64.delta
53% : man-db-2.5.2-2_to_2.5.4-1-x86_64.delta
54% : libxcb-1.1.90.1-1_to_1.1.93-1-x86_64.delta
56% : alsa-lib-1.0.18-1_to_1.0.19-1-x86_64.delta
56% : xf86-input-keyboard-1.3.1-1_to_1.3.2-1-x86_64.delta
58% : libtasn1-1.7-1_to_1.8-1-x86_64.delta
58% : lvm2-2.02.43-1_to_2.02.44-1-x86_64.delta
58% : xterm-239-1_to_241-1-x86_64.delta
59% : xfsprogs-2.10.2-1_to_3.0.0-1-x86_64.delta
60% : dnsutils-9.5.0.P2-1_to_9.6.0.P1-1-x86_64.delta
62% : libxi-1.1.4-2_to_1.2.0-1-x86_64.delta
65% : libxslt-1.1.24-1_to_1.1.24-2-x86_64.delta
65% : hdparm-9.6-1_to_9.10-1-x86_64.delta
67% : libcanberra-0.10-1_to_0.11-2-x86_64.delta
69% : libvorbis-1.2.1rc1-2_to_1.2.0-1-x86_64.delta
73% : smbclient-3.3.0-1_to_3.3.1-1-x86_64.delta
80% : grep-2.5.3-3_to_2.5.4-1-x86_64.delta
83% : smbclient-3.2.7-1_to_3.3.0-1-x86_64.delta
86% : libxext-1.0.4-1_to_1.0.5-1-x86_64.delta
88% : libice-1.0.4-1_to_1.0.5-1-x86_64.delta
89% : faac-1.26-1_to_1.28-1-x86_64.delta
96% : xulrunner-1.9.0.5-2_to_1.9.0.6-1-x86_64.delta
96% : gtk-engine-murrine-0.53.1-1_to_0.53.1-3-x86_64.delta
98% : libcroco-0.6.1-1_to_0.6.2-1-x86_64.delta
100% : pacman-git-20090226-1_to_20090227-2-x86_64.delta
101% : cabextract-1.2-1_to_1.2-2-x86_64.delta

That's right, 101%, no error here. The delta is 32427 B while the
package is 32045 B :D

Since pacman indeed has a max ratio (#define MAX_DELTA_RATIO 0.7), it
doesn't make any sense to put deltas bigger than 70% in a database,
because pacman will never use them.
Do we enforce this check somewhere? If yes where? In pkgdelta or in
repo-add and/or in an external cleanup script?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: generate-deltas.py
Type: application/octet-stream
Size: 2000 bytes
Desc: not available
URL: <http://www.archlinux.org/pipermail/pacman-dev/attachments/20090301/f33cf33c/attachment.obj>