[arch-dev-public] tar based sync backend implemented!
allan at archlinux.org
Sun Oct 10 01:49:09 EDT 2010
I have implemented the reading directly from the db tarball for sync
databases. See the patchset on by "backend" branch
Some of the changes are quite big (e.g. 9 files changed, 2184
insertions, 1571 deletions), but that is mainly moving and duplicating
code in the process of splitting the handling of local and sync dbs.
There is still _a lot_ of clean-up to be done in the code, but it is
This breaks pactests... lots of pactests... but that is expected as the
pactest suite does not understand a tar based sync db. I would like
help fixing that as I get lost in the pactest code.
Anyway, here is the timing for some operations. First time is with
dropped caches, the second running it again with the db cached.
pacman -Syy (download time ~15-20sec)
old: 1m25.197s, 0m22.445s
new: 0m19.391s, 0m22.084s (download speed variation...)
pacman -Sl extra
old: 0m6.973s, 0m0.193s
new: 0m3.974s, 0m0.390s
pacman -Ss pacman
old: 0m11.521s, 0m0.395s
new: 0m4.171s, 0m0.477s
pacman -Si pacman
old: 0m6.067s, 0m0.101s
new: 0m1.929s, 0m0.437s
echo "n" | pacman -Su (five packages to update)
old: 0m17.823s, 0m0.306s
new: 0m10.577s, 0m0.489s
So the tar backend gets some quite significant speed gains in the
uncached case, and has marginal losses when the db is cached.
Note that the entire sync db is read and parsed no matter the (sync)
operation. That explains the difference in the cases with the db
cached. This could be adjusted to read the package list (pkgname,
pkgver-pkgrel) first and read the desc/depends files for the entire db
or a single package as needed but that is probably distant future stuff...
More information about the arch-dev-public