[pacman-dev] Tar backend for local db

[keenerd]

On 11/1/12, Allan McRae <allan at archlinux.org> wrote:
> # echo 3 > /proc/sys/vm/drop_caches
> # time pacman -Qi pacman-git
> ...
> real	0m13.242s
>
> From experience with the tar backend for sync databases, this would
> decrease by at least 10 seconds.

>From my experience writing non-libalpm pacman clients, you could
reduce that by 13 seconds if you lazily loaded the local DB.  There is
no reason to walk and parse the data of hundreds of packages when the
user is explicitly asking for one.

One minor quibble (we discussed this elsewhere but I just want to note
it here), GNU tar can delete files from tarballs and add files to
tarballs.  This has three major problems. First, it only works on
uncompressed tarballs.  Second, the deleted space remains in use
(pacman -D --vacuum will be needed).  Third, it only works with magic
in GNU tar and is seemingly not exposed by libarchive [1].  But I
think there is serious potential here.

Allan, I do like the general thrust of everything you are proposing.
It does take an abysmally long time to load the local DB.  But it
seems silly to add what is essentially a union file system to pacman
when there are simpler alternatives.

-Kyle
http://kmkeen.com


[1] Delete/Append in GNU tar
Sources at http://ftp.gnu.org/gnu/tar/tar-1.26.tar.gz
Two files of interest: coverage tests (tests/testsuite line 10460) and
src/delete.c.  I do not understand how it works, but it does work and
there you can find it.

Some things to note.  Deleting a file does not free up space and does
not change the size of the tar file.  But the file will no longer
appear in --list or when extracted.

Appending a file may or may not take up additional space.  It appears
from casual experimentation that GNU tar will reuse space if the
appended file is small enough to fit inside a hole left by deletion.