[pacman-dev] Tar backend for local db

[Allan McRae]

On 01/11/12 19:41, Alexander Rødseth wrote:
> Hi,
> 
> I would recommend sqlite. It is really fast (faster than mysql and
> postgresql, when we're not talking about huge datasets), and just a
> small dependency to add. It's also used by several other projects, so
> it's likely that it would be installed as a dependency on a system
> sooner or later. My previous workplace used tar as a sort of database
> format for geological files, and tar files only worked okay, not
> great. When making a program read tar files directly from within a
> program, instead of extracting it first with the tar utility, it's
> cumbersome, and not KISS. Sqlite is well tested, and used for several
> projects (for instance firefox, not that firefox is a good example of
> speedy execution times). Also, one could argue that sqlite is a better
> fit for pacman, since it's like it was made to do this job, rather
> than tar, a tool that was originally made for making backups to tapes.

The thing you are missing is EVERYTHING in pacman apart from the local
db is in tar format.  The packages are tarballs, the sync dbs are
tarballs.   We use a very good library for reading from tarballs without
extracting them.

> I agree with Klaus-J. Wolf here, and am convinced that you would
> decrease the time it takes more by sqlite than with tar.

>From memory, when we implemented reading the sync db from the tarball,
the actually time required to read it was measured as a fraction of a
second.  It is all the other stuff that takes the time.  I have never
been convinced that adding the extra dependency is worth the minor savings.

Allan