[pacman-dev] Tar backend for local db
allan at archlinux.org
Thu Nov 1 07:24:13 EDT 2012
On 01/11/12 15:12, Allan McRae wrote:
> I was thinking about the local database backend in pacman and how we
> could improve it.
> The tar-based backed for sync dbs have been quite a success. Ever since
> we did that, I have been wanting to do the same with the local database.
> But there were two issues:
> 1) tar is not designed for removing and updating files in place
> 2) with a directory/files per package structure, we are quite robust to
> corruption as a single file effects only a single package.
> Well... I have a cunning plan... How about we do both!
> Have the local database in a tarball but also extracted. All reading is
> done from the tarball, so -Q operations would be fast due to not require
> reading from lots of small files. With an operation that
> adds/removes/updates a package, the local database is still read from
> the tarball, but the modifications are done on the files backend and
> then the tarball is recreated at the end. We could even be efficient
> during the recreation and read all the old files from the old tarball
> and only read the new files from the filesystem (which will be in the
> kernel cache...).
> This would also give another use for "pacman -D" - an option could be
> added to recreate the local db tarball - in case it became corrupt or
> the files were manually edited.
> What do people think?
Just to be clear, I wanted comments on the dual local database (one
"binary", one filesystem based) would be a good solution to increase
read speed (due to not having many small files), but also keeping the
robustness of the non-binary format to corruption. I.e would the extra
10-20MB be an acceptable trade-off.
Given I have absolutely no interest in using sqlite, bdb, etc... and I
can almost guarantee that I will be the one to provide a patchset that
changes the local backend, comments about choosing a relational database
are not needed.
More information about the pacman-dev