[pacman-dev] Tar backend for local db
rodseth at gmail.com
Thu Nov 1 07:50:52 EDT 2012
@Thomas, I see, thanks.
@Allan, Very well, choose what you want, just wanted to comment on the
use of tar vs sqlite on a general basis.
2012/11/1 Allan McRae <allan at archlinux.org>:
> On 01/11/12 15:12, Allan McRae wrote:
>> I was thinking about the local database backend in pacman and how we
>> could improve it.
>> The tar-based backed for sync dbs have been quite a success. Ever since
>> we did that, I have been wanting to do the same with the local database.
>> But there were two issues:
>> 1) tar is not designed for removing and updating files in place
>> 2) with a directory/files per package structure, we are quite robust to
>> corruption as a single file effects only a single package.
>> Well... I have a cunning plan... How about we do both!
>> Have the local database in a tarball but also extracted. All reading is
>> done from the tarball, so -Q operations would be fast due to not require
>> reading from lots of small files. With an operation that
>> adds/removes/updates a package, the local database is still read from
>> the tarball, but the modifications are done on the files backend and
>> then the tarball is recreated at the end. We could even be efficient
>> during the recreation and read all the old files from the old tarball
>> and only read the new files from the filesystem (which will be in the
>> kernel cache...).
>> This would also give another use for "pacman -D" - an option could be
>> added to recreate the local db tarball - in case it became corrupt or
>> the files were manually edited.
>> What do people think?
> Just to be clear, I wanted comments on the dual local database (one
> "binary", one filesystem based) would be a good solution to increase
> read speed (due to not having many small files), but also keeping the
> robustness of the non-binary format to corruption. I.e would the extra
> 10-20MB be an acceptable trade-off.
> Given I have absolutely no interest in using sqlite, bdb, etc... and I
> can almost guarantee that I will be the one to provide a patchset that
> changes the local backend, comments about choosing a relational database
> are not needed.
xyproto / trontonic / TU
More information about the pacman-dev