[pacman-dev] Repository management

[Allan McRae]

Hi all,

Every time I attempt to work on repo-add, I find it to be a very
difficult endeavour.  Even though it is half the size of makepkg
(without even including any of libmakepkg), it is much more convoluted
to work on.

We also have a weird repository database system.  We have:
- .db dbs with package information, signatures and delta information
- .files dbs that are the same as .db dbs but additionally include filelists

There are two reasons the .files dbs replicate all information in the
.db dbs
 - .db and .files dbs getting out of sync could cause issues
 - a complete database is useful for things like archweb, mostly to
avoid the above

I would also like to include information on source packages to these
databases.  The files information is separate due to wanting our primary
database to be small.  Likewise, source package information needs to be
separate (the signatures take most of the size in the .db dbs, so adding
source package signatures effectively doubles the size).

So two points up for discussion:


1) Sync repository layout?  I don't see any point in leaving the tar
based format, as reading of sync databases is not a bottleneck.  (The
local db format can be a bottleneck, but that is a separate discussion...)

Do we split the information in .db out of .files and add a .full db with
complete information?  Then any .src db could follow suit and just have
source package information.  How do we get around the out of sync issue
(e.g., a package is removed from .db, but we have an old .files database
with it).  Do we add timestamps, and print a warning on -F operations
when the two are out of sync?


2) Do we need a better (read "more easily maintainable") tool for
handling database generation and updates?  libalpm already can read in
information package files, so we could add libalpm/db_write.c with the
database creation functions.   Should we unify our repo format with our
local database format which we already write?


I am looking for ideas here.  Please brainstorm to your hearts content.

Cheers,
Allan