[pacman-dev] [PATCH] Added a new database backend

[Nagy Gabor]

> This is an attempt to add a one-file-per-database backend. This leads to
> significatly faster loading of the database when the database is not
> cached. As you can see from the patch minimal changes to the rest of
> ALPM was needed. The only change was to make reading of the changelog
> files happen through the backend. The database is basically a scaled
> down ext2-like filesystem. To make corruption detection possible in an
> early phase I have planned to add a checksum, but have not come that far
> yet. I have been using it on my laptop for a week or so now, and
> everything seems to be working as expected. I'm sorry it became such a
> large patch, but about half of it is mostly old code (be_packed.c is
> essentially be_files.c with fopen fprintf etc changed with new
> functions).
> 
> Signed-off-by: Sivert Berg <sivertb at stud.ntnu.no>

First of all, many thanks for your huge work. I will support your
attempt on reworking our db backend stuff. I haven't completely reviewed
your patch yet (it is quite advanced, I guess I will learn a lot from
here), but I have some "pre-reading" comments. (Keep in my mind: I am
just a pacman contributor here.)

1. I like your modification on changelog reading (package.c should not
assume anything about db backend), you may create a separate patch for
this. (We prefer shorter patches.) But we have the same problem with
changelog creation (extraction), which is hardcoded in add.c. And the
same for install scriptlets. The implementation of these fields is quite
hackish, we don't use pmpkg_t and db_write here...

> diff --git a/lib/libalpm/Makefile.am b/lib/libalpm/Makefile.am
> index 871855e..79b9d26 100644
> --- a/lib/libalpm/Makefile.am
> +++ b/lib/libalpm/Makefile.am
> @@ -25,7 +25,7 @@ libalpm_la_SOURCES = \
>  	alpm.h alpm.c \
>  	alpm_list.h alpm_list.c \
>  	backup.h backup.c \
> -	be_files.c \
> +	be_ at BACKEND@.c \
>  	be_package.c \
>  	cache.h cache.c \
>  	conflict.h conflict.c \

2. I think we need a more sophisticated approach here. I would be happy,
if we had database backend plugins (be_files.so, be_packed.so,
be_sqlite.so, ...) for dlopen(), and some way to inform pacman about the
database handler (for example: /var/lib/pacman/local/.backend, which has
one line: packed). I think this is quite straightforward (but needed) to
implement.

3.* Our whole db back-end system needs some redesign (independent from
your work): If we have fast database back-end, I am not sure we need
this ugly pkgcache stuff. Probably pkgcache should be moved to back-end
level (if needed).

4.* When we rework pmpkg_t, we must keep in mind, that at this moment it
is the structure of database entry *and* package file. When we
restructure things, we must find a good place for ".tar.gz".

(* == difficult .-)

Bye