[arch-dev-public] New /arch/db-* scripts

[James Rayner]

On Sun, July 8, 2007 19:39, Thomas Bächler wrote:
> I started writing new database scripts to replace the old
> db-*/updatesync-many/pkgdb2 scripts. The reason are support for the new
> pacman 3 naming scheme and fixing some design issues.
>
> The old scripts use PKGBUILDs a lot for their operation and don't check
> the package file for consistency with the PKGBUILD (only the filename).
> They also move every file from the staging dir, regardless of whether
> the package will be added in any repository. And they recreate much of
> the functionality from updatesync, sort of reinventing the wheel.
>
> My new draft has a cleaner design, but some new problems appear. It
> performs the following steps:
>
> 1) Check every file in the staging/add dir using a small libalpm-based
> tool and obtain the pkgname, pkgver, pkgrel and architecture. Compare
> the arch specified in the commandline with the arch from the package
> (this step is missing in the old scripts). Find the PKGBUILD and compare
> pkgname, pkgver and pkgrel with the values from the package. If all
> checks are okay, add the package to a list. If additionally, a force
> flag is set in the PKGBUILD, add it to a "force-list".
>
> 2) Check every file in the staging/del dir and obtain its pkgname. Add
> this package to a delete list.
>
> 3) Lock the database
>
> 4) Move all packages and force-packages to the ftp dir, add them with
> repo-add.
>
> 5) Pass the package files to a pkgdb2-like tool to add them to the web
> interface.
>
> 6) Delete all package from the delete-list with repo-remove.
>
> 7) Pass the package names to a pkgdb2-like tool to remove them from the
> web interface.
>
> 8) Release the database lock.
>
> The problems I run into are these:
>
> a) In step 4) I can't determinte the filename of the old package and
> remove it from the ftp. I could scan for certain filename schemes, but
> take into account that the package filename could be anything now, the
> script doesn't care. We could however rely on our current filename
> scheme to find/remove the dupes (who would go through the trouble to
> rename his package to sth like wrongname-notapackage.zip.bz8 anyway?).
>
> b) The same thing counts for step 6). But we mv our packages to the
> staging/del anyway so the script won't have to remove them from the ftp.
>
> c) This is the biggest problem currently. In step 5), due to the new
> script design, I don't have any data from the PKGBUILD any more and
> don't want to go find it again. That means I only use data from the
> package file itself to add it to the mysql db. The package file lacks
> the "package category" and "source" which are in the web interface. I
> could only solve this by either removing this data from the web
> interface or adding it to the package file.

IIRC, you also need the PKGBUILD for force=y. That might have been fixed
in pacman3.

James