[arch-dev-public] Important: upcoming dbscripts release

[Eric Bélanger]

On Mon, Aug 16, 2010 at 1:49 PM, Pierre Schmitz <pierre at archlinux.de> wrote:
> Hi all,
>
> as the new dbscripts are quite done I'll send you a list with the most
> important changes. (If you are in a hurry read at least the next
> paragraph)
>
> Incompatible changes:
> * db-update will now update all packages found in your staging dir from
>  all repositories
> * db-move has now a different syntax:
>  "<repo-from> <repo-to> <pkgname|pkgbase> ..."
>  You can move as many packages as you like at once. There is no need
>  to specify an architecture here.
> * The wrapper scripts like testing2core have been removed as they are
>  no longer needed; use db-move instead
> * testing2x now moves both arches at the same time (testing2x64 has
>  been removed)
> * I have also started a user documentation at
>  http://wiki.archlinux.org/index.php/DeveloperWiki:dbscripts
>
> Important changes:
> * implementation of a package pool: all packages are now stored in
>  pool/packages or pool/community directories on the ftp server. The
>  current repository directories like extra/os/i686 only contain
>  symlinks.
>  As a results moving pacakges between repositories is now a matter of
>  altering symlinks and as such very cheap. This should solve the
>  problems with not fully synced mirrors we had in the past when a lot
>  of packages were moved from e.g. testing to extra.
> * Packages are only switched to this new layout if they are updated or
>  moved. This way the migration should be slow but smooth.
> * db-move can now handle multiple packages within one "transaction".
>
> General improvements:
> * unit tests based on shunit2
> * heavy refactoring to share common functions
> * Added more integrity checks which are run before the repositories are
>  touched.
> * rewritten functions for easier and smaller code
> * the scripts can handle other db compressions than gz (but there is no
>  migration path yet)
>
> Note: even though the result is more or less a rewrite there are still
> things that can be improved or new useful features that can be added.
> The tests only cover some basic use cases so we should still have a
> look at it once the new version is deployed.
>
> I think we should deploy these soon (before Allan's big python rebuild
> party) with CLEANUP_DRYRUN enabled during the first days.
>
> Greetings,
>
> Pierre
>
> --
> Pierre Schmitz, https://users.archlinux.de/~pierre
>
>

Hi,

I just notice problems with the sourceballs script.

First , it removes sourceballs for packages that it incorrectly thinks
are no longer in svn. The packages are in svn.  I suppose it is caused
by svn seg faulting.  The sourceballs cleanup script sends the svn
output to /dev/null so I'm not sure but there is several seg fault in
the logs when it creates sourceballs. As the cron job is about to
run/running, the old logsare in
http://dev.archlinux.org/~eric/sourceballslogs/

Another more important problem is that since the dbscripts update, the
sourceballs script has stopped to download sourceballs and to log the
makepkg output in /var/log/sourceballs.

Maybe we should disable the sourceballs cleanup (comment out
$dirname/../misc-scripts/sourceballs-cleanup in cron-jobs/sourceballs)
until the script is fixed.

Eric