[arch-dev-public] New DB scripts

[eliott]

On 5/6/08, Aaron Griffin <aaronmgriffin at gmail.com> wrote:
> Hey all,
>  I'm comfortable enough with the new db scripts to throw them out in the wild.
>
>  So, I've placed them at /arch-new for you to play with. Here are some
>  caveats, however:
>
>  * This will not update the website DB. Use at your discresion. Once
>  these are "approved", then Eliott can move the new-model web changes
>  out and the front end WILL be updated on a cron job.

I just want to say that this is pretty fucking great. Decoupling for the win.
This allows us WAY more flexibility on the web frontend side.
1. We can pull in the db when we need it, refactor, fix, change,
whatever is needed.
2. We could host the website and the sql db on a different box from
the pacman repo if we wanted/needed to.

>  * There will be a warning about ~/staging/${REPO}64/ existing. This is
>  expected. Because we have the arch in the filename, we don't need
>  extra dirs anymore. Don't worry, the scripts handle it gracefully and
>  a new devtools release will do away with pushing to the '64' dirs.
>  Then we can clean up our staging dirs a bit.
>
>  * I didn't fully test removal, because I didn't have anything to
>  remove. If someone has a package to remove, let me know so I can test
>  it properly. If you try, let me know if it fails.
>
>  Neat things:
>
>  * This can now _kinda_ create on-the-fly repos, but I locked it down a
>  tad to prevent abuse. The following pattern should make a brand new
>  repo with no problem:
>    archrelease foobar-i686
>    ssh archlinux.org
>    /arch-new/db-update foobar i686
>
>  Now, this actually requires someone with the right privileges to
>  create /home/ftp/foobar, because I didn't want 400 repos being sent
>  out to all our mirrors.
>
>  * Should be much faster on the svn checkout and all that stuff
>
>  * Uses repo-add which means we get the 5 or 6 DB values that we've
>  been missing since pacman 3.0 was released.

w00t!

>  Now, when I "move these live", the following need to be done at the same time.
>
>  * I need to rebuild ALL dbs with repo-add once, just to get some data
>  for old packages in there.
>
>  * We need to move the newmodel code so the front end is updated
>
>  * We _should_ release a new devtools, for the staging dir changes
>
>  Extra notes:
>
>  * All cleanup scripts should be able to function off of a repo DB
>  file, and be runnable as a cron job. I am going to be setting up
>  Thomas' ftpdir-cleanup script as a cron job which will clean up all
>  package files that are NOT in the db. No more dupe cleaning, yay!
>
>  * packages.txt has been broken since we moved to svn. No one has
>  noticed, I think. Should this be done away with, or should it be
>  adapted as a cron-able task?

Awesome work Aaron.
Progress is good.
Thanks. :)