On Sun, 29 Aug 2004 11:35:29 -0700, Jason Chu jason@archlinux.org wrote:

Here is my explanation and my current status with respect to pacbuild.

I've been toying with the idea of an autobuilder for a while now, but only implemented it in the last 4 months. As it stands, I use it to keep i586 relatively up to date.

Right now it uses python, postgresql, and user mode linux (UML). It's broken into three parts:

1. waka
2. catchup
3. checkbuilds

Waka is the actual building daemon. It starts UML sessions using copy on write (COW) files, so that I can just throw away the build session after it's done.

Catchup scans /var/abs looking for builds that are newer than the ones in the currently defined repo.

Checkbuilds gives you the logs from built packages so they can be reviewed and marked as correct or not. If they are incorrect, they are marked in the db as incorrect and there is no attempt at building the same version again. If they are correct, they're added to the repo, old packages are removed and the db is updated. Rarely do I have to mark a build as incorrect, because most of them actually fail and are detected as such.

As it stands, I have run into a few limitations of my current implementation:

• distributed building with multiple wakas on remote machines
• provides support
• no support for extra scripts (such as a generated packages.txt needed for

the current repo)

Just glancing at the code, I might actually be able to add provides support fairly easily, to the catchup script. I'd have more trouble getting proper provides working within waka and makepkg.

Distributed building is more difficult. My initial implementation uses the filesystem to store builds and pass around files and whatnot. I really don't know if I want to go to a db-storage solution or something else entirely. I would have to abstract the building and the managing aspects a bit more to make it work better. This will be something that needs a lot of thinking and designing.

The extra scripts support will probably come along when I actually implement support for multiple repos. In my first version, only one repo is supported, that's one of the reasons only current is being built.

I'm not exactly sure why I'm explaining this, I don't know exactly what you people could do without a lot more explanation...

Jason

This is wonderful, wonderful, wonderful. I was waiting to hear whether you had done anything about this idea, and I see you have been silently slaving away on it for some time ;)

I'd love to test and help out with this, as I was just moaning today about the need for controlled build environments (especially since nasty unwanted dependencies keep creeping into my GNUstep packages). I'm most interested in the Waka portion, as that seems to take care of the most difficult task (the controlled builds). How modular is it? Is it necessary to use this with the other scripts, or can I integrate Waka into something else. How easily can I script it?

Cheers,

Mike Baehr