[arch-projects] pacbuild

[Michael Baehr]

On Sun, 29 Aug 2004 11:35:29 -0700, Jason Chu <jason at archlinux.org> wrote:
> Here is my explanation and my current status with respect to pacbuild.
> 
> I've been toying with the idea of an autobuilder for a while now, but only
> implemented it in the last 4 months.  As it stands, I use it to keep i586
> relatively up to date.
> 
> Right now it uses python, postgresql, and user mode linux (UML).  It's
> broken into three parts:
> 1) waka
> 2) catchup
> 2) checkbuilds
> 
> Waka is the actual building daemon.  It starts UML sessions using copy on
> write (COW) files, so that I can just throw away the build session after
> it's done.
> 
> Catchup scans /var/abs looking for builds that are newer than the ones in
> the currently defined repo.
> 
> Checkbuilds gives you the logs from built packages so they can be reviewed
> and marked as correct or not.  If they are incorrect, they are marked in
> the db as incorrect and there is no attempt at building the same version
> again.  If they are correct, they're added to the repo, old packages are
> removed and the db is updated.  Rarely do I have to mark a build as
> incorrect, because most of them actually fail and are detected as such.
> 
> As it stands, I have run into a few limitations of my current
> implementation:
> - distributed building with multiple wakas on remote machines
> - provides support
> - no support for extra scripts (such as a generated packages.txt needed for
> the current repo)
> 
> Just glancing at the code, I might actually be able to add provides support
> fairly easily, to the catchup script.  I'd have more trouble getting proper
> provides working within waka and makepkg.
> 
> Distributed building is more difficult.  My initial implementation uses the
> filesystem to store builds and pass around files and whatnot.  I really
> don't know if I want to go to a db-storage solution or something else
> entirely.  I would have to abstract the building and the managing aspects a
> bit more to make it work better.  This will be something that needs a lot
> of thinking and designing.
> 
> The extra scripts support will probably come along when I actually
> implement support for multiple repos.  In my first version, only one repo
> is supported, that's one of the reasons only current is being built.
> 
> I'm not exactly sure why I'm explaining this, I don't know exactly what you
> people could do without a lot more explanation...
> 
> Jason
> 

This is wonderful, wonderful, wonderful.  I was waiting to hear
whether you had done anything about this idea, and I see you have been
silently slaving away on it for some time ;)

I'd love to test and help out with this, as I was just moaning today
about the need for controlled build environments (especially since
nasty unwanted dependencies keep creeping into my GNUstep packages). 
I'm most interested in the Waka portion, as that seems to take care of
the most difficult task (the controlled builds).  How modular is it? 
Is it necessary to use this with the other scripts, or can I integrate
Waka into something else.  How easily can I script it?

Cheers,

Mike Baehr
-- 
Michael Baehr - Arch Linux GNUstep Geek