[arch-dev-public] automatic package builder

[Allan McRae]

Aaron Griffin wrote:
> On Fri, Sep 25, 2009 at 4:01 AM, Xavier <shiningxc at gmail.com> wrote:
>   
>> By the way, I am curious about pacbuild :
>> http://wiki.archlinux.org/index.php/Pacbuild_Explained
>> What do people think about it ?
>>     
>
> Honest opinion? It was over-engineered from the start and is far to
> complex to use. You need like 3 daemons running, and have to do manual
> database insertions (why does it have a database?!) to configure
> things.
>
> - - - - -
>
> makechrootpkg was made in such a way that is can and should be used
> for this purpose. Here's how I would setup a "build machine".
>
> a) Create two global chroots, one for "current" and one for "testing".
> We'll call them /var/archroot and /var/archroot-testing (alternative:
> append arch on there and create 4 total. Requires a 64bit machine and
> linux32 usage)
>
> b) Scan svn commits SOMEHOW. This could be done by watching the
> arch-commits list, or simply running "svn up" in a job and parse
> changes
>
> c) For each trunk commit found, do an svn export of the trunk and run
> "makechrootpkg -c -u -r/var/archroot -l $pkgname"
>   

This is the part I really do not like about this.  I commit to trunk when
1) I have updated, _built_ and am in the process of committing a package.
2) I have an idea/bug fix for the next release and do not want to make a 
new release now.

In both those cases, a commit to trunk does not need a build.  In all 
honesty, I would think that having  downloaded the source and updated 
the PKGBUILD, you would probably have built the package locally anyway 
to test it all out.  So building for #1 seems annoying as I would then 
need to download the built package and retest etc, or just not use it...

How about, to build a package with the build bot, you send an email to 
buildbot at archlinux with content "pkgname carch", and if it is from an 
authorized user, the buildbot starts.  Multiple lines = multiple 
builds?  Needs more thought...


Anyway, what exactly are the goals of the buildbot? I see these uses:
1) A developer needs to build packages for an arch they do not have
2) Helping out with big rebuilds

#1 does not need to monitor trunk as one arch would already be built and 
tested by the developer and it probably should not start automatically 
as who commits both arches at the same time...  For #2, you need to 
provide a tree with the dependency chain anyway, and I have yet to 
strike a rebuild that did not require manual intervention for fixing 
PKGBUILDs (build fixing patching, update versioned deps).


So this is what I would like to see set-up:
1) A way to build individual (or a small group of) packages on request 
(e.g. a dev does not have access to an arch)
2) A way to handle major rebuilds, including generating a build order (I 
have a script that somewhat generates a build order and Aaron started a 
script to do the building)
3) When there are no packages queued to be built, check packages in our 
repos for being able to build in a clean chroot.

I started to write a daemon to do #3 but decided my approach was bad so 
scrapped most of it.  It really needed written in a language other than 
bash to be useful (needs to store all info about last build time and 
success in a database and generate web reports).  From doing many large 
rebuilds, I see #3 as being essential to successfully doing #2.

Allan