[arch-dev-public] automatic package builder

[Ronald van Haren]

On Sat, Sep 26, 2009 at 10:35 PM, Aaron Griffin <aaronmgriffin at gmail.com>wrote:

> On Fri, Sep 25, 2009 at 6:17 PM, Allan McRae <allan at archlinux.org> wrote:
> > Aaron Griffin wrote:
> >>
> >> On Fri, Sep 25, 2009 at 4:01 AM, Xavier <shiningxc at gmail.com> wrote:
> >>
> >>>
> >>> By the way, I am curious about pacbuild :
> >>> http://wiki.archlinux.org/index.php/Pacbuild_Explained
> >>> What do people think about it ?
> >>>
> >>
> >> Honest opinion? It was over-engineered from the start and is far to
> >> complex to use. You need like 3 daemons running, and have to do manual
> >> database insertions (why does it have a database?!) to configure
> >> things.
> >>
> >> - - - - -
> >>
> >> makechrootpkg was made in such a way that is can and should be used
> >> for this purpose. Here's how I would setup a "build machine".
> >>
> >> a) Create two global chroots, one for "current" and one for "testing".
> >> We'll call them /var/archroot and /var/archroot-testing (alternative:
> >> append arch on there and create 4 total. Requires a 64bit machine and
> >> linux32 usage)
> >>
> >> b) Scan svn commits SOMEHOW. This could be done by watching the
> >> arch-commits list, or simply running "svn up" in a job and parse
> >> changes
> >>
> >> c) For each trunk commit found, do an svn export of the trunk and run
> >> "makechrootpkg -c -u -r/var/archroot -l $pkgname"
> >>
> >
> > This is the part I really do not like about this.  I commit to trunk when
> > 1) I have updated, _built_ and am in the process of committing a package.
> > 2) I have an idea/bug fix for the next release and do not want to make a
> new
> > release now.
> >
> > In both those cases, a commit to trunk does not need a build.  In all
> > honesty, I would think that having  downloaded the source and updated the
> > PKGBUILD, you would probably have built the package locally anyway to
> test
> > it all out.  So building for #1 seems annoying as I would then need to
> > download the built package and retest etc, or just not use it...
> >
> > How about, to build a package with the build bot, you send an email to
> > buildbot at archlinux with content "pkgname carch", and if it is from an
> > authorized user, the buildbot starts.  Multiple lines = multiple builds?
> >  Needs more thought...
> >
> >
> > Anyway, what exactly are the goals of the buildbot? I see these uses:
> > 1) A developer needs to build packages for an arch they do not have
> > 2) Helping out with big rebuilds
> >
> > #1 does not need to monitor trunk as one arch would already be built and
> > tested by the developer and it probably should not start automatically as
> > who commits both arches at the same time...  For #2, you need to provide
> a
> > tree with the dependency chain anyway, and I have yet to strike a rebuild
> > that did not require manual intervention for fixing PKGBUILDs (build
> fixing
> > patching, update versioned deps).
> >
> >
> > So this is what I would like to see set-up:
> > 1) A way to build individual (or a small group of) packages on request
> (e.g.
> > a dev does not have access to an arch)
> > 2) A way to handle major rebuilds, including generating a build order (I
> > have a script that somewhat generates a build order and Aaron started a
> > script to do the building)
> > 3) When there are no packages queued to be built, check packages in our
> > repos for being able to build in a clean chroot.
> >
> > I started to write a daemon to do #3 but decided my approach was bad so
> > scrapped most of it.  It really needed written in a language other than
> bash
> > to be useful (needs to store all info about last build time and success
> in a
> > database and generate web reports).  From doing many large rebuilds, I
> see
> > #3 as being essential to successfully doing #2.
>
> The building from trunk thing is more based on my workflow, but I
> really don't maintain much these days, so you can ignore it.
>
> Here's why I was thinking that way: when I'm at work and see that some
> package needs something changed, I just update svn considering I'm not
> on an Arch machine. I don't actually do the build, I just make changes
> and commit them.
>
> Now, your idea with some way to poke the thing to actually do a build
> (via email, a web page, command line app, etc) would still work here,
> so it sounds like a better idea.
>

both getting changes from trunk or getting build packages on request should
be pretty easy to implement.

My initial thought was building from requests and, when no requests
available somehow determine what packages from core/extra may be good to
have a rebuild. I also would like to be able to add priority flags (high,
normal, low, default to normal) so that we can add packages in front of the
queue if needed sometime.
If no requests are pending, we should do something else usefull, like
building old packages against new toolchains and such, but there may be
other more important variables (we really should define this better). We
should probably also automatically bump the pkgver in these cases.

I'd like to have some more dev input on what some of the others like to see
so everyone can be happy in the end.

Ronald
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.archlinux.org/pipermail/arch-dev-public/attachments/20090927/a1e77ebb/attachment.html>