[arch-general] Question about automated builder

[Thomas S Hatch]

On Thu, Jan 27, 2011 at 2:03 PM, Thomas Dziedzic <gostrc at gmail.com> wrote:

> On Thu, Jan 27, 2011 at 2:06 PM, Thomas S Hatch <thatch45 at gmail.com>
> wrote:
> > On Thu, Jan 27, 2011 at 12:57 PM, C Anthony Risinger <anthony at extof.me
> >wrote:
> >
> >> On Thu, Jan 27, 2011 at 1:48 PM, Thomas S Hatch <thatch45 at gmail.com>
> >> wrote:
> >> >
> >> > Awesome, I actually have a few servers I will use, and since it will
> be
> >> > distributed, we will be able to use a lot of servers as builders all
> >> > reporting to a single master.
> >>
> >> why all the distributed goodness, then a crippling single master?  it
> >> would not be much more difficult to use 1-to-1 queues-to-server.
> >> nodes that fill up return a redirect, or denial.  a simple DNS scheme
> >> can facilitate auto-discovery... nodes register for an ID under the
> >> domain, any simply move down the list.  TXT records can provide the
> >> lists of root nodes if you really want.  with more work, this can be
> >> made to work through NAT, a la p2-esque, but that's more ambitious.
> >>
> >> > As for the web end, I was thinking of having the web frontend just act
> as
> >> a
> >> > notification area about the queue and the builds, so people could
> check
> >> on
> >> > build stats and download experimental pkgs etc. Then the queue would
> be
> >> > managed via the scms.
> >>
> >> no way man, go big or go home!  web interface is full out AUR
> replacement.
> >>
> >> C Anthony
> >>
> >
> > Hmm, a pure peer setup? My thought on the master has more to do with
> central
> > job control, and the fact that distributing can be easily done, the
> master
> > is a lightwieght server compared tot he builders, and the master will
> just
> > present what needs to be build and pull whats built into repos, this way
> the
> > master can scale out to a ton of builders and all of the decisions about
> who
> > builds is done by the builders talking to each other.
> >
> > Of course.... we could have separate master dedicated to specific repos
> and
> > configurations but share the builder pool, so that we have a simple many
> to
> > many relationship.....
> >
> I prefer the central server idea rather then a purely distributed
> system simply because we can't distribute workload well with a purely
> distributed one. Imagine serving openoffice and libreoffice to one
> server and some python module to another server. At least with the
> central server, we have more control over this.
>

hmmmm, I don't think that that will be a problem, my concern is mostly that
we would have duplicate packages getting built. A single builder will be
able to take hours to build a package and another builder can just spin
through say 50 packages in the meantime, also the individual builders will
be configurable to do multiple builds at once.

But with that said, I am still leaning away from having a set of master
peers,  I am still chewing on the concept, and how it would be best to do
it.