[arch-dev-public] [RFC] Add ARM to archlinux.org

[Dave Reisner]

On Tue, Mar 25, 2014 at 12:55:51AM +0100, Sébastien Luttringer wrote:
> On 24/03/2014 23:50, Dave Reisner wrote:
> > On Mon, Mar 24, 2014 at 11:35:36PM +0100, Sébastien Luttringer wrote:
> > Does 2013 cover the whole year? 
> Four full months. More details in listing this page[1]
> 
> > You're suggesting that 2014 will occupy
> > over 200GB. We'd need new infrastructure for this, and it comes with a
> > monetary cost we'd need to accomodate (or are you proposing that you'll
> > be paying for this forever?). 
> I don't suggest to pay forever, one of the benefits of moving this, is
> to stop rely on one guy. We already suffer of a previous shutdown on
> this service.

Right, but throwing this responsibility on the arms of Arch developers
isn't really a solution, either, without quiescence. You seem to be of
the mindset of that adding this to the archlinux.org domain will
magically lighten your own sysadmin load.

While I think the idea is neat, I don't think is has a place underneath
archlinux.org -- we'd be taking a step towards supporting partial
upgrades (which we refuse to support everywhere else). Before you
mention the AUR, remember that the AUR hosts source packages; not binary
packages.

> > Have you looked into how much this would
> > be per month with a potential provider? How much bandwidth is associated
> > with this? 
> The current server cost is 16.99€/month excl. VAT with 2x1TB HDD with
> 1Gbit/s connectivity, 200Mbit/s internet bandwidth and unlimited traffic.
> 
> A cheap solution based on a dedibox classic[2] with 2x1TB, 1 Gbit/s con,
> 150Mbit/s internet and unlimited traffic for 29.99€/month excl. VAT.
> should be sufficient.
> 
> A more future proof solution may based on a dedibox pro[3] with 2x3TB
> drives, with 400Mbit/s internet bandwidth and unlimited traffic for
> 69€/month excl. VAT.

And the SLA? This all seems reasonable at a glance, but I can't speak to
how this would impact our budgeting.

> Of course, any Hetzner alternative may fit, but traffic limitation may
> cause extra cost that we don't have with online.net offers.
> 
> > How long would packages be retained? What about resource
> > planning for future growth?
> My suggestion would to keep them as far as we can, until costs is not a
> showstopper. With a 1TB server and 300GB by year, we can keep ~3 years.
> 
> >> - add AUR
> > 
> > This doesn't make sense. We already have a git repo which does a far
> > better job of compressing the deltas between versions of text files.
> Keeping the git backend or not, it makes sense to me to move this from
> the build server (pkgbuild.com) to the archive server (something like
> archive.archlinux.org).

Sure, I agree with this. Our serving structure is pretty strange and it
doesn't make sense to host the repo on the build server for any other
reason than: "this is where we had unallocated resources."

> Git is an awesome SCM, but it was not think to backup stuff, especially

You lost me here. git is made up entirely of deltas and serves as a
distributed backup. It's *really* *good* at this, even on wide trees.

> with a big directory tree. My feeling trying to find a previous version
> of a PKGBUILD with aur-mirror.git is:
> - a very long time to fetch the repository
> - a space based dig into changelog to find the right commit for my date
> - a long checkout to get the tree at this commit

Seems like this is just three different ways to spin the same point. We
can restructure to repo to make it more clone-friendly if that's all
that's needed.

> Having something with faster access and in a similar hierarchy may have
> use cases.

How often do you use the git repo?

> >> - backup them (by mirroring or others)
> > 
> > There's going to be a large cost here and I doubt it's any potential
> > benefit. Do you get complaints/requests from your users to mirror your
> > implementation?
> No extra cost is expected for mirroring, which is a kind of distributed
> backup. I see at least 2 potentials benefits for mirroring:

Are you really suggesting that mirroring 300GB-3TB of data has no
cost? Who's hosting the mirrors? Have they agreed to shoulder the
additional storage and bandwidth requirements?

> - Better latency around the world.
> - Get our data back in case of disaster recovery.
> 
> Of course, I suggest to keep these mirrors separate from our packages
> mirrors. Purpose and HW requirement are not the same.
> 
> The only complain I get is because this service is not official and rely
> on my health.
> I had 3 requests:
> - One guy for mirroring because of latency.
> - One from ArchARM asking to add ARM to their project :)
> - One from a guy asking to add AUR support.

Do you have any metrics to show how many 30 day users you have?
Downloads per day? Week? Bandwidth consumed? I'm really interested in
knowing how much impact this actually has....

d