[arch-devops] Mirrorbrain

[Florian Pritz]

On Wed, Sep 12, 2018 at 03:17:39PM +1000, Phillip Smith via arch-devops <arch-devops at lists.archlinux.org> wrote:
> 1. Ensuring users always receive up-to-date packages (mirrorbrain won't
> redirect to a mirror if that mirrors version of that package is outdated
> compared to the authoritative repository).

I can see the appeal since we don't have that now.

> 2. Automated monitoring of our mirror network to proactively detect stale
> or broken mirrors. Mirrorlist files (ie, /etc/pacman.d/mirrorlist) can be
> automatically generated based on MirrorBrain's data.

While nice, we already have mirror list generation from monitored data
directly in archweb if you check the "Use mirror status" option here:
https://www.archlinux.org/mirrorlist/

> 3. Reduced load on core mirrors, and load-balancing (for example, a 1Gbit
> mirror can be weighted to receive 10x the traffic of a 100MBps mirror).

We already have `rankmirrors` which selects mirrors based on their
performance from a client's point of view (I think download speed and
response time). Though, on its own, it won't detect new mirrors if we
add them and the users don't update their lists. I don't really know if
new mirror are actually getting any use or not right now, so I can't tell
if that is an issue or not.

> 4. Automatic MetaLink and Torrent file generation, with web-seeds
> (currently handled by hefur on luna?).

I think that's done manually by Pierre when he creates a new ISO. I'm
not 100% sure though.

I don't know if we really want or need mirrorbrain. It's a great fit for
projects with few big files which are downloaded by many people upon
a new release. If we were to use mirrorbrain, the pacman mirror list
would only contain the redirector URL, correct? That would then mean
that pacman asks the redirector for each individual package which I
think generates way too much unnecessary load compared to our current
system. Also, right now people can update without having to connect to a
central server and I don't like the idea of adding a SPOF for something
that already works rather well.

Mirrorbrain sounds fine for our ISOs, but I don't know if those get
enough downloads (via HTTP) to warrant setting it up.

Florian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.archlinux.org/pipermail/arch-devops/attachments/20180912/7d75e3f0/attachment.asc>