Hello Mirror Admins,
the DevOps team [wants to split][0] the repo server and the archive server, which are currently still located on the same box (gemini.archlinux.org).
The server in question is currently crumbeling under the load the I/O intensive tasks (such as btrfs scrub, hardlinking service, tons of rsync processes at the same time) are putting on it and is not well equipped to handle these loads as it has still regular HDDs (aka spinning rust) in it. This is of course great from a storage capacity perspective but not good for the peak performance scenarios mentioned before.
Our plan is therefore to split out the repo server to a new one with SSDs and use the current server for archiving purposes only going forwad.
We have drafted a rough plan on how we imagine this transition to go:
1. **Phase 1:** We will setup the new server and have the domain "rsync.archlinux.org" point to it, since this is the domain used by the T1 Mirrors. At this point the box itself is syncing from the "old" repo server.
2. **Phase 2:** After a grace period of 2 week the rsync deamon on the old box is shut off. This can possibly create problems as the mirrors might have hardcoded the IP, hence the afforementioned grace period.
3. **Phase 3:** We make the new box the cannonical home of our repo services. This change should not really be visible from outside and mostly concerns our packagers, which will be notified separately.
We will update this thread with a schedule soon!
It is important for us to make this process transparent to you because we recognize the effort it can take to run a mirror and don't want to make life hard for you. So if you see any problems with the plans outlined above feel free to let us know!
Thanks for being awesome! 🎉
Cheers,
Kristian and Christian
from the Arch DevOps Team
[0]: https://gitlab.archlinux.org/archlinux/infrastructure/-/issues/531