New Arch Linux RFC - Proposal for new mirror administration routines and mirror requirements
Hi all, This is a new RFC ( Request for comment ) that was put together by the Arch Linux mirror team and is now open for discussion. The proposal better defines what a mirror within Arch Linux is, how it is operated, better, clearer guidelines for becoming a mirror and decommissioning. To view and participate in the discussion, you can check it out here: https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/29/ . For more information about what is an RFC and how it operates you can view the README.rst in the root of the repo: https://gitlab.archlinux.org/archlinux/rfcs. Best regards, Arun Arch Linux mirror team
Hello, I am not a staff member, but I like to read the RFCs and arch news and I have a few things.. Firstly, you got a typo on line 21 [1] and 314 [2], "scarecely doccumented" --> "scarcely documented". Just something I noticed when reading, I wasn't proactively checking. (Unless this is a localisation thing and other countries spell it differently?)
Tier Structure ^^^^^^^^^^^^^^
This RFC outlines a new proposal for the tier model, along side t naming of the different tiers. The new model consists of promoting more decentralized mirrors both by protocol but also by tier levels.
This aims to combat overcrowding regions with high tier mirrors as well as enhance the user experience. Despite it being uplifting to have many high tier mirrors, it defeats the purpose of the tier model slightly. The proposal is therefore to reorganize the current T1 in the regions where there's an abundance of them to T2, and this should not be considered a downgrade - but a sideways migration of duties. The same goes for T2 in overpopulated regions where some will be changed to a T3 status.
The migration will depend on gathered service statistics among > geographical factors where applicable.
An idea, that might already be planned by not explicitly stated is that "regions" expand the country name into "city, country". Something like: "London, UK" or "Berlin, Germany", and if it is a big city maybe even closer, so "North London, UK", this would be especially useful for Tier 3 mirrors where they are designed to be more local and decentralised, possibility of even specifying the town might be cool, so if there is a city with a ton of mirrors you can pick one in your local area. Issue would be privacy, especially if they are individuals running the mirror (say someone with symmetric fibre at their house and want to contribute some of their bandwidth), giving your city or even worse, the town/region of the city, might make these individuals uneasy?
Tier1 This mirror type is intended to be a near real time copy of the Tier0 mirror. This puts higher demand on this mirror type as it needs to update as soon as possible with little to no delay, and uptime is paramount for the success of other mirrors following this tier.
Maybe a webhook? Mirrors provide an endpoint for a notification, and when a new package is updated a POST request is sent to that endpoint, which tells the mirror to rsync immediately? I have no source or benchmark, but this seems more efficient than polling more often, so its just a possible suggestion which *could* be tested. Maybe hook it into the "status update" API which is proposed, a simple endpoint within it to tell the mirror to update? This then means there is no sync frequency needed for Tier 1, as they will sync immediately, or possibly a "delay period" of up to 5 minutes to prevent Tier 0 being hit all at once? That would theoretically mean that a Tier 1 mirror will have a 1:1 within 5 minutes of a new package being built, instead of waiting upwards of 1 hour, speeding up the delivery to tier 2 and then to tier 3.
- Malicious behaviour, such as attempting to serve malicious files, or domain hijacking.
Possibly a public process on how malicious mirrors are dealt with? How will the userbase be informed if they got infected? However is it even possible to serve malicious files considering a web of trust is employed and unsigned files will be flagged by pacman?
Tier 3 Requirements
- HTTPS TLS1.2+ - Sync at least once per day - Downtime can last no longer than 48 hours *(after which it gets disabled, not removed)*
There is no bandwidth requirement here, theoretically allowing someone with aDSL with 1-2mbps upload to provide a mirror?
**Tier 4** *(LocalMirror)* - A fully user-controlled mirror. - Not to be listed in official mirror listing
These are currently unadvised by Arch Linux [3], and you are meant to network share the pacman cache [4]. Will this remain or will it be more acceptable to have local mirrors? (seen as it is being entered into the new RFC) Anyways just some things which came to mind and I would be interested in knowing the answers to, hope its not too much of a hassle :) Take care, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@icebound.dev [1] https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/29//diffs#72d8b... [2] https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/29//diffs#72d8b... [3] https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors#Notes_about_privat... [4] https://wiki.archlinux.org/title/Pacman/Tips_and_tricks#Network_shared_pacma...
Hello, First of all, I think the mirror team put a lot of effort in this RFC and it's great. I would like to suggest one more thing, outside the gitlab. On Fri, Jan 12, 2024 at 4:10 AM Polarian <polarian@polarian.dev> wrote:
Hello,
I am not a staff member, but I like to read the RFCs and arch news and I have a few things..
Polarian's review was (more or less) reflected to gitlab. So now, I think we can agree (more or less) on the standardization of the mirror tier model, requirements and recommendations, although details like storage requirements can be fine-tuned. What remains to be discussed, possibly even controversial, is some new features that were proposed, namely "validation of ownership", "Status endpoint JSON manifest". Therefore, I suggest splitting this RFC into two RFCs. We can leave new features to the second RFC for more discussions and maybe even trial runs and testing. I hope by splitting the RFC, it can improve our efficiency and not cause the whole RFC to stall. -- Jing Luo About me: https://jing.rocks/about/ PGP Fingerprint: 4E09 8D19 00AA 3F72 1899 2614 09B3 316E 13A1 1EFC
participants (3)
-
JING LUO
-
pitastrudl
-
Polarian