Arch-devops

Download

arch-devops@lists.archlinux.org

July 2017

6 participants
2 discussions

[arch-devops] Caching problem with http://www.archlinux.org/check_network_status.txt
by Daniel Aleksandersen 21 Jul '17

21 Jul '17

Hi, Just wanted to report a Arch Linux infrastrucutre problem, and hope this was the right spot. The NetworkManager test probe page, located at http://www.archlinux.org /check_network_status.txt, should be made cache-aware. Currently, users are testing the connection to their web accelerator or intermediary proxy cache and not their internet connection as expected. Please add a `Cache-Control: max-age=0, must-revalidate` HTTP response header to this page to resolve this problem. I wrote a blog post earlier this week about the problem, but had some problem finding the appropriate place to report this to Arch. https://ctrl.blog/entry/network-connection-http-checks -- Daniel

3 2

[arch-devops] Cleaning up archive.archlinux.org
by Florian 11 Jul '17

11 Jul '17

Hi, While I like the idea of the archive, it currently does not implement any kind of cleanup and forever grows in size. Right now is is somewhere around 50 million files with a total size of 1.7TB. Those are very rough numbers since calculating them for real takes considerable time. This creates multiple problems: - The disks only have ~700gb more space before they are full. We will reach that eventually - The backup creation takes somewhere between 4 to 5 hours depending on machine load. I am thinking about increasing the backup frequency where possible, but if one backup takes 4 hours that won't happen. I am also worried that if we ever need to restore, this could easily take well over 24 hours. The initial backup (which read all data rather than ignoring unchanged data) took something around 24 hours, but that was with less files. Essentially this means we need to put some kind of automatic deletion of old data in place and reduce the size of the archive. What would be a good time frame here? Have we defined a clear goal for the archive that we can use as a guideline? My gut says 6 months should be enough, but feel free to disagree. Once we have decided how long data should be kept, we also need something to actually delete it. Deleting files from the ./repos directory should be simple, but the ./packages directory is more complicated because it is not nicely separated into directories per day. Sebastien, do you have an idea about how we could delete old data or do you have a script for that? Florian

6 8