Hi, On 23-01-19, Florian Pritz wrote:
Hi Baptiste,
On Thu, Jun 14, 2018 at 10:28:17AM +0200, Florian Pritz via arch-dev-public <arch-dev-public@archlinux.org> wrote:
On 10.06.2018 01:35, Baptiste Jonglez wrote:
Archival of all packages between September 2013 and December 2016 is finished:
https://archive.org/details/archlinuxarchive
Here is some documentation on this "Historical Archive" hosted on archive.org:
https://wiki.archlinux.org/index.php/Arch_Linux_Archive#Historical_Archive
So the archive is still growing and we'll need to clean it up again soonish. Where do you have the archive.org uploader and is it in a state where we (devops team) can also easily use it? If not, could you help us getting it to that point?
I have just pushed the script I wrote last time: https://github.com/zorun/arch-historical-archive It's a bit hackish and requires some manual work to correctly upload all packages for a given year, because archive.org rate-limits quite aggressively when they are overloaded.
We also have an archive cleanup script here[1]. Maybe the uploader can be integrated there? I don't know how complicated it is.
[1] https://github.com/archlinux/archivetools/blob/master/archive-cleaner
What about uploading to archive.org as soon as we archive packages on orion? https://github.com/archlinux/archivetools/blob/master/archive.sh It would avoid hammering the archive.org server, because we would only send one package at a time. In any case, we need a retry mechanism to cope with the case where the upload fails. Baptiste