[arch-mirrors] CDN based/caching mirror?

[Kristian Klausen]

On 30.01.2020 11.30, Artis Steinbergs via arch-mirrors wrote:
> Hi,
>
> Have you thought about the maximum file size Cloudflare caches or 
> supports ?
> Via: 
> https://support.cloudflare.com/hc/en-us/articles/200172516-Understanding-Cloudflare-s-CDN
> Cloudflare cache maximum file size is 512MB for Free, Pro, and 
> Business customers.

I have thought about it, but there are only 6 packages which is bigger 
than 512M and nginx is gonna cache them anyway, so I don't see it as a 
big issue.
iso is optional so I'm not sure if I gonna mirror them.

>
> In the archlinux reposoitory there are files that are larger than 512M:
> # find . -type f -size +512M
> ./iso/archboot/2016.08/archlinux-2016.08-1-archboot.iso
> ./iso/archboot/2016.12/archlinux-2016.12-1-archboot.iso
> ./iso/archboot/2018.06/archlinux-2018.06-1-archboot.iso
> ./iso/2019.12.01/archlinux-2019.12.01-x86_64.iso
> ./iso/2019.12.01/arch/x86_64/airootfs.sfs
> ./iso/2020.01.01/archlinux-2020.01.01-x86_64.iso
> ./iso/2020.01.01/arch/x86_64/airootfs.sfs
> ./iso/2019.11.01/archlinux-2019.11.01-x86_64.iso
> ./iso/2019.11.01/arch/x86_64/airootfs.sfs
> ./pool/community/sauerbraten-data-2013_02_03_collect_edition-3-any.pkg.tar.xz 
>
> ./pool/community/cuda-10.2.89-3-x86_64.pkg.tar.zst
> ./pool/community/xonotic-data-0.8.2-3-any.pkg.tar.zst
> ./pool/community/kea-devel-docs-1.5.0-1-any.pkg.tar.xz
> ./pool/community/0ad-data-a23.1-1-any.pkg.tar.xz
> ./pool/community/supertuxkart-1.1-1-x86_64.pkg.tar.zst
>
> ---
> Artis Šteinbergs
>
>
> Kristian Klausen via arch-mirrors @ 26.01.2020 18:19 rakstīja:
>> Hi
>>
>> I'm considering setting up a Arch Linux mirror and I'm considering a
>> different design.
>>
>> So instead of mirroring the whole thing, the idea is to mirror only
>> the database files (core.db etc) and download the packages on demand
>> from a Tier 1 mirror (and let nginx cache them). By doing it that way,
>> I only download requested packages from the Tier 1 mirrors, instead of
>> downloading the whole thing (saving Tier 1 bandwidth).
>>
>> To provide even better performance a CDN (ex: Cloudflare) could be
>> used to provide more caching. So we end up with a setup like this:
>> Cloudflare -> Nginx cache -> Tier1 mirrors (nginx with multiple 
>> upstream)
>>
>> Do I miss something? Is this a bad idea?
>> If I do setup a mirror like that, is there any chance it could be
>> added as a official mirror?
>>
>> Best regards
>> Kristian Klausen