[arch-mirrors] Possibility of adding debug repositories

[Erwin Bronkhorst - Studenten Net Twente]

Hi Eli,

> So, question to mirror admins: if Arch was to add debug repositories,
> would you be okay syncing them? And should it be opt-in or opt out?

We (ftp.snt.utwente.nl) would be okay syncing them. I really don't mind whether it will be an opt-in or opt-out solution. I'm okay with an increase in size of the current mirror, but I also don't mind setting up an extra rsync-cronjob if that is what it takes to opt-in.

Kind regards,

Erwin Bronkhorst
SNT FTPCom

> -----Oorspronkelijk bericht-----
> Van: arch-mirrors <arch-mirrors-bounces at archlinux.org> Namens Eli
> Schwartz
> Verzonden: vrijdag 5 juni 2020 21:58
> Aan: arch-mirrors at archlinux.org
> Onderwerp: [arch-mirrors] Possibility of adding debug repositories
> 
> If Arch Linux were to add repositories containing split debug packages
> for all our x86_64 packages, this would obviously add a fair amount of
> space to the mirror requirements. It could possibly double or triple
> the
> size taken by non-data packages. I don't have real-world numbers for
> how
> much space it would take up, but I do have some comparisons.
> 
> For my custom repo as a sample, which I do upload debug packages for, I
> am using 2.9Gi of space, and 1.7Gi comes from debug packages.
> 
> On the other side of things, our biggest 2 official packages are cuda,
> 4137.96 MiB of proprietary blobs that aren't currently stripped and
> therefore even if we did split it into a debug package it wouldn't
> increase space usage at all, and kicad-library-3d, 5171.97 MiB of pure
> data in /usr/share, which is not eligible for debug packages anyway.
> 
> A naive list of packages which would probably generate debug packages:
> 
> $ expac -H M '%m %n %a' | grep -v 'any$' | sort --human-numeric-sort
> [...]
> 1109.50 MiB emscripten x86_64
> 1136.27 MiB python-tensorflow x86_64
> 1145.23 MiB python-tensorflow-opt x86_64
> 1286.69 MiB python-pytorch-cuda x86_64
> 1289.44 MiB python-pytorch-opt-cuda x86_64
> 1518.42 MiB ghc-static x86_64
> 2589.23 MiB python-tensorflow-cuda x86_64
> 2597.38 MiB python-tensorflow-opt-cuda x86_64
> 3757.68 MiB tensorflow-cuda x86_64
> 3765.75 MiB tensorflow-opt-cuda x86_64
> 4137.96 MiB cuda x86_64
> 
> (Basically all of the really big stuff is tensorflow/cuda/machine
> learning bits. We could selectively disable debug packages for two
> PKGBUILDs and avoid all the worst offenders, if we needed to. heh.)
> 
> Packages which definitely would not (there's some big, high-profile
> packages here, and my custom repo doesn't reflect this sort of spread
> at
> all):
> 
> $ expac -H M '%m %n %a' | grep -v 'any$' | sort --human-numeric-sort
> [...]
> 1202.02 MiB texlive-fontsextra any
> 1307.98 MiB texlive-fontsextra any
> 2006.67 MiB 0ad-data any
> 3112.39 MiB nltk-data any
> 5171.97 MiB kicad-library-3d any
> 
> ...
> 
> Anyway, providing these symbols would be generally desirable for users,
> and ideally it would work opt-out to make it easier for users to get
> access to them. It's something we've generally wanted to do, see for
> example https://bugs.archlinux.org/task/38755
> And it's possible we may actually, at long last, get around to
> implementing this.
> 
> So, question to mirror admins: if Arch was to add debug repositories,
> would you be okay syncing them? And should it be opt-in or opt out?
> 
> The answers to these questions will influence the direction I will take
> in trying to devise a satisfactory resolution to this outstanding
> infrastructure request. So I would love to get some input from the
> people who would be affected by such a change.
> 
> --
> Eli Schwartz
> Bug Wrangler and Trusted User
> 
>