[arch-general] [arch-dev-public] Large packages in repositories

[Thomas Dziedzic]

On Tue, Aug 17, 2010 at 9:12 AM, Dan McGee <dpmcgee at gmail.com> wrote:

> Hey guys,
>
> A package went in so big today that it made reporead blow up on my
> local database due to the installed size being > 2GB:
> http://www.archlinux.org/packages/community/i686/sage-mathematics/
> http://www.archlinux.org/packages/community/x86_64/sage-mathematics/
>
> File
> "/usr/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py",
> line 44, in execute
>    return self.cursor.execute(query, args)
> django.db.utils.DatabaseError: integer out of range
>
> I'm wondering if we need to be more careful when it comes to these big
> packages entering our repositories. This one is especially suspect as
> of its 71096 files (and 71094 in the other architecture), a ton of
> them are things like *.py, *.pyc, *.html, or *.png files. This is ripe
> for splitting into a -data package (or not including some of this
> junk, if it is that, at all).
>
> mysql> select count(*), substring_index(path, '.', -1) from
> package_files where pkg_id = 49860 and path like '%.%' group by
> substring_index(path, '.', -1) order by count(*) desc limit 25;
> +----------+--------------------------------+
> | count(*) | substring_index(path, '.', -1) |
> +----------+--------------------------------+
> |    29787 | png                            |
> |     9410 | py                             |
> |     6602 | pyc                            |
> |     1804 | h                              |
> |     1722 | html                           |
> |     1659 | txt                            |
> |     1168 | pyo                            |
> |      988 | doctree                        |
> |      988 | rst                            |
> |      886 | hpp                            |
>
>
> tl;dr: I think we need some standards with these huge packages, and
> people need to be a lot more cognizant as to how big they are. We have
> lost more than one mirror due to complaints over needed space and
> stuff like this doesn't help.
>
> If any TU's would like to forward this along and solicit their
> thoughts I'd appreciate it.
>
> -Dan
>

There was a 4 day long thread about this in aur-general
http://mailman.archlinux.org/pipermail/aur-general/2010-August/009854.html