[arch-dev-public] Large packages in repositories

[Dan McGee]

Hey guys,

A package went in so big today that it made reporead blow up on my
local database due to the installed size being > 2GB:
http://www.archlinux.org/packages/community/i686/sage-mathematics/
http://www.archlinux.org/packages/community/x86_64/sage-mathematics/

File "/usr/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py",
line 44, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: integer out of range

I'm wondering if we need to be more careful when it comes to these big
packages entering our repositories. This one is especially suspect as
of its 71096 files (and 71094 in the other architecture), a ton of
them are things like *.py, *.pyc, *.html, or *.png files. This is ripe
for splitting into a -data package (or not including some of this
junk, if it is that, at all).

mysql> select count(*), substring_index(path, '.', -1) from
package_files where pkg_id = 49860 and path like '%.%' group by
substring_index(path, '.', -1) order by count(*) desc limit 25;
+----------+--------------------------------+
| count(*) | substring_index(path, '.', -1) |
+----------+--------------------------------+
|    29787 | png                            |
|     9410 | py                             |
|     6602 | pyc                            |
|     1804 | h                              |
|     1722 | html                           |
|     1659 | txt                            |
|     1168 | pyo                            |
|      988 | doctree                        |
|      988 | rst                            |
|      886 | hpp                            |


tl;dr: I think we need some standards with these huge packages, and
people need to be a lot more cognizant as to how big they are. We have
lost more than one mirror due to complaints over needed space and
stuff like this doesn't help.

If any TU's would like to forward this along and solicit their
thoughts I'd appreciate it.

-Dan