[arch-dev-public] The docs/!docs thing again

Dan McGee

28 Oct 2009 28 Oct '09

4:14 a.m.

What are we doing this month with docs? I ask because I was rather shocked to see this: dmcgee@kilkenny /usr/share/doc/gtkmm-2.4 $ pacman -Si gtkmm | grep Size Download Size : 14396.06 K Installed Size : 61660.00 K dmcgee@kilkenny /usr/share/doc/gtkmm-2.4 $ du -sh . 50M . That means ~80% of this package is stuff almost no-one is ever going to look at. That seems a bit overkill to me to include in our packages. -Dan

Show replies by date

Allan McRae

28 Oct 28 Oct

4:49 a.m.

Dan McGee wrote:

...

What are we doing this month with docs? I ask because I was rather shocked to see this:

dmcgee@kilkenny /usr/share/doc/gtkmm-2.4 $ pacman -Si gtkmm | grep Size Download Size : 14396.06 K Installed Size : 61660.00 K

dmcgee@kilkenny /usr/share/doc/gtkmm-2.4 $ du -sh . 50M .

That means ~80% of this package is stuff almost no-one is ever going to look at. That seems a bit overkill to me to include in our packages.

I thought we were splitting them out if they were massive like this. e.g. ruby/ruby-docs. It would seem the best solution. Allan

Eric Bélanger

10:32 a.m.

On Wed, Oct 28, 2009 at 12:49 AM, Allan McRae wrote:

...

Dan McGee wrote:

...
What are we doing this month with docs? I ask because I was rather shocked to see this:

dmcgee@kilkenny /usr/share/doc/gtkmm-2.4 $ pacman -Si gtkmm | grep Size Download Size : 14396.06 K Installed Size : 61660.00 K

dmcgee@kilkenny /usr/share/doc/gtkmm-2.4 $ du -sh . 50M .

That means ~80% of this package is stuff almost no-one is ever going to look at. That seems a bit overkill to me to include in our packages.

I thought we were splitting them out if they were massive like this. e.g. ruby/ruby-docs. It would seem the best solution.

Allan

Yes, that's what we're supposed to do; it was probably an oversight. Splitting big docs like this will be easy once the new makepkg with multi-arch support for split package will be out (assuming the dbscripts can handle it).

Allan McRae

10:44 a.m.

New subject: [arch-dev-public] changes needed in db-scritps - Was: The docs/!docs thing again

Eric Bélanger wrote:

...

Yes, that's what we're supposed to do; it was probably an oversight. Splitting big docs like this will be easy once the new makepkg with multi-arch support for split package will be out (assuming the dbscripts can handle it).

The dbscripts do not handle that. They also do not handle only uploading one or some packages from a split package (pkgrel can now be overridden and a selection of packages updated). Allan

Eric Bélanger

11:22 a.m.

New subject: [arch-dev-public] changes needed in db-scritps - Was: The docs/!docs thing again

On Wed, Oct 28, 2009 at 6:44 AM, Allan McRae wrote:

...

Eric Bélanger wrote:

...
Yes, that's what we're supposed to do; it was probably an oversight. Splitting big docs like this will be easy once the new makepkg with multi-arch support for split package will be out (assuming the dbscripts can handle it).

The dbscripts do not handle that. They also do not handle only uploading one or some packages from a split package (pkgrel can now be overridden and a selection of packages updated).

Allan

They also can't handle split packages with different pkgver or pkgrel. So they'll need a major maintainance update indeed. There is a pending dbscripts update. Maybe we should release it before starting to work on them to support the new features in the upcoming makepkg.

Aaron Griffin

6:56 p.m.

New subject: [arch-dev-public] changes needed in db-scritps - Was: The docs/!docs thing again

On Wed, Oct 28, 2009 at 6:22 AM, Eric Bélanger wrote:

...

On Wed, Oct 28, 2009 at 6:44 AM, Allan McRae wrote:

...
Eric Bélanger wrote:

...
Yes, that's what we're supposed to do; it was probably an oversight. Splitting big docs like this will be easy once the new makepkg with multi-arch support for split package will be out (assuming the dbscripts can handle it).

The dbscripts do not handle that. They also do not handle only uploading one or some packages from a split package (pkgrel can now be overridden and a selection of packages updated).

Allan

They also can't handle split packages with different pkgver or pkgrel. So they'll need a major maintainance update indeed. There is a pending dbscripts update. Maybe we should release it before starting to work on them to support the new features in the upcoming makepkg.

I think everything is in there and live in /arch-test/ Can I get one or two "go aheads" to move it live? Then we can work on this malarky. As for the db-scripts changes here - we simply need to be able to grok info internal to the other functions, yes?

Eric Bélanger

10:05 p.m.

New subject: [arch-dev-public] changes needed in db-scritps - Was: The docs/!docs thing again

On Wed, Oct 28, 2009 at 2:56 PM, Aaron Griffin wrote:

...

On Wed, Oct 28, 2009 at 6:22 AM, Eric Bélanger wrote:

...
On Wed, Oct 28, 2009 at 6:44 AM, Allan McRae wrote:

...
Eric Bélanger wrote:

...
Yes, that's what we're supposed to do; it was probably an oversight. Splitting big docs like this will be easy once the new makepkg with multi-arch support for split package will be out (assuming the dbscripts can handle it).

The dbscripts do not handle that. They also do not handle only uploading one or some packages from a split package (pkgrel can now be overridden and a selection of packages updated).

Allan

They also can't handle split packages with different pkgver or pkgrel. So they'll need a major maintainance update indeed. There is a pending dbscripts update. Maybe we should release it before starting to work on them to support the new features in the upcoming makepkg.

I think everything is in there and live in /arch-test/

Can I get one or two "go aheads" to move it live? Then we can work on this malarky.

I've been using /arch-test/ since you set it up. I haven't encountered any problems. You'll also need to add the sourceballs and source cleanup scripts to the cron jobs.

...

As for the db-scripts changes here - we simply need to be able to grok info internal to the other functions, yes?

Aaron Griffin

10:58 p.m.

New subject: [arch-dev-public] changes needed in db-scritps - Was: The docs/!docs thing again

On Wed, Oct 28, 2009 at 5:05 PM, Eric Bélanger wrote:

...

On Wed, Oct 28, 2009 at 2:56 PM, Aaron Griffin wrote:

...
On Wed, Oct 28, 2009 at 6:22 AM, Eric Bélanger wrote:

...
On Wed, Oct 28, 2009 at 6:44 AM, Allan McRae wrote:

...
Eric Bélanger wrote:

...
Yes, that's what we're supposed to do; it was probably an oversight. Splitting big docs like this will be easy once the new makepkg with multi-arch support for split package will be out (assuming the dbscripts can handle it).

The dbscripts do not handle that. They also do not handle only uploading one or some packages from a split package (pkgrel can now be overridden and a selection of packages updated).

Allan

They also can't handle split packages with different pkgver or pkgrel. So they'll need a major maintainance update indeed. There is a pending dbscripts update. Maybe we should release it before starting to work on them to support the new features in the upcoming makepkg.

I think everything is in there and live in /arch-test/

Can I get one or two "go aheads" to move it live? Then we can work on this malarky.

I've been using /arch-test/ since you set it up. I haven't encountered any problems. You'll also need to add the sourceballs and source cleanup scripts to the cron jobs.

Ok, I'll do the move soon. How often should the sourceball cleanup run? As often as the package cleanup?

Eric Bélanger

29 Oct 29 Oct

12:54 a.m.

New subject: [arch-dev-public] changes needed in db-scritps - Was: The docs/!docs thing again

On Wed, Oct 28, 2009 at 6:58 PM, Aaron Griffin wrote:

...

On Wed, Oct 28, 2009 at 5:05 PM, Eric Bélanger wrote:

...
On Wed, Oct 28, 2009 at 2:56 PM, Aaron Griffin wrote:

...
On Wed, Oct 28, 2009 at 6:22 AM, Eric Bélanger wrote:

...
On Wed, Oct 28, 2009 at 6:44 AM, Allan McRae wrote:

...
Eric Bélanger wrote:

...
Yes, that's what we're supposed to do; it was probably an oversight. Splitting big docs like this will be easy once the new makepkg with multi-arch support for split package will be out (assuming the dbscripts can handle it).

The dbscripts do not handle that. They also do not handle only uploading one or some packages from a split package (pkgrel can now be overridden and a selection of packages updated).

Allan

They also can't handle split packages with different pkgver or pkgrel. So they'll need a major maintainance update indeed. There is a pending dbscripts update. Maybe we should release it before starting to work on them to support the new features in the upcoming makepkg.

I think everything is in there and live in /arch-test/

Can I get one or two "go aheads" to move it live? Then we can work on this malarky.

I've been using /arch-test/ since you set it up. I haven't encountered any problems. You'll also need to add the sourceballs and source cleanup scripts to the cron jobs.

Ok, I'll do the move soon. How often should the sourceball cleanup run? As often as the package cleanup?

No. For reasons explained in the patch comment, weekly is frequent enough for the source cleanup cronjob. For the regular souceballs cron job set it to run every 8 hrs as before. As it hadn't ran for a while, it'll have some catch up to do. If it's too slow or too resource expensive, we could have it run less often (1 or 2 times per day). I'll keep an eye on it. And perhaps profile it when time will permits.

Aaron Griffin

28 Oct 28 Oct

6:57 p.m.

On Tue, Oct 27, 2009 at 11:14 PM, Dan McGee wrote:

...

What are we doing this month with docs? I ask because I was rather shocked to see this:

dmcgee@kilkenny /usr/share/doc/gtkmm-2.4 $ pacman -Si gtkmm | grep Size Download Size : 14396.06 K Installed Size : 61660.00 K

dmcgee@kilkenny /usr/share/doc/gtkmm-2.4 $ du -sh . 50M .

That means ~80% of this package is stuff almost no-one is ever going to look at. That seems a bit overkill to me to include in our packages.

Perhaps a namcap warning for this case would be helpful. Something like: WARNING: At least 50% of this package is documentation. Consider splitting the docs to a separate package.

Dan McGee

9 Nov 9 Nov

3:13 a.m.

New subject: [arch-dev-public] [PATCH] Add a warning for packages that are over 50% docs

Look at the sizes of things in usr/share/docs, and flag the package if it is over 50% documentation. Signed-off-by: Dan McGee --- Finally got around to doing this. I tested it on the package in question (81% docs, by the way) and also ran it on the gtk-doc package to make sure the name check was working and excluding packages that are meant to contain docs. Namcap guys, let me know if you have any questions/problems/concerns. This was in response to this email: http://mailman.archlinux.org/pipermail/arch-dev-public/2009-October/014112.h...: -Dan Namcap/__init__.py | 1 + Namcap/lotsofdocs.py | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++ namcap-tags | 1 + 3 files changed, 53 insertions(+), 0 deletions(-) create mode 100644 Namcap/lotsofdocs.py diff --git a/Namcap/__init__.py b/Namcap/__init__.py index 5d8d911..8b39e7a 100644 --- a/Namcap/__init__.py +++ b/Namcap/__init__.py @@ -31,6 +31,7 @@ __tarball__ = """ infodirectory libtool licensepkg + lotsofdocs mimefiles perllocal permissions diff --git a/Namcap/lotsofdocs.py b/Namcap/lotsofdocs.py new file mode 100644 index 0000000..2acb350 --- /dev/null +++ b/Namcap/lotsofdocs.py @@ -0,0 +1,51 @@ +# +# namcap rules - lotsofdocs +# Copyright (C) 2009 Dan McGee +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +# + +import os +import tarfile + +class package: + def short_name(self): + return "lots-of-docs" + def long_name(self): + return "See if a package is carrying more documentation than it should" + def prereq(self): + return "tar" + def analyze(self, pkginfo, tar): + ret = [[],[],[]] + if hasattr(pkginfo, 'name'): + if pkginfo.name.endswith('-doc'): + return ret + docdir = 'usr/share/doc' + size = 0 + docsize = 0 + + for i in tar.getmembers(): + if i.name.startswith(docdir): + docsize += i.size + size += i.size + + ratio = docsize / float(size) + if size > 0 and ratio > 0.50: + ret[1].append(("lots-of-docs %f", ratio * 100)) + + return ret + def type(self): + return "tarball" +# vim: set ts=4 sw=4 noet: diff --git a/namcap-tags b/namcap-tags index 710077c..3245760 100644 --- a/namcap-tags +++ b/namcap-tags @@ -36,6 +36,7 @@ info-dir-file-present %s :: Info directory file (%s) needs removed. insecure-rpath %s :: Insecure RPATH (%s). If present, RPATH should be only /usr/lib. libtool-file-present %s :: File (%s) is a libtool file. link-level-dependence %s in %s :: Link-level dependence (%s) in file %s +lots-of-docs %f :: Package was %.0f%% docs by size; maybe you should split out a docs package mime-cache-not-updated :: Mime-file found. Add "update-mime-database usr/share/mime" to the install file missing-contributor :: Missing Contributor tag missing-custom-license-dir usr/share/licenses/%s :: Missing custom license directory (usr/share/licenses/%s) -- 1.6.5.2

5281

Age (days ago)

5293

Last active (days ago)

List overview

Download

10 comments

5 participants

participants (5)

Aaron Griffin
Allan McRae
Dan McGee
Dan McGee
Eric Bélanger