On Wed, May 21, 2008 at 4:16 PM, Jan de Groot wrote:

On Wed, 2008-05-21 at 15:49 -0500, Aaron Griffin wrote:

...

So, yeah people are blowing smoke about this hosting the sources thing. That's fine, it's a legit violation...

I made a little script which should generate some of these things... I'm testing it now, as it's not fully "live" just yet: http://dev.archlinux.org/~aaron/sources/

Here's something neat which is an added bonus: any one of these things can be used to build the entire package when offline. Download the src.tar.gz, extract, and makepkg.

One big lingering issue: do we need different source tarballs for each arch? My instincts tell me yes, for a few reasons: * I recall some PKGBUILDs having CARCH based 'if's around the sources * Some packages are only in one arch or another

Does anyone know of a way to solve this... I don't want to bloat the disk usage for a few special cases? mayeb md5sum and symlink?

I don't see the need for per-arch source tarballs. Sure, there's inconsistency between architectures (epiphany 2.22.0 in i686 vs 2.22.2.1 in x86_64 for example), but that would mean two source packages, as we have two different binary packages. The script could omit the architecture and only create a package for amd64 if there isn't exactly the same package for i686 already.

The only packages I know that have checks for CARCH around sources are swt (which is not GPL anyways) and some binary non-free things that are built for a specific architecture. As these binary blobs aren't GPL either (and redistribution rules are questionable), I guess we won't see this much around.

Ok, I will modify that then and remove the CARCH from the filename. Another thing - I am not checking licenses. I figured it's not a bad idea to just do it for everything. In the grand scheme of things, the amount of non-GPL packages is probably small, and it's not really a BAD idea to make extras. /me shrugs However, now that you mentioned swt, it brings up a good point. We can't do this for things like flashplugin just yet. So, should I create some sort of list of packages to NEVER generate sources for?

On Thu, May 22, 2008 at 7:56 AM, Thayer Williams wrote:

On 5/21/08, Aaron Griffin wrote:

...

However, now that you mentioned swt, it brings up a good point. We can't do this for things like flashplugin just yet. So, should I create some sort of list of packages to NEVER generate sources for?

I think it would be a good idea to implement a blacklist. A lot of factors may warrant its use (size, license, version, etc.)

It's a script, so you could just source the PKGBUILD and check the license=() variable for GPL.

Am Freitag, 23. Mai 2008 18:22:34 schrieb Aaron Griffin:

Our non-GPL licensed packages are fairly small compared to the GPL ones

I just did some queries against the db: packages under GPL: count 5789 sum(isize) 33.88 GB sum(csize) 14.23 GB packages with other licenses count 1873 sum(isize) 9.82 GB sum(csize) 4.14 GB packages without licenses count 1578 sum(isize) 4.86 GB sum(csize) 1.80 GB Those data include both arches and all repos (core, extra, testing, unstable, community) The sum of csize might be a hint about how much space the src-packages might need. -- archlinux.de

On Fri, May 23, 2008 at 12:16 PM, Paul Mattal wrote:

Pierre Schmitz wrote:

...

Am Freitag, 23. Mai 2008 18:22:34 schrieb Aaron Griffin:

...

Our non-GPL licensed packages are fairly small compared to the GPL ones

I just did some queries against the db:

packages under GPL: count 5789 sum(isize) 33.88 GB sum(csize) 14.23 GB

packages with other licenses count 1873 sum(isize) 9.82 GB sum(csize) 4.14 GB

packages without licenses count 1578 sum(isize) 4.86 GB sum(csize) 1.80 GB

Those data include both arches and all repos (core, extra, testing, unstable, community)

The sum of csize might be a hint about how much space the src-packages might need.

Remember that GPL3 has new language, which appears at least at first glance to allow you to provide a link to a third party hosting the source. Our PKGBUILDs constitute quite "clear directions" as to where to find the sourc. So I think you only have to mirror for GPL2 holdouts.

From section 6d of GPLv3:

Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.

Regardless, mirroring source for all packages seems good. It's easy, and I don't see a reason not to do it. I just wanted to point out that it seems GPL3+ has a less stringent source hosting obligation which it seems clear we already meet.

Agreed. The GPL3 has a clause to the effect of "places like sourceforge are way more trustworthy a server than someotherlinux.org", so yeah they "got it".

On Sat, May 31, 2008 at 3:04 AM, Roman Kyrylych wrote:

2008/5/22 Thayer Williams :

...

On 5/21/08, Aaron Griffin wrote:

...

However, now that you mentioned swt, it brings up a good point. We can't do this for things like flashplugin just yet. So, should I create some sort of list of packages to NEVER generate sources for?

I think it would be a good idea to implement a blacklist. A lot of factors may warrant its use (size, license, version, etc.)

Hmm, I can only think about packages that we have a special permission to redistribute for: virtualbox-additions in Community and (I may be totally wrong here) flashplayer in Extra), don't know about others. (BTW both packages' "source" and binary are almost the same) What I don't know if redistribution from out site covers our mirrors too.

Ok, so this ran for all repos. I blacklisted a few known bad ones (bash and readline, the mirror used for the source is borked), and the following huge list of packages failed: http://dev.archlinux.org/~aaron/sources/failed.pkgs Take a second and scan these please - if you can, please correct the source. As Eric pointed out elsewhere, we can merge source change from trunk with archrelease, without the need to re-release the package (this will fix abs too)

On Sat, May 31, 2008 at 1:27 PM, Aaron Griffin wrote:

On Sat, May 31, 2008 at 1:23 PM, Aaron Griffin wrote:

...

On Sat, May 31, 2008 at 3:04 AM, Roman Kyrylych wrote:

...

2008/5/22 Thayer Williams :

...

On 5/21/08, Aaron Griffin wrote:

...

However, now that you mentioned swt, it brings up a good point. We can't do this for things like flashplugin just yet. So, should I create some sort of list of packages to NEVER generate sources for?

I think it would be a good idea to implement a blacklist. A lot of factors may warrant its use (size, license, version, etc.)

Hmm, I can only think about packages that we have a special permission to redistribute for: virtualbox-additions in Community and (I may be totally wrong here) flashplayer in Extra), don't know about others. (BTW both packages' "source" and binary are almost the same) What I don't know if redistribution from out site covers our mirrors too.

Ok, so this ran for all repos. I blacklisted a few known bad ones (bash and readline, the mirror used for the source is borked), and the following huge list of packages failed: http://dev.archlinux.org/~aaron/sources/failed.pkgs

Take a second and scan these please - if you can, please correct the source. As Eric pointed out elsewhere, we can merge source change from trunk with archrelease, without the need to re-release the package (this will fix abs too)

Keep in mind there's probably dupes, as it runs for both x86_64 and i686 (and checks for an existing source for the correct version). Let me try to clean those up real quick

Sorted. 431 packages: http://dev.archlinux.org/~aaron/sources/failed.pkgs

On Mon, 2 Jun 2008, Travis Willard wrote:

On Sat, May 31, 2008 at 3:09 PM, Aaron Griffin wrote:

...

On Sat, May 31, 2008 at 1:27 PM, Aaron Griffin wrote:

...

On Sat, May 31, 2008 at 1:23 PM, Aaron Griffin wrote:

...

On Sat, May 31, 2008 at 3:04 AM, Roman Kyrylych wrote:

...

2008/5/22 Thayer Williams :

...

On 5/21/08, Aaron Griffin wrote: > However, now that you mentioned swt, it brings up a good point. We > can't do this for things like flashplugin just yet. So, should I > create some sort of list of packages to NEVER generate sources for?

I think it would be a good idea to implement a blacklist. A lot of factors may warrant its use (size, license, version, etc.)

Hmm, I can only think about packages that we have a special permission to redistribute for: virtualbox-additions in Community and (I may be totally wrong here) flashplayer in Extra), don't know about others. (BTW both packages' "source" and binary are almost the same) What I don't know if redistribution from out site covers our mirrors too.

Ok, so this ran for all repos. I blacklisted a few known bad ones (bash and readline, the mirror used for the source is borked), and the following huge list of packages failed: http://dev.archlinux.org/~aaron/sources/failed.pkgs

Take a second and scan these please - if you can, please correct the source. As Eric pointed out elsewhere, we can merge source change from trunk with archrelease, without the need to re-release the package (this will fix abs too)

Keep in mind there's probably dupes, as it runs for both x86_64 and i686 (and checks for an existing source for the correct version). Let me try to clean those up real quick

Sorted. 431 packages: http://dev.archlinux.org/~aaron/sources/failed.pkgs

Hm - that's cool and all, but I'm finding it hard to grep. Want me to make a todo list?

Aaron: Could you rerun your script to update the sources and the failed.pkgs file? I'll try to fix them on my free time. I'll merge the change with archrelease if Ronald didn't added a license to their trunk yet. It would be easier for me if I could run the script myself (where is it? do i have enough privs to run it?) so I could keep failed.pkgs up-to-date. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

On Thu, Aug 14, 2008 at 3:14 PM, Eric Belanger wrote:

On Mon, 2 Jun 2008, Travis Willard wrote:

...

On Sat, May 31, 2008 at 3:09 PM, Aaron Griffin wrote:

...

On Sat, May 31, 2008 at 1:27 PM, Aaron Griffin wrote:

...

On Sat, May 31, 2008 at 1:23 PM, Aaron Griffin wrote:

...

On Sat, May 31, 2008 at 3:04 AM, Roman Kyrylych wrote:

...

2008/5/22 Thayer Williams : > > On 5/21/08, Aaron Griffin wrote: >> >> However, now that you mentioned swt, it brings up a good point. We >> can't do this for things like flashplugin just yet. So, should I >> create some sort of list of packages to NEVER generate sources for? > > I think it would be a good idea to implement a blacklist. A lot of > factors may warrant its use (size, license, version, etc.)

Hmm, I can only think about packages that we have a special permission to redistribute for: virtualbox-additions in Community and (I may be totally wrong here) flashplayer in Extra), don't know about others. (BTW both packages' "source" and binary are almost the same) What I don't know if redistribution from out site covers our mirrors too.

Ok, so this ran for all repos. I blacklisted a few known bad ones (bash and readline, the mirror used for the source is borked), and the following huge list of packages failed: http://dev.archlinux.org/~aaron/sources/failed.pkgs

Take a second and scan these please - if you can, please correct the source. As Eric pointed out elsewhere, we can merge source change from trunk with archrelease, without the need to re-release the package (this will fix abs too)

Keep in mind there's probably dupes, as it runs for both x86_64 and i686 (and checks for an existing source for the correct version). Let me try to clean those up real quick

Sorted. 431 packages: http://dev.archlinux.org/~aaron/sources/failed.pkgs

Hm - that's cool and all, but I'm finding it hard to grep. Want me to make a todo list?

Aaron: Could you rerun your script to update the sources and the failed.pkgs file?

I'll try to fix them on my free time. I'll merge the change with archrelease if Ronald didn't added a license to their trunk yet.

It would be easier for me if I could run the script myself (where is it? do i have enough privs to run it?) so I could keep failed.pkgs up-to-date.

It's not complete yet, which is why I run it in my homedir for now. Eventually, I will move it so the source goes to /home/ftp and it will be on a cron job. I will update this list when I get a chance, because a few have been fixed, I'm sure