On Sat, May 31, 2008 at 3:47 PM, Miklos Vajna wrote:

On Sat, May 31, 2008 at 12:07:45PM -0500, Dan McGee wrote:

...

This supplements the --source option and does nearly the same thing, except downloaded source files are included in the archive as well.

This is mainly an RFC patch and interested parties are encouraged to comment. I was also planning on making a patch for adding the package/ prefix to the archived files, but apparently no archive program makes this easy to do with is just plain stupid. If someone knows how without creating a temporary dir and moving files there, let me know.

i don't want to flame or anything, but what's the purpose of this? is there any program that would make use of such an archive? except that if you put it out to http then broken out files are browsable, while if you tar it up (like the rpm guys do with srpm) then the user has to download the whole source package, even if he/she is only interested in the buildscript or patches.

Yes there is use for this- to help people using makepkg comply with the GPL and providing source and the necessary tools to build it when they provide binary packages. I didn't remove the --source option on purpose; you can still generate a src.tar.gz package with the minimal files needed and none of the downloaded source. -Dan

Miklos Vajna wrote:

On Sat, May 31, 2008 at 04:32:15PM -0500, Dan McGee wrote:

...

Yes there is use for this- to help people using makepkg comply with the GPL and providing source and the necessary tools to build it when they provide binary packages. I didn't remove the --source option on purpose; you can still generate a src.tar.gz package with the minimal files needed and none of the downloaded source.

ah, i see. it's your decision, but maybe it would worth considering what Gentoo and Frugalware does: just run a script from cron that extracts urls from buildscripts, checks for missing downloaded source tarballs, obtains them and you're done.

http://git.frugalware.org/gitweb/gitweb.cgi?p=frugalware-current.git;a=blob;...

of course this src.tar.gz approach may have other benefits i don't see..

Don't only think of the for use by the main distro. There are plenty of people hosting a small custom repo that does not warrant such a cron job. Allan

On Sat, May 31, 2008 at 10:47 PM, Miklos Vajna wrote:

i don't want to flame or anything, but what's the purpose of this? is there any program that would make use of such an archive? except that if you put it out to http then broken out files are browsable, while if you tar it up (like the rpm guys do with srpm) then the user has to download the whole source package, even if he/she is only interested in the buildscript or patches.

I agree, I also think that keeping the buildscript/patches (archlinux specific) and the real source files (upstream) separate is nicer and more flexible.

On Mon, Jun 02, 2008 at 06:47:36AM -0500, Dan McGee wrote:

So what are you saying, did anyone actually look at the patch? I didn't change the existing --source option one bit, that *still* only includes local source files.

yes, i read that patch :)

All I did was add a completely new --allsource option that allows you to also build an src.tar.gz with all source, local and remote. How much more flexible can we get than having two options here?

i just said that creating such a huge srpm-like tarball is a bad idea, and encouraging people to do so by adding an option for that is not something i would recommend. just my two cents, though.

On Mon, Jun 2, 2008 at 9:56 AM, Allan McRae wrote:

Dan McGee wrote:

...

<snip> + local netfile + for netfile in ${source[@]}; do + local file=$(strip_url "$netfile") + if [ -f $netfile ]; then + msg2 "$(gettext "Adding %s...")" "$netfile" + comp_files="$comp_files $netfile" + elif [ "$SOURCEONLY" = "2" -a -f "$srcdir/$file" ]; then + msg2 "$(gettext "Adding %s...")" "$file" + comp_files="$comp_files -C $srcdir $file"

I'm a little confused here. The file $srcdir/$file is just a link to $SRCDEST/$file which may be $startdir/$file. So does this not just add the symbolic link to the tarball. Also, the -C option flags all following files to be from the $srcdir directory. So, anything added after this will also get its symbolic link added.

If I'm not going down the wrong track here, this should fix it...

Yeah, this wasn't tested super hard or anything so I think you are on to something here. The -C option is definitely a bit odd but it kind of does what we want. Since all files are available in src/ (although some might be symlinks), can't we use 1 -C flag in combination with -H to follow symlinks to our source files? While we are making changes, can you see an easy way to get everything under a pkgname/ dir (maybe with pkgver-pkgrel too) inside the tarball? Unfortunately bsdtar has no --prefix option to easily do this, and our friend Loui says it is doable but has not offered a line of code.

+ local netfile + for netfile in ${source[@]}; do + local file=$(strip_url "$netfile") + if [ -f $netfile ]; then + msg2 "$(gettext "Adding %s...")" "$netfile" + comp_files="$comp_files -C $startdir $netfile" + elif [ "$SOURCEONLY" = "2" -a -f "$SRCDEST/$file" ]; then + msg2 "$(gettext "Adding %s...")" "$file" + comp_files="$comp_files -C $SRCDEST $file"

_______________________________________________ pacman-dev mailing list pacman-dev@archlinux.org http://archlinux.org/mailman/listinfo/pacman-dev

On Mon, Jun 2, 2008 at 10:56 AM, Allan McRae wrote:

Dan McGee wrote:

...

On Mon, Jun 2, 2008 at 9:56 AM, Allan McRae wrote:

...

Dan McGee wrote:

...

<snip> + local netfile + for netfile in ${source[@]}; do + local file=$(strip_url "$netfile") + if [ -f $netfile ]; then + msg2 "$(gettext "Adding %s...")" "$netfile" + comp_files="$comp_files $netfile" + elif [ "$SOURCEONLY" = "2" -a -f "$srcdir/$file" ]; then + msg2 "$(gettext "Adding %s...")" "$file" + comp_files="$comp_files -C $srcdir $file"

I'm a little confused here. The file $srcdir/$file is just a link to $SRCDEST/$file which may be $startdir/$file. So does this not just add the symbolic link to the tarball. Also, the -C option flags all following files to be from the $srcdir directory. So, anything added after this will also get its symbolic link added.

If I'm not going down the wrong track here, this should fix it...

Yeah, this wasn't tested super hard or anything so I think you are on to something here. The -C option is definitely a bit odd but it kind of does what we want. Since all files are available in src/ (although some might be symlinks), can't we use 1 -C flag in combination with -H to follow symlinks to our source files?

The files are not in src/ if the --source was used so we have to be careful there.

...

While we are making changes, can you see an easy way to get everything under a pkgname/ dir (maybe with pkgver-pkgrel too) inside the tarball? Unfortunately bsdtar has no --prefix option to easily do this, and our friend Loui says it is doable but has not offered a line of code.

How about: - Make directory of form pkgname-pkgver-pkgrel in a temporary directory - Instead of collecting files in "comp_files", just place links into that directory. - Use bsdtar with -L flag on that directory. - Clean up

I think that is along the lines of what Loui was suggesting. It also removes the whole -C issue without adding any further SOURCEONLY=2 checks.

Yeah, I think I like this idea. Feel free to rip my patch apart. :) We might want to take a closer look at how this is all done as the original --source patch was written quickly, and the --allsource option above was kind of jury-rigged in. I did like the fact that if we download all sources we can verify the checksums- perhaps we can still find a way to check md5sums for the local-only source as well before we package it up in an src.tar.gz? We should be able to do some kind of similar action: for file in source: if netfile && allsource then verify md5sum if localfile then verify md5sum Maybe add a flag or something to the checksum function to determine whether to check netfiles or not. Wow, this patch is going to turn into a sequence of 5. -Dan