[pacman-dev] [RFC] Generation of a .SRCINFO file for source packages
Hello all, I had the idea of adding a .SRCINFO file to source packages generated by makepkg. I did some research to see if this had been previously proposed and where the discussion went. I've collected my thoughts on such an addition for your consideration. Previous proposal: The idea of generating a .SRCINFO file was previously proposed on the mailing list [1]. No format was ever agreed upon and discussion died off when no fixes were sent based off of developer suggestions. I've included my own proposed format to fuel discussion and maybe reach an ideal format. Proposed format with example (using "boost" PKGBUILD [2]): - Every variable (pkgver, depends, etc) is preceded by ($pkgname). - Every "global" (defined outside a package_* function) variable is explicitly listed for each $pkgname. - Every array (arch, depends, etc) has one line for each array element # Generated by makepkg 4.0.3 # Mon Aug 27 01:23:45 UTC 2012 (boost) pkgbase = boost (boost) pkgname = boost (boost) pkgver = 1.50.0 (boost) pkgrel = 2 (boost) epoch = 0 (boost) pkgdesc = Free peer-reviewed portable C++ source libraries - Development (boost) arch = i686 (boost) arch = x86_64 (boost) url = http://www.boost.org/ (boost) depends = boost-libs=1.50.0 (boost) optdepends = python: for python bindings (boost) optdepends = python2: for python2 bindings (boost) optdepends = boost-build: to use boost jam for building your project. (boost) makedepends = icu (boost) makedepends = python (boost) makedepends = python2 (boost) makedepends = bzip2 (boost) makedepends = zlib (boost) makedepends = openmpi (boost) source = http://downloads.sourceforge.net/boost/boost_1_50_0.tar.gz (boost) license = custom (boost-libs) pkgbase = boost (boost-libs) pkgname = boost-libs (boost-libs) pkgver = 1.50.0 (boost-libs) pkgrel = 2 (boost-libs) epoch = 0 (boost-libs) pkgdesc = Free peer-reviewed portable C++ source libraries - Runtime (boost-libs) arch = i686 (boost-libs) arch = x86_64 (boost-libs) url = http://www.boost.org/ (boost-libs) depends = gcc-libs (boost-libs) depends = bzip2 (boost-libs) depends = zlib (boost-libs) depends = icu (boost-libs) optdepends = openmpi: for mpi support (boost-libs) makedepends = icu (boost-libs) makedepends = python (boost-libs) makedepends = python2 (boost-libs) makedepends = bzip2 (boost-libs) makedepends = zlib (boost-libs) makedepends = openmpi (boost-libs) source = http://downloads.sourceforge.net/boost/boost_1_50_0.tar.gz (boost-libs) license = custom Advantages of proposed format: - Don't need a bash parser to determine PKGBUILD variables in a source package. This would make things a lot easier for projects (including AUR) that depend on information from PKGBUILD variables. - Previously proposed implementation could run into issues for a package named "global", this proposal avoids need for "global" label. Disadvantages of proposed format: - Adds additional dot file to every generated source package. - Compared to the previously proposed implementation, this implementation results in more lines (and a bit of redundancy) in each .SRCINFO file due to the avoidance of any "global" variables. Thanks for the consideration. I am more than happy to answer any questions relating to anything above. Regards, Jason [1] http://mailman.archlinux.org/pipermail/pacman-dev/2010-August/011461.html [2] https://projects.archlinux.org/svntogit/packages.git/plain/trunk/PKGBUILD?h=...
On Mon, Aug 27, 2012 at 05:08:45PM -0400, canyonknight@gmail.com wrote:
Hello all,
I had the idea of adding a .SRCINFO file to source packages generated by makepkg. I did some research to see if this had been previously proposed and where the discussion went. I've collected my thoughts on such an addition for your consideration.
Whoa. Someone who actually did their homework. I'm already excited.
Previous proposal: The idea of generating a .SRCINFO file was previously proposed on the mailing list [1]. No format was ever agreed upon and discussion died off when no fixes were sent based off of developer suggestions. I've included my own proposed format to fuel discussion and maybe reach an ideal format.
Proposed format with example (using "boost" PKGBUILD [2]): - Every variable (pkgver, depends, etc) is preceded by ($pkgname).
I don't like this. Why surround it with parenthesis? pacman in git is already fairly strict about what it allows in pkgname=, so if you're going to advertise this columnar format, I'd say just leave it untouched. (i.e. pkgname="foo bar" won't get past makepkg). The extra characters wrapping the name just makes parsing a little more tedious.
- Every "global" (defined outside a package_* function) variable is explicitly listed for each $pkgname. - Every array (arch, depends, etc) has one line for each array element
I like this, since its consistent with how we do .PKGINFO.
# Generated by makepkg 4.0.3 <snipped example>
As a counter example, what about the following. Similar to how we do the pacman DBs, each package is a "field", e.g. %PACKAGE% pkgname = boost pkgver = 1.50.0 depends = boot-libs=1.50.0 ... %PACKAGE% pkgname = boost-libs pkgver = 1.50.0 depends = icu depends = zlib depends = bzip2 ... Each section is delimited with a blank line, with the field name telling the parser ahead of time what sort of data to expect next.
Advantages of proposed format: - Don't need a bash parser to determine PKGBUILD variables in a source package. This would make things a lot easier for projects (including AUR) that depend on information from PKGBUILD variables. - Previously proposed implementation could run into issues for a package named "global", this proposal avoids need for "global" label.
Disadvantages of proposed format: - Adds additional dot file to every generated source package. - Compared to the previously proposed implementation, this implementation results in more lines (and a bit of redundancy) in each .SRCINFO file due to the avoidance of any "global" variables.
I don't think the redundancy here is a problem. Writing metadata should be dead simple to make reading it back in as equally straight forward.
Thanks for the consideration. I am more than happy to answer any questions relating to anything above.
Regards,
Jason
[1] http://mailman.archlinux.org/pipermail/pacman-dev/2010-August/011461.html [2] https://projects.archlinux.org/svntogit/packages.git/plain/trunk/PKGBUILD?h=...
On Mon, Aug 27, 2012 at 8:51 PM, Dave Reisner <d@falconindy.com> wrote: <snip>
- Every variable (pkgver, depends, etc) is preceded by ($pkgname).
I don't like this. Why surround it with parenthesis? pacman in git is already fairly strict about what it allows in pkgname=, so if you're going to advertise this columnar format, I'd say just leave it untouched. (i.e. pkgname="foo bar" won't get past makepkg). The extra characters wrapping the name just makes parsing a little more tedious.
My mistake, you are correct parentheses are unneeded thanks to the strictness of makepkg. <snip>
As a counter example, what about the following. Similar to how we do the pacman DBs, each package is a "field", e.g.
%PACKAGE% pkgname = boost pkgver = 1.50.0 depends = boot-libs=1.50.0 ...
%PACKAGE% pkgname = boost-libs pkgver = 1.50.0 depends = icu depends = zlib depends = bzip2 ...
Each section is delimited with a blank line, with the field name telling the parser ahead of time what sort of data to expect next.
I do think there is a certain nicety to my proposal where any random line can be parsed and the parser can know the package name and the variable declaration from that single line. I like your proposal as well though, and would be more than happy with it. Thanks for your response Dave. Regards, Jason
On 28/08/12 07:08, canyonknight@gmail.com wrote:
Hello all,
I had the idea of adding a .SRCINFO file to source packages generated by makepkg. I did some research to see if this had been previously proposed and where the discussion went. I've collected my thoughts on such an addition for your consideration.
Previous proposal: The idea of generating a .SRCINFO file was previously proposed on the mailing list [1]. No format was ever agreed upon and discussion died off when no fixes were sent based off of developer suggestions. I've included my own proposed format to fuel discussion and maybe reach an ideal format.
I am composing this offline so have not gone back through the old posts... but I thought the main issue was that the split package variables are not defined until running the package_* functions. I.e. we do not know the values of any overridden values unless we actually build the package. This is the main issue that needs solved.
Proposed format with example (using "boost" PKGBUILD [2]): - Every variable (pkgver, depends, etc) is preceded by ($pkgname). - Every "global" (defined outside a package_* function) variable is explicitly listed for each $pkgname. - Every array (arch, depends, etc) has one line for each array element
<snip> So much duplication! How about just: pkgbase = boost ... (all global values) ... pkgname = boost ... (overrides for boost) ... pkgname = boost-libs ... (overrides for boost-libs) ...
Advantages of proposed format: - Don't need a bash parser to determine PKGBUILD variables in a source package. This would make things a lot easier for projects (including AUR) that depend on information from PKGBUILD variables.
But you do if you need to extract the values without running a full build.
Disadvantages of proposed format: - Adds additional dot file to every generated source package.
No need to worry about that. Allan
On Mon, Aug 27, 2012 at 11:23 PM, Allan McRae <allan@archlinux.org> wrote: <snip>
Previous proposal: The idea of generating a .SRCINFO file was previously proposed on the mailing list [1]. No format was ever agreed upon and discussion died off when no fixes were sent based off of developer suggestions. I've included my own proposed format to fuel discussion and maybe reach an ideal format.
I am composing this offline so have not gone back through the old posts... but I thought the main issue was that the split package variables are not defined until running the package_* functions. I.e. we do not know the values of any overridden values unless we actually build the package.
This is the main issue that needs solved.
I believe the previously sent patch [1] was able to extract override values set within a package_* function. I think one of the bigger issues may have been related to values that are set for a specific architecture. I'll rebase the old patch and play around with it to see what the issues are. <snip>
So much duplication! How about just:
pkgbase = boost ... (all global values) ...
pkgname = boost ... (overrides for boost) ...
pkgname = boost-libs ... (overrides for boost-libs) ...
I'm not a huge fan of having global values in a .SRCINFO file. A .SRCINFO parser would have to process globals and then override them where appropriate, rather than processing all values directly. There is more duplication when global values aren't used, but it makes it dead simple for whatever parses the .SRCINFO file. Are globals something you would absolutely want in a .SRCINFO file? Regards, Jason [1] http://mailman.archlinux.org/pipermail/pacman-dev/2010-August/011462.html
On 29/08/12 07:48, canyonknight@gmail.com wrote:
On Mon, Aug 27, 2012 at 11:23 PM, Allan McRae <allan@archlinux.org> wrote:
<snip>
Previous proposal: The idea of generating a .SRCINFO file was previously proposed on the mailing list [1]. No format was ever agreed upon and discussion died off when no fixes were sent based off of developer suggestions. I've included my own proposed format to fuel discussion and maybe reach an ideal format.
I am composing this offline so have not gone back through the old posts... but I thought the main issue was that the split package variables are not defined until running the package_* functions. I.e. we do not know the values of any overridden values unless we actually build the package.
This is the main issue that needs solved.
I believe the previously sent patch [1] was able to extract override values set within a package_* function. I think one of the bigger issues may have been related to values that are set for a specific architecture. I'll rebase the old patch and play around with it to see what the issues are.
I can make it break.... package_bar() { _url="http://foo.com" url=$url/bar } Look at how crap our check_sanity function is to see how difficult it is to do this.
So much duplication! How about just:
pkgbase = boost ... (all global values) ...
pkgname = boost ... (overrides for boost) ...
pkgname = boost-libs ... (overrides for boost-libs) ...
I'm not a huge fan of having global values in a .SRCINFO file. A .SRCINFO parser would have to process globals and then override them where appropriate, rather than processing all values directly. There is more duplication when global values aren't used, but it makes it dead simple for whatever parses the .SRCINFO file. Are globals something you would absolutely want in a .SRCINFO file?
No, but I think it makes sense. There are global values in the PKGBUILD and they are overridden. I do not see how there is duplication when the global values are not used - unless they are unnecessarily put in the PKGBUILD. Allan
participants (3)
-
Allan McRae
-
canyonknight＠gmail.com
-
Dave Reisner