[pacman-dev] Fwd: [makepkg patch - generate .SRCINFO file when running "--source"]
Hello, Here is a mail I sent to the arch-general list containing a patch for makepkg which generates a .SRCINFO file when running "makepkg --source". Below I'll make some more comments, present some more ideas on this topic. ----- Forwarded message from vlad <vla@uni-bonn.de> ----- Date: Tue, 27 Jul 2010 15:25:55 +0200 From: vlad <vla@uni-bonn.de> Subject: makepkg patch - generate .SRCINFO file when running "--source" To: arch-general <arch-general@archlinux.org> User-Agent: Mutt/1.5.20 (2009-06-14) Hello list, Here is a patch against makepkg from git which introduces a new function "write_srcinfo()". This generates a file .SRCINFO - like the .PKGINFO one - when "makepkg --source" is run and then it is added to the src.tar.gz archive. I think having such a file is an important step for getting split packages to the AUR. It is also useful for third party applications. Since this file is generated during making the source archive, there is no need for parsing/sourcing the PKGBUILD everytime meta infos are needed afterwards. This is also a way of standardizing the source archive. .SRCINFO looks like (generated for gcc from core): " # Generated by makepkg 3.4.0 # Tue Jul 27 12:56:38 UTC 2010 global pkgbase = gcc global pkgname = gcc global pkgname = gcc-libs global pkgname = gcc-fortran global pkgname = gcc-objc global pkgname = gcc-ada global pkgver = 4.5.0-6 global pkgdesc = The GNU Compiler Collection global url = http://gcc.gnu.org global packager = Vlad George <donvla@users.sourceforge.net> global builddate = 1280235398 global source = ftp://gcc.gnu.org/pub/gcc/snapshots/4.5-20100610/gcc-core-4.5-20100610.tar.bz2 global source = ftp://gcc.gnu.org/pub/gcc/snapshots/4.5-20100610/gcc-g++-4.5-20100610.tar.bz2 global source = ftp://gcc.gnu.org/pub/gcc/snapshots/4.5-20100610/gcc-fortran-4.5-20100610.tar.bz2 global source = ftp://gcc.gnu.org/pub/gcc/snapshots/4.5-20100610/gcc-objc-4.5-20100610.tar.bz2 global source = ftp://gcc.gnu.org/pub/gcc/snapshots/4.5-20100610/gcc-ada-4.5-20100610.tar.bz2 global source = ftp://gcc.gnu.org/pub/gcc/libstdc++/doxygen/libstdc++-man.20100312.tar.bz2 global source = gcc_pure64.patch global source = gcc-hash-style-both.patch global source = r160561.patch global license = GPL global license = LGPL global license = custom global arch = i686 global arch = x86_64 global makedepends = binutils>=2.20.1 global makedepends = libmpc>=0.8.2-2 global makedepends = cloog-ppl>=0.15.8 global makedepends = libelf global makedepends = gcc-ada gcc depends = binutils>=2.20.1 gcc depends = libmpc>=0.8.1-2 gcc depends = cloog-ppl>=0.15.8 gcc depends = libelf gcc groups = base-devel gcc install = gcc.install gcc-libs pkgdesc = Runtime libraries shipped by GCC for C and C++ languages gcc-libs groups = base gcc-libs depends = glibc>=2.11.1-2 gcc-libs install = gcc-libs.install gcc-fortran pkgdesc = Fortran front-end for GCC gcc-fortran depends = gcc=4.5.0-6 gcc-fortran install = gcc-fortran.install gcc-objc pkgdesc = Objective-C front-end for GCC gcc-objc depends = gcc=4.5.0-6 gcc-ada pkgdesc = Ada front-end for GCC (GNAT) gcc-ada depends = gcc=4.5.0-6 gcc-ada install = gcc-ada.install " As you see this file consists of different parts separated by a leading tag. "global" denotes the global PKGBUILD vars and the name of the split package denotes the local package variables. However, this is just a first formatting approach. This can be changed according to further needs. I think this is a clear and easily readable/parseable output. Note that "packager" defaults to the person who actually build the source archive. Perhaps this has to be changed to AUR user name, not to the makepkg.conf PACKAGER entry. ----- End forwarded message ----- The main idea behind this is to unify and standardize how PKGBUILDs are handled (mainly AUR). For now information about a package has to be parsed from the PKGBUILD which creates a big overhead and has many security holes. There are some more problems with this. As a user said, there is a problem with if clauses for different arches. Imo there is no sane way to handle this. So we should try a different approach. A possible one is to restrict the way architecture specific variables are written to PKGBUILDs; perhaps only smth like: "[[ $CARCH == "i686" ]] && sources= ... " Though this is quite restrictive, and I see a discussion coming up why this and not the other way. A more elegant way - which I personally prefer - is to introduce a new function "arch_$CARCH" to PKGBUILDs. Actually these are two functions, arch_i686 and arch_x86_64, one for each architecture. These functions appear in the first (global) part of the PKGBUILD before the build/package functions and contain arch specific variables. This may look like (eg for new wine PKGBUILD): " arch=(i686 x86_64) license=(LGPL) arch_i686() { depends=(fontconfig mesa libxcursor libxrandr libxdamage libxxf86dga alsa-lib) makedepends=(autoconf ncurses bison perl fontforge flex prelink 'gcc>=4.5.0-2' cross32-gcc libgphoto2 giflib libxpm libpng libxinerama libxcomposite \ libxmu libxxf86vm libxml2 libxslt libldap lcms mpg123 openal esound jack cups sane samba) optdepends=(libgphoto2 giflib libpng libldap lcms libxml2 mpg123 openal esound jack cups sane samba) } arch_x86_64() { depends=(...) } source=("stuff") md5sums=("numbers") build() { .... } " You get the point. Note that only vars are allowed in these functions; other entries are ignored. There should also be a list/array of allowed vars (like "splitpkg_overrides"). There are: depends, makedepends, optdepends, source, checksums. Note that this does not affect the build/package functions in any way. The main advantages are: * This is a unified way to handle arch specific vars in PKGBUILDs. There are no dirty hacks, no barely readable if clauses, etc. * This is easy to parse - if needed - and easily integrable into makepkg. After sourcing the buildfile, you can first check if there is such a function and than simply run it (eg: "declare -F arch_$CARCH && arch_$CARCH"). In combination with .SRCINFO there is no need to parse the PKGBUILD at all afterwards. * And for the .SRCINFO file: with these functions you can expand (only an example - as I said above, this formatting is just one possibility) the separation tags to "global_i686" and "global_x86_64" which point to the arch specific vars. This completes the output/writing of this file. And now to the AUR. Every "taurball" which is uploaded to the AUR has to be built with "makepkg --source". It contains then a .SRCINFO file which makes the information about this package easy readable. So there is a sane and consistent way to provide and get information for wanted purposes (AUR web interface, third party apps, etc). Thank you for reading, Vlad --
Sorry, forgot the patch. Here it is. Vlad --
On 03/08/10 19:22, vlad wrote:
Hello,
Here is a mail I sent to the arch-general list containing a patch for makepkg which generates a .SRCINFO file when running "makepkg --source". Below I'll make some more comments, present some more ideas on this topic.
Overall, I am quite happy with this idea. Obviously we could tidy up the implementation a bit at a later stage. First I think we need to get the formatting issues sorted, as once the format is set, it becomes very difficult to change. <snip>
.SRCINFO looks like (generated for gcc from core): " # Generated by makepkg 3.4.0 # Tue Jul 27 12:56:38 UTC 2010 global pkgbase = gcc global pkgname = gcc global pkgname = gcc-libs global pkgname = gcc-fortran global pkgname = gcc-objc global pkgname = gcc-ada global pkgver = 4.5.0-6
This needs to be separated into pkgver/pkgrel as both of these can be overridden in split packages. ...
gcc depends = binutils>=2.20.1 gcc depends = libmpc>=0.8.1-2 gcc depends = cloog-ppl>=0.15.8 gcc depends = libelf gcc groups = base-devel gcc install = gcc.install
I think the whole "global"/"splitpkg" leading entry seems fine. <snip>
Note that "packager" defaults to the person who actually build the source archive. Perhaps this has to be changed to AUR user name, not to the makepkg.conf PACKAGER entry.
Pacman and makepkg are distribution agnostic so the user would have to change the PACKAGER entry if it was deemed necessary to match the AUR username (which it should not have to...).
The main idea behind this is to unify and standardize how PKGBUILDs are handled (mainly AUR). For now information about a package has to be parsed from the PKGBUILD which creates a big overhead and has many security holes. There are some more problems with this. As a user said, there is a problem with if clauses for different arches. Imo there is no sane way to handle this. So we should try a different approach. A possible one is to restrict the way architecture specific variables are written to PKGBUILDs; perhaps only smth like: "[[ $CARCH == "i686" ]]&& sources= ... " Though this is quite restrictive, and I see a discussion coming up why this and not the other way.
I am against imposing restrictions to formatting like that as there is always something that needs to be done differently...
A more elegant way - which I personally prefer - is to introduce a new function "arch_$CARCH" to PKGBUILDs. Actually these are two functions, arch_i686 and arch_x86_64, one for each architecture. These functions appear in the first (global) part of the PKGBUILD before the build/package functions and contain arch specific variables. This may look like (eg for new wine PKGBUILD): " arch=(i686 x86_64) license=(LGPL)
arch_i686() { depends=(fontconfig mesa libxcursor libxrandr libxdamage libxxf86dga alsa-lib) makedepends=(autoconf ncurses bison perl fontforge flex prelink 'gcc>=4.5.0-2' cross32-gcc libgphoto2 giflib libxpm libpng libxinerama libxcomposite \ libxmu libxxf86vm libxml2 libxslt libldap lcms mpg123 openal esound jack cups sane samba) optdepends=(libgphoto2 giflib libpng libldap lcms libxml2 mpg123 openal esound jack cups sane samba) }
arch_x86_64() { depends=(...) }
source=("stuff") md5sums=("numbers")
build() { .... } " You get the point. Note that only vars are allowed in these functions; other entries are ignored. There should also be a list/array of allowed vars (like "splitpkg_overrides"). There are: depends, makedepends, optdepends, source, checksums. Note that this does not affect the build/package functions in any way. The main advantages are: * This is a unified way to handle arch specific vars in PKGBUILDs. There are no dirty hacks, no barely readable if clauses, etc. * This is easy to parse - if needed - and easily integrable into makepkg. After sourcing the buildfile, you can first check if there is such a function and than simply run it (eg: "declare -F arch_$CARCH&& arch_$CARCH"). In combination with .SRCINFO there is no need to parse the PKGBUILD at all afterwards. * And for the .SRCINFO file: with these functions you can expand (only an example - as I said above, this formatting is just one possibility) the separation tags to "global_i686" and "global_x86_64" which point to the arch specific vars. This completes the output/writing of this file.
I am not sure about this. Despite it being simpler on the surface, how would you handle this within a split package function? I could see the need for different dependencies depending on architecture within a split package. Allan
On Wed, Aug 04, 2010 at 01:27:43PM +1000, Allan McRae wrote:
Overall, I am quite happy with this idea. Obviously we could tidy up the implementation a bit at a later stage.
First I think we need to get the formatting issues sorted, as once the format is set, it becomes very difficult to change. It is just a start.
Note that "packager" defaults to the person who actually build the source archive. Perhaps this has to be changed to AUR user name, not to the makepkg.conf PACKAGER entry.
Pacman and makepkg are distribution agnostic so the user would have to change the PACKAGER entry if it was deemed necessary to match the AUR username (which it should not have to...). I think the "packager" should be the maintainer. I see no problem
... therein. This can be simply overriden with "PACKAGER="" ; makepkg --source". ...
A possible one is to restrict the way architecture specific variables are written to PKGBUILDs; perhaps only smth like: "[[ $CARCH == "i686" ]]&& sources= ... " Though this is quite restrictive, and I see a discussion coming up why this and not the other way.
I am against imposing restrictions to formatting like that as there is always something that needs to be done differently... Me too. But I also dislike to see here too creative constructs instead of a simple "if" or "[[". See the wine PKGBUILD. There are tons of dirty hacks in bash which are totally intransparent, but still somehow work.
Note that only vars are allowed in these functions; other entries are ignored. There should also be a list/array of allowed vars (like "splitpkg_overrides"). There are: depends, makedepends, optdepends, source, checksums. Note that this does not affect the build/package functions in any way. The main advantages are: * This is a unified way to handle arch specific vars in PKGBUILDs. There are no dirty hacks, no barely readable if clauses, etc. * This is easy to parse - if needed - and easily integrable into makepkg. After sourcing the buildfile, you can first check if there is such a function and than simply run it (eg: "declare -F arch_$CARCH&& arch_$CARCH"). In combination with .SRCINFO there is no need to parse the PKGBUILD at all afterwards. * And for the .SRCINFO file: with these functions you can expand (only an example - as I said above, this formatting is just one possibility) the separation tags to "global_i686" and "global_x86_64" which point to the arch specific vars. This completes the output/writing of this file.
I am not sure about this. Despite it being simpler on the surface, how would you handle this within a split package function? I could see the need for different dependencies depending on architecture within a split package. That's a point I also stumbled upon (not so hard to do so...). I have here two suggestions. First, since the .SRCINFO file should not be a "replacement" for the buildfile, some information can simply be dropped :D . The arch_$CARCH should only appear in the "global" part of
... the buildfile. Though this results in a pretty inconsitent handling (why here and not there...?). Another idea I have, is to provide some internal makepkg "helper functions" like gentoo has inside ebuilds. Some like "msg" or "error" are already used in PKGBUILDs. But these are only output formatting functions. Some are also used in install files. So this is not too unorthodox. It is possible to introduce arch specific helper functions. As a thought: " for_i686 "depends=('bla' 'bla')" for_x86_64 "source=(http://... file://)" " And these functions could look inside makepkg like: " for_i686() { [[ "$CARCH" == "i686" ]] && eval "$@" } " As you see, it moves the "if" checks inside makepkg and you get a uniform way to handle this inside the buildfile. And last but not least, these functions can also be used inside "package_" functions. What do you think? Vlad --
On Tue 03 Aug 2010 11:22 +0200, vlad wrote:
Here is a mail I sent to the arch-general list containing a patch for makepkg which generates a .SRCINFO file when running "makepkg --source". Below I'll make some more comments, present some more ideas on this topic.
As you see this file consists of different parts separated by a leading tag. "global" denotes the global PKGBUILD vars and the name of the split package denotes the local package variables. However, this is just a first formatting approach. This can be changed according to further needs. I think this is a clear and easily readable/parseable output. Note that "packager" defaults to the person who actually build the source archive. Perhaps this has to be changed to AUR user name, not to the makepkg.conf PACKAGER entry.
Nah. I think an online repo would keep the PACKAGER variable as-is. There might be something separate like 'uploader', or 'maintainer'. It would be like uploading an mp3 to a site. The artist id3 tag wouldn't replaced with the uploader's name. Not that anyone does such things.
participants (3)
-
Allan McRae
-
Loui Chang
-
vlad