[pacman-dev] [PATCH] add makedepends to package and database
Hi, I think it would be nice to have information about makedepnds available in each pacakge and the pacakge database. Think about rebuilds (statically linked) or the question if a certain package can be (re)moved. -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Am Sonntag 14 Juni 2009 10:08:27 schrieb Pierre Schmitz:
Hi,
I think it would be nice to have information about makedepnds available in each pacakge and the pacakge database. Think about rebuilds (statically linked) or the question if a certain package can be (re)moved.
Let's try it inline this time... From 7d3392a930a995c34e641f77a1936eedd3fff1c3 Mon Sep 17 00:00:00 2001 From: Pierre Schmitz <pierre@archlinux.de> Date: Sun, 14 Jun 2009 09:51:52 +0200 Subject: [PATCH 2/2] add makedepends to package and database This information makes it possible to create a complete build time dependency tree from a package itself or even the db file. --- scripts/makepkg.sh.in | 3 +++ scripts/repo-add.sh.in | 4 +++- 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 8e7d535..c59dbcc 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -927,6 +927,9 @@ create_package() { for it in "${groups[@]}"; do echo "group = $it" >>.PKGINFO done + for it in "${makedepends[@]}"; do + echo "makedepend = $it" >>.PKGINFO + done for it in "${depends[@]}"; do echo "depend = $it" >>.PKGINFO done diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index ccb4dda..5455290 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -184,7 +184,7 @@ db_write_entry() # blank out all variables local pkgfile="$1" local pkgname pkgver pkgdesc csize size md5sum url arch builddate packager force \ - _groups _licenses _replaces _depends _conflicts _provides _optdepends + _groups _licenses _replaces _makedepends _depends _conflicts _provides _optdepends local OLDIFS="$IFS" # IFS (field separator) is only the newline character @@ -203,6 +203,7 @@ db_write_entry() group) _groups="$_groups$group\n" ;; license) _licenses="$_licenses$license\n" ;; replaces) _replaces="$_replaces$replaces\n" ;; + makedepend) _makedepends="$_makedepends$makedepend\n" ;; depend) _depends="$_depends$depend\n" ;; conflict) _conflicts="$_conflicts$conflict\n" ;; provides) _provides="$_provides$provides\n" ;; @@ -265,6 +266,7 @@ db_write_entry() msg2 "$(gettext "Creating 'depends' db entry...")" # create the file even if it will remain empty touch "depends" + write_list_entry "MAKEDEPENDS" "$_makedepends" "depends" write_list_entry "DEPENDS" "$_depends" "depends" write_list_entry "CONFLICTS" "$_conflicts" "depends" write_list_entry "PROVIDES" "$_provides" "depends" -- 1.6.3.2 -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Pierre Schmitz wrote:
Hi,
I think it would be nice to have information about makedepnds available in each pacakge and the pacakge database. Think about rebuilds (statically linked) or the question if a certain package can be (re)moved.
I'm not really a fan of this as it fills the PKGINFO file and pacman DB with info that is really not needed. At the moment you can simply grep the ABS tree to check for depends/makedepends. Allan
Am Sonntag 14 Juni 2009 10:28:00 schrieb Allan McRae:
I'm not really a fan of this as it fills the PKGINFO file and pacman DB with info that is really not needed. At the moment you can simply grep the ABS tree to check for depends/makedepends.
I am not sure myselfM; that's why I brought it up here. Of course this information is completely useless for user and just wastes space. But here I wonder if such a few bytes more really matter. On the other side this is really usefull if you write tools for packagers. The advantage of retreiving the information from the package/db over grepping the ABS tree is: * it does not depend on Arch * there is no need to check out the complete tree * tools can use a unified way to access all kinds of deps So, only a very limited group will benefit from this and if all others will have disadvantages we should probably drop the idea. -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
On Sunday 14 June 2009 10:47:38 Pierre Schmitz wrote:
But here I wonder if such a few bytes more really matter.
To add some facts here. I checked the overhead of adding makedepends to all packages in core, extra, community and testing, The uncompressed overall size is 7,9KB and gip compressed about 1,6KB. Here the little script: #!/bin/bash for i in $(find /var/abs/ -name PKGBUILD); do unset makedepends . $i if [ "$makedepends" != "" ]; then echo 'MAKEDEPENDS' for j in ${makedepends[@]}; do echo $j done fi done -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Pierre Schmitz wrote:
Am Sonntag 14 Juni 2009 10:28:00 schrieb Allan McRae:
I'm not really a fan of this as it fills the PKGINFO file and pacman DB with info that is really not needed. At the moment you can simply grep the ABS tree to check for depends/makedepends.
I am not sure myselfM; that's why I brought it up here.
Of course this information is completely useless for user and just wastes space. But here I wonder if such a few bytes more really matter.
On the other side this is really usefull if you write tools for packagers. The advantage of retreiving the information from the package/db over grepping the ABS tree is: * it does not depend on Arch * there is no need to check out the complete tree * tools can use a unified way to access all kinds of deps
So, only a very limited group will benefit from this and if all others will have disadvantages we should probably drop the idea.
Seeing the numbers you provided in the other email and the advantages indicated here, I am now fine with including this in the pacman-db. It would be quite useful for my rebuild order script which currently misses makedepends.... Allan
On Mon, Jun 15, 2009 at 8:30 PM, Allan McRae<allan@archlinux.org> wrote:
Pierre Schmitz wrote:
Am Sonntag 14 Juni 2009 10:28:00 schrieb Allan McRae:
I'm not really a fan of this as it fills the PKGINFO file and pacman DB with info that is really not needed. At the moment you can simply grep the ABS tree to check for depends/makedepends.
I am not sure myselfM; that's why I brought it up here.
Of course this information is completely useless for user and just wastes space. But here I wonder if such a few bytes more really matter.
On the other side this is really usefull if you write tools for packagers. The advantage of retreiving the information from the package/db over grepping the ABS tree is: * it does not depend on Arch * there is no need to check out the complete tree * tools can use a unified way to access all kinds of deps
So, only a very limited group will benefit from this and if all others will have disadvantages we should probably drop the idea.
Seeing the numbers you provided in the other email and the advantages indicated here, I am now fine with including this in the pacman-db. It would be quite useful for my rebuild order script which currently misses makedepends....
To further rationalize, it also helps answer the question "How was this package built?"
On Mon, Jun 15, 2009 at 8:30 PM, Allan McRae<allan@archlinux.org> wrote:
Pierre Schmitz wrote:
Am Sonntag 14 Juni 2009 10:28:00 schrieb Allan McRae:
I'm not really a fan of this as it fills the PKGINFO file and pacman DB with info that is really not needed. At the moment you can simply grep the ABS tree to check for depends/makedepends.
I am not sure myselfM; that's why I brought it up here.
Of course this information is completely useless for user and just wastes space. But here I wonder if such a few bytes more really matter.
On the other side this is really usefull if you write tools for packagers. The advantage of retreiving the information from the package/db over grepping the ABS tree is: * it does not depend on Arch * there is no need to check out the complete tree * tools can use a unified way to access all kinds of deps
So, only a very limited group will benefit from this and if all others will have disadvantages we should probably drop the idea.
Seeing the numbers you provided in the other email and the advantages indicated here, I am now fine with including this in the pacman-db. It would be quite useful for my rebuild order script which currently misses makedepends....
To further rationalize, it also helps answer the question "How was this package built?"
Would this let you rebuild a package from the database alone? Is the other info from the PKGBUILD along with the local source files included somewhere? Can someone give some concrete examples of how this could be useful in the absence of the other packaging information and who would be likely to use this?
On Tue, Jun 16, 2009 at 12:44 PM, Xyne<xyne@archlinux.ca> wrote:
On Mon, Jun 15, 2009 at 8:30 PM, Allan McRae<allan@archlinux.org> wrote:
Pierre Schmitz wrote:
Am Sonntag 14 Juni 2009 10:28:00 schrieb Allan McRae:
I'm not really a fan of this as it fills the PKGINFO file and pacman DB with info that is really not needed. At the moment you can simply grep the ABS tree to check for depends/makedepends.
I am not sure myselfM; that's why I brought it up here.
Of course this information is completely useless for user and just wastes space. But here I wonder if such a few bytes more really matter.
On the other side this is really usefull if you write tools for packagers. The advantage of retreiving the information from the package/db over grepping the ABS tree is: * it does not depend on Arch * there is no need to check out the complete tree * tools can use a unified way to access all kinds of deps
So, only a very limited group will benefit from this and if all others will have disadvantages we should probably drop the idea.
Seeing the numbers you provided in the other email and the advantages indicated here, I am now fine with including this in the pacman-db. It would be quite useful for my rebuild order script which currently misses makedepends....
To further rationalize, it also helps answer the question "How was this package built?"
Would this let you rebuild a package from the database alone? Is the other info from the PKGBUILD along with the local source files included somewhere?
Can someone give some concrete examples of how this could be useful in the absence of the other packaging information and who would be likely to use this?
Maybe "built" was too strong of a word... what I was trying to say: You can see, somewhat, what was enabled at build time, to see what things support, in some cases.
Can someone give some concrete examples of how this could be useful in the absence of the other packaging information and who would be likely to use this?
Maybe "built" was too strong of a word... what I was trying to say: You can see, somewhat, what was enabled at build time, to see what things support, in some cases.
Sorry if it's a noobish question, but can you give some example of when that info might be useful? I can't think of anything practical at the moment that would rely on knowing the makedepends but not the rest of it, at least not in the pacman database.
On Tue, Jun 16, 2009 at 2:14 PM, Xyne<xyne@archlinux.ca> wrote:
Can someone give some concrete examples of how this could be useful in the absence of the other packaging information and who would be likely to use this?
Maybe "built" was too strong of a word... what I was trying to say: You can see, somewhat, what was enabled at build time, to see what things support, in some cases.
Sorry if it's a noobish question, but can you give some example of when that info might be useful? I can't think of anything practical at the moment that would rely on knowing the makedepends but not the rest of it, at least not in the pacman database.
I'm not thinking in terms of scripting and the like, I'm thinking in terms of information. Like... "oh this needed mysql-libs to build, so maybe it has mysql support"
I'm not thinking in terms of scripting and the like, I'm thinking in terms of information. Like... "oh this needed mysql-libs to build, so maybe it has mysql support"
I don't think it would be all that useful in that case. You wouldn't query the database to find all packages that might have mysql support. If you were interested in that you would probably have a few apps in mind the features of which you would like to compare. If you're trying to determine if an app supports what you need, you would check the website or the manpage, wouldn't you? As pacman would have no direct use for the makedepends, I don't think it would be worth adding them to the database. To achieve what you've just mentioned, adding some sort of tag support would make more sense and be far more useful. I don't actually support that idea though, but listing the makedepends as currently suggested would effectively create a glorified tag system with restricted input. Perhaps another solution would be to convert the abs tree into a third database alongside sync and local. It would be completely optional as abs currently is but once installed pacman (or another binary built for that purpose) would be able to query it for such things as makedepends in a format similar to the current sync and local queries. It would also make it possible to seemlessly build files from source with dependency resolution. Maintaining the database itself is not an issue as the abs tree already exists. Distribution would require archiving each package's PKGBUILD and local source files for distribution on the mirrors similar to the way binary packages currently are (which would take a load of the current abs server and be far smaller than any of the binary repos). The database archive itself would contain files with information extracted from PKGBUILD in the same format followed by desc and depends files in the sync and local databases (including information such as makedepends, optdepends, sourceURLs, etc).
On Tuesday 16 June 2009 22:38:52 Aaron Griffin wrote:
I'm not thinking in terms of scripting and the like, I'm thinking in terms of information. Like... "oh this needed mysql-libs to build, so maybe it has mysql support"
Imho the other way round is more usefull: * What packages depend on e.g. mysql at build time? ** Maybe I want to move/remove it. Quite often we want to know if a package can safely be removed. ** Maybe I have a .so bump and want to know what might break in future rebuilds ** Maybe I want to rebuild those that are statically linked and rebuild them (e.g. bug fixes, security etc.) * With a complete dependency graph I can automaticaly check for unused libs etc. (think about repo cleanup) * Make a repo check/validation easier. E.g. cross repo dependencies, ... -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Pierre Schmitz wrote:
On Tuesday 16 June 2009 22:38:52 Aaron Griffin wrote:
I'm not thinking in terms of scripting and the like, I'm thinking in terms of information. Like... "oh this needed mysql-libs to build, so maybe it has mysql support"
Imho the other way round is more usefull: * What packages depend on e.g. mysql at build time? ** Maybe I want to move/remove it. Quite often we want to know if a package can safely be removed. ** Maybe I have a .so bump and want to know what might break in future rebuilds ** Maybe I want to rebuild those that are statically linked and rebuild them (e.g. bug fixes, security etc.) * With a complete dependency graph I can automaticaly check for unused libs etc. (think about repo cleanup) * Make a repo check/validation easier. E.g. cross repo dependencies, ...
Note that all of these are easily done already... either grepping through the ABS tree or using the rebuild scripts that I posted to the arch-dev-public list. The only advantage I see of this is that instead of grepping through the ABS tree (which may not be implemented for all users of pacman...), you can grep through the pacman sync db. I see no reason for adding this to the -S/Qi output as when it is important, it will be displayed in optdepends. Allan
Note that all of these are easily done already... either grepping through the ABS tree or using the rebuild scripts that I posted to the arch-dev-public list.
The only advantage I see of this is that instead of grepping through the ABS tree (which may not be implemented for all users of pacman...), you can grep through the pacman sync db. I see no reason for adding this to the -S/Qi output as when it is important, it will be displayed in optdepends.
Considering that only a very few people will actually use this, is it really worth adding to the database of all users if it's already possible to do this with the abs tree? Won't most people who would use this already have the abs tree installed?
On Wednesday 17 June 2009 06:39:49 Xyne wrote:
Considering that only a very few people will actually use this, is it really worth adding to the database of all users if it's already possible to do this with the abs tree? Won't most people who would use this already have the abs tree installed?
Sure, only few people would benefit from these changes. But it will make tools easier and queries are more unified. But let's not forget that the only downside of this extremely simple patch is adding a few bytes to the db/package bytes. (What was it? ~1,6KB for all repos together including testing and community) -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Xyne wrote:
Note that all of these are easily done already... either grepping through the ABS tree or using the rebuild scripts that I posted to the arch-dev-public list.
The only advantage I see of this is that instead of grepping through the ABS tree (which may not be implemented for all users of pacman...), you can grep through the pacman sync db. I see no reason for adding this to the -S/Qi output as when it is important, it will be displayed in optdepends.
Considering that only a very few people will actually use this, is it really worth adding to the database of all users if it's already possible to do this with the abs tree? Won't most people who would use this already have the abs tree installed?
Do not think of pacman as Arch specific. Can you currently get this info easily for KDEmod? How about any of the other distros using pacman? Allan
On Wed, Jun 17, 2009 at 12:08 AM, Allan McRae<allan@archlinux.org> wrote:
Xyne wrote:
Note that all of these are easily done already... either grepping through the ABS tree or using the rebuild scripts that I posted to the arch-dev-public list.
The only advantage I see of this is that instead of grepping through the ABS tree (which may not be implemented for all users of pacman...), you can grep through the pacman sync db. I see no reason for adding this to the -S/Qi output as when it is important, it will be displayed in optdepends.
Considering that only a very few people will actually use this, is it really worth adding to the database of all users if it's already possible to do this with the abs tree? Won't most people who would use this already have the abs tree installed?
Do not think of pacman as Arch specific. Can you currently get this info easily for KDEmod? How about any of the other distros using pacman?
As a rather wild aside here... what would be the downside of sticking the entire PKGBUILD in the pacman DB? How much size would that add?
On Wednesday 17 June 2009 07:27:11 Aaron Griffin wrote:
As a rather wild aside here... what would be the downside of sticking the entire PKGBUILD in the pacman DB? How much size would that add?
Wont make that much sense (as it is not easily parsable and without patches etc. quite useless) However for community/ core/ extra/ kde-unstable/ local/ testing/ it is (gzip compressed) 1,1M But you're just kidding, right? -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
On Wed, Jun 17, 2009 at 12:34 AM, Pierre Schmitz<pierre@archlinux.de> wrote:
On Wednesday 17 June 2009 07:27:11 Aaron Griffin wrote:
As a rather wild aside here... what would be the downside of sticking the entire PKGBUILD in the pacman DB? How much size would that add?
Wont make that much sense (as it is not easily parsable and without patches etc. quite useless) However for community/ core/ extra/ kde-unstable/ local/ testing/ it is (gzip compressed) 1,1M
But you're just kidding, right?
Not exactly - I'm half serious. I mean, a) it'd give us all the information needed to build the package, b) it'd allow these scripts to work without an ABS tree. I'm not saying pacman would _use_ it, I just think it'd be kinda neat.
As a rather wild aside here... what would be the downside of sticking the entire PKGBUILD in the pacman DB? How much size would that add?
Wont make that much sense (as it is not easily parsable and without patches etc. quite useless) However for community/ core/ extra/ kde-unstable/ local/ testing/ it is (gzip compressed) 1,1M
But you're just kidding, right?
Not exactly - I'm half serious. I mean, a) it'd give us all the information needed to build the package, b) it'd allow these scripts to work without an ABS tree.
I'm not saying pacman would _use_ it, I just think it'd be kinda neat.
Did anyone consider the idea that I mentioned in my previous reply? A quick recap: 1) Don't change sync and local databases. 2) Add a new database, possibly named "abs" (/var/lib/pacman/abs) 3) Create an archive for each package that contains the files in the current abs tree: -- PKGBUILD -- local source files 4) Create an abs database archive similar to the current sync db archive that contains a file (or more) with information from the parsed PKGBUILD, e.g.: %NAME% foo %VERSION% 2.3 %DEPENDS% bar baz %MAKEDEPENDS% this that something else %SOURCES% foo-2.3.tar.gz example.patch foo.png etc 5) Distribute this abs repo on the servers alongside the other repos. The user would then sync the abs database (which would only contain small files with the PKGBUILD info, so it wouldn't take much space) and be able to download packages containing everything needed to build from source The user would have the choice of installing the abs database and would only download PKGBUILDs & local source files for what they actually intend to build, unlike the current abs tree which dumps everything on the users system (wasted bandwidth, wasted space). It would also be non-Arch specific.
participants (4)
-
Aaron Griffin
-
Allan McRae
-
Pierre Schmitz
-
Xyne