[pacman-dev] [PATCH] add support for .so dependencies

newer
[pacman-dev] [PATCH] makepkg: exit...

older
[pacman-dev] [PATCH] Configure:...

bluewind＠server-speed.net

15 Aug 2009 15 Aug '09

11:03 a.m.

From: Florian Pritz Support-by: brain0 Support-by: GNU\caustic --- etc/makepkg.conf.in | 3 ++- scripts/makepkg.sh.in | 43 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 1 deletions(-) diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index 3a3a415..c3caf3c 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -67,8 +67,9 @@ BUILDENV=(fakeroot !distcc color !ccache) #-- emptydirs: Leave empty directories in packages #-- zipman: Compress manual (man and info) pages in MAN_DIRS with gzip #-- purge: Remove files specified by PURGE_TARGETS +#-- sodepends: add .so files to depends/provides arrays # -OPTIONS=(strip docs libtool emptydirs zipman purge) +OPTIONS=(strip docs libtool emptydirs zipman purge !sodepends) #-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512 INTEGRITY_CHECK=(md5) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index daa9585..405f3cb 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -872,6 +872,33 @@ tidy_install() { fi } +find_sodependencies() +{ + find $pkgdir | while read filename + do + arch=$(readelf -h "$filename" 2> /dev/null| sed -nr 's/.*Class:.*ELF(.*).*/\1/p') + [ -n "$arch" ] || continue + + readelf -d "$filename" 2> /dev/null | sed -nr 's/.*Shared library: \[(.*)\].*/\1/p' \ + | while read dep + do + echo sodep-${arch}-${dep} + done + done +} + +find_soprovides() +{ + find $pkgdir -name \*.so | while read filename + do + arch=$(readelf -h "$filename" 2> /dev/null| sed -nr 's/.*Class:.*ELF(.*).*/\1/p') + [ -n "$arch" ] || continue + soname=$(readelf -d "$filename" 2>/dev/null | sed -nr 's/.*Library soname: \[(.*)\].*/\1/p') + [ -n "$soname" ] || continue + echo sodep-${arch}-${soname} + done +} + write_pkginfo() { local builddate=$(date -u "+%s") if [ -n "$PACKAGER" ]; then @@ -902,6 +929,22 @@ write_pkginfo() { echo "force = true" >> .PKGINFO fi + if [ "$(check_option sodepends)" = "y" ]; then + sodepends=$(find_sodependencies) + soprovides=$(find_soprovides) + + OLDIFS="$IFS" + IFS="" + + # filter .so files provided by the package itself + sodepends=$(echo $sodepends $soprovides $soprovides | sort | uniq -u) + + IFS="$OLDIFS" + + depends=($depends $sodepends) + provices=($provides $soprovides) + fi + local it for it in "${license[@]}"; do echo "license = $it" >>.PKGINFO -- 1.6.4

Show replies by date

Thomas Bächler

15 Aug 15 Aug

11:37 a.m.

First of all, this was partially my idea, so I second that such an approach bluewind@server-speed.net schrieb:

...

diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index 3a3a415..c3caf3c 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -67,8 +67,9 @@ BUILDENV=(fakeroot !distcc color !ccache) #-- emptydirs: Leave empty directories in packages #-- zipman: Compress manual (man and info) pages in MAN_DIRS with gzip #-- purge: Remove files specified by PURGE_TARGETS +#-- sodepends: add .so files to depends/provides arrays # -OPTIONS=(strip docs libtool emptydirs zipman purge) +OPTIONS=(strip docs libtool emptydirs zipman purge !sodepends)

These should be separate options for sodepends and soprovides. There are some reservations against sodepends, but adding soprovides as default option should be safe for all uses. The default options array should then be OPTIONS=(strip docs libtool emptydirs zipman purge soprovides !sodepends) Also, automatically adding sodepends will be problematic with optdepends. As for the rest of the patch, I'll let Dan or Allan verify that it will be okay in combination with split packages.

Allan McRae

2:08 p.m.

Hi, I'm guessing this patch was more a request for comments rather than for actual consideration given a few of the issues I point out below mean this will not actually work... So I will not comment on implementation at all. As far as the idea goes, I do not like it... This turns makepkg into a bit of a black box as you can no longer see the depends and provides from the PKGBUILD. Although, I do like the idea of having soname provides and depends in general. Allan

...

From: Florian Pritz

Support-by: brain0 Support-by: GNU\caustic --- etc/makepkg.conf.in | 3 ++- scripts/makepkg.sh.in | 43 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 1 deletions(-)

diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index 3a3a415..c3caf3c 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -67,8 +67,9 @@ BUILDENV=(fakeroot !distcc color !ccache) #-- emptydirs: Leave empty directories in packages #-- zipman: Compress manual (man and info) pages in MAN_DIRS with gzip #-- purge: Remove files specified by PURGE_TARGETS +#-- sodepends: add .so files to depends/provides arrays # -OPTIONS=(strip docs libtool emptydirs zipman purge) +OPTIONS=(strip docs libtool emptydirs zipman purge !sodepends)

#-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512 INTEGRITY_CHECK=(md5) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index daa9585..405f3cb 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -872,6 +872,33 @@ tidy_install() { fi }

+find_sodependencies() +{ + find $pkgdir | while read filename + do + arch=$(readelf -h "$filename" 2> /dev/null| sed -nr 's/.*Class:.*ELF(.*).*/\1/p') + [ -n "$arch" ] || continue

Hmmm.... not good. Think arch=('i686' 'x86_64')

...

+ + readelf -d "$filename" 2> /dev/null | sed -nr 's/.*Shared library: \[(.*)\].*/\1/p' \ + | while read dep + do + echo sodep-${arch}-${dep} + done + done +} + +find_soprovides() +{ + find $pkgdir -name \*.so | while read filename + do + arch=$(readelf -h "$filename" 2> /dev/null| sed -nr 's/.*Class:.*ELF(.*).*/\1/p') + [ -n "$arch" ] || continue

Again.

...

+ soname=$(readelf -d "$filename" 2>/dev/null | sed -nr 's/.*Library soname: \[(.*)\].*/\1/p') + [ -n "$soname" ] || continue + echo sodep-${arch}-${soname} + done +} + write_pkginfo() { local builddate=$(date -u "+%s") if [ -n "$PACKAGER" ]; then @@ -902,6 +929,22 @@ write_pkginfo() { echo "force = true" >> .PKGINFO fi

+ if [ "$(check_option sodepends)" = "y" ]; then + sodepends=$(find_sodependencies) + soprovides=$(find_soprovides) + + OLDIFS="$IFS" + IFS=""

Huh... IFS?

...

+ + # filter .so files provided by the package itself + sodepends=$(echo $sodepends $soprovides $soprovides | sort | uniq -u) + + IFS="$OLDIFS" + + depends=($depends $sodepends) + provices=($provides $soprovides)

"provides"

...

+ fi + local it for it in "${license[@]}"; do echo "license = $it" >>.PKGINFO

Christoph Schied

2:29 p.m.

Hi! Allan McRae wrote:

...

...
+find_sodependencies() +{ + find $pkgdir | while read filename + do + arch=$(readelf -h "$filename" 2> /dev/null| sed -nr 's/.*Class:.*ELF(.*).*/\1/p') + [ -n "$arch" ] || continue

Hmmm.... not good. Think arch=('i686' 'x86_64')

This was actually brain0s idea to make this feature compatible for an eventually coming multilib system. It should work without problems as your binaries should be linked against shared objects that match your arch. Or is it the naming of the variables that bothers you?

...

Huh... IFS?

I am not sure about the IFS messing, but I think its necessary to make sort and uniq work.

...

I'm guessing this patch was more a request for comments rather than for actual consideration given a few of the issues I point out below mean this will not actually work... So I will not comment on implementation at all.

I don't see a problem which really prevents this approach on dependency generation.

...

As far as the idea goes, I do not like it... This turns makepkg into a bit of a black box as you can no longer see the depends and provides from the PKGBUILD.

This shouldn't generate new dependencies, as makepkg already needs to know which packages are needed for building the package. So its a specialization of the already provided dependencies. If it's not, it's a bug in the PKGBUILD.

...

Although, I do like the idea of having soname provides and depends in general.

Good to hear that :) greetings, Christoph Schied

Allan McRae

2:45 p.m.

Christoph Schied wrote:

...

Hi!

Allan McRae wrote:

...
...
+find_sodependencies() +{ + find $pkgdir | while read filename + do + arch=$(readelf -h "$filename" 2> /dev/null| sed -nr 's/.*Class:.*ELF(.*).*/\1/p') + [ -n "$arch" ] || continue

Hmmm.... not good. Think arch=('i686' 'x86_64')

This was actually brain0s idea to make this feature compatible for an eventually coming multilib system. It should work without problems as your binaries should be linked against shared objects that match your arch. Or is it the naming of the variables that bothers you?

"arch" is a reserved variable name in PKGBUILDs. This will break split package support for packages that have both binary and arch=any packages (which will be supported in the future).

...

...
Huh... IFS?

I am not sure about the IFS messing, but I think its necessary to make sort and uniq work.

Please do not delete context. As far as I can tell, IFS does absolutely nothing here.

...

...
I'm guessing this patch was more a request for comments rather than for actual consideration given a few of the issues I point out below mean this will not actually work... So I will not comment on implementation at all.

I don't see a problem which really prevents this approach on dependency generation.

Well, the patch being broken and so I assume untested was my main issue. I will not review a patch the submitter has not even tested.

...

...
As far as the idea goes, I do not like it... This turns makepkg into a bit of a black box as you can no longer see the depends and provides from the PKGBUILD.

This shouldn't generate new dependencies, as makepkg already needs to know which packages are needed for building the package. So its a specialization of the already provided dependencies. If it's not, it's a bug in the PKGBUILD.

Well, then... what is the point? Versioned dependencies already cover soname values if the package is listed as a dep in addition to is library. Allan

matthewbruenig＠gmail.com

2:51 p.m.

When I worked on zenwalk's buildpkg which is basically a port of makepkg, we had something like this to find dependencies. It is painfully slow on applications of any significant size. Not a good idea. On Aug 15, 2009 9:45am, Allan McRae wrote:

...

Christoph Schied wrote:

...

Hi!

...

Allan McRae wrote:

...

+find_sodependencies()

...

+{

...

+ find $pkgdir | while read filename

...

+ do

...

+ arch=$(readelf -h "$filename" 2> /dev/null| sed -nr

...

's/.*Class:.*ELF(.*).*/\1/p')

...

+ [ -n "$arch" ] || continue

...

Hmmm.... not good. Think arch=('i686' 'x86_64')

...

This was actually brain0s idea to make this feature compatible for an

...

eventually coming multilib system. It should work without problems as

...

your binaries should be linked against shared objects that match your

...

arch. Or is it the naming of the variables that bothers you?

...

"arch" is a reserved variable name in PKGBUILDs. This will break split package support for packages that have both binary and arch=any packages (which will be supported in the future).

...

Huh... IFS?

...

I am not sure about the IFS messing, but I think its necessary to make

...

sort and uniq work.

...

Please do not delete context. As far as I can tell, IFS does absolutely nothing here.

...

I'm guessing this patch was more a request for comments rather than

...

for actual consideration given a few of the issues I point out below

...

mean this will not actually work... So I will not comment on

...

implementation at all.

...

I don't see a problem which really prevents this approach on dependency

...

generation.

...

Well, the patch being broken and so I assume untested was my main issue. I will not review a patch the submitter has not even tested.

...

As far as the idea goes, I do not like it... This turns makepkg into

...

a bit of a black box as you can no longer see the depends and provides

...

from the PKGBUILD.

...

This shouldn't generate new dependencies, as makepkg already needs to

...

know which packages are needed for building the package. So its a

...

specialization of the already provided dependencies. If it's not, it's a

...

bug in the PKGBUILD.

...

Well, then... what is the point? Versioned dependencies already cover soname values if the package is listed as a dep in addition to is library.

...

Allan

Christoph Schied

3:13 p.m.

On Sat, Aug 15, 2009 at 02:51:15PM +0000, matthewbruenig@gmail.com wrote:

...

When I worked on zenwalk's buildpkg which is basically a port of makepkg, we had something like this to find dependencies. It is painfully slow on applications of any significant size. Not a good idea.

I think it shouldn't that much of a problem, thats what we have split packages for, and one could disable the feature for big packages (thinking of texlive and that stuff). greetings, Christoph Schied

matthewbruenig＠gmail.com

3:25 p.m.

Therein lies a bit of the paradox of this kind of testing. Smaller packages really shouldn't need this kind of a check because a packager should be able to identify the dependencies necessary for a small package. Only the bigger packages that have dozens of dependencies would really be significantly aided by this. But for those packages, the feature would have to be disabled. Shouldn't namcap just do this and the packager can go back and fix his missed dependencies? Also I do agree that it decreases readability of PKGBUILD, the blackbox argument above. Just my thoughs. On Aug 15, 2009 10:13am, Christoph Schied wrote:

...

On Sat, Aug 15, 2009 at 02:51:15PM +0000, matthewbruenig@gmail.com wrote:

...

...
When I worked on zenwalk's buildpkg which is basically a port of makepkg,

...

...
we had something like this to find dependencies. It is painfully slow on

...

...
applications of any significant size. Not a good idea.

...

I think it shouldn't that much of a problem, thats what we have split

...

packages for, and one could disable the feature for big packages

...

(thinking of texlive and that stuff).

...

greetings, Christoph Schied

Florian Pritz

3:34 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 matthewbruenig@gmail.com wrote:

...

When I worked on zenwalk's buildpkg which is basically a port of makepkg, we had something like this to find dependencies. It is painfully slow on applications of any significant size. Not a good idea.

I just tested it with openoffice (15 sec), jre (4 sec), xorg-server (2 sec) mysql (1 sec). If you compare that to the compile time it's nothing. - -- Florian Pritz -- {flo,bluewind}@server-speed.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIcBAEBAgAGBQJKhtWOAAoJEG0WVcFM4cE+plYQAOa8C5PkZmzYT/KjdvU/Cyi7 mzJovqGD+29p2SHktnF3oKFMvi7jki4fbRKwul9zZKlRv1TOsH4ph3IdWiUj33T3 +YcUqfwn/VaEJCDePqmaI8RTwn+xzRX17m+jCmW0oADa4H4dRAi/rz3Php0vdoJF 99W/lfSa6RcscBlycVS4AQ+eyeaTnhEi7vGKsknS+zPRbX4e2+JCSThbb5ZqmZKb 37uk4Bxl2HkPHnRw8Bqkfj83drq2LikCBhagiCNsVQfK+/E0c24NABe3GE/mzYeC PuoLAZgIYDtwdVA94OSqyOGPFm/VsEaa5akHnbcRpmktNwu1nVQfJGAuDd1qCnp7 o1kBRisKdBVKF0g0ipxWl+mhf4gpURVc0M5o6cynFQqqeCffHgZZMO/uLfnZ/K8T f2QO5AECRMJMm/SEg2VxqUzL2FaXTTpVrNgBytd8qBF4T7bWKBOhGcjQAOuIW/a7 16hwIWMXhV/7Hoac1seSp7mPKM50Yr59ZtJDpwY1xoDcpg/0wxaY6OVPoVbiPLaP VJYiP9VLaMvuaUAZgBr2S+2QIKy/lo5N6IgATxtOFHVuy0NGt+RN2ViQ5F4i9kds G3xcTxNh02w+3GlSDI65JaZ1XCSw/kYK3eWZkPp0Af7ApJHXZ1TgHuvsRMlI/5Qb 6fj6r0ORqZLshbMqL/+V =U4AH -----END PGP SIGNATURE-----

Florian Pritz

3:04 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

...

...
...
I'm guessing this patch was more a request for comments rather than for actual consideration given a few of the issues I point out below mean this will not actually work... So I will not comment on implementation at all.

I don't see a problem which really prevents this approach on dependency generation.

Well, the patch being broken and so I assume untested was my main issue. I will not review a patch the submitter has not even tested.

I tested it by unpacking some packages and running a script that outputs the arrays. (copied from the patch) For the part where it merges the 2 arrays I only tested the depends stuff though and copied the provides part. Sadly made a mistake when replacing. Sorry. - -- Florian Pritz -- {flo,bluewind}@server-speed.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIcBAEBAgAGBQJKhs5nAAoJEG0WVcFM4cE+PUkP/1YrLkNyQqS1feacnMXfvaZt c34HZB/uRBTBa5O1XsQdbYPHAAn703+BnjaEi5+KbBsVQrxuoHGDEAputWyTQQPi 3eDEdNFCNlYNRYTK7G7y3ca2w94nlRwZy8ZV3IFlkcF2RnoDhGds3SV+7D7g4jod /Qg5fesYILwcBvlp7BQIRhFUejELMMRMs7+0F9VlNlTBS7MEMyCMeYaF+GsiP0Qs Og6u5tfBEe3bpnEHbyz5fcpoYvs+r3UBdKem0RfDh16b4h7BcEfXEfZ+LA7lIThn F1XOkU0HFICi3LSL9C/j3LVVM5cfDtECVq8NzRy2t3P7g0g4+MhnmEsTe10xfxaP LnT0A8c5GVt06dfXToQTDvua7anzcjomLUkL5Y8mmIZHJ7ihvlFaxA6Siy7kESl3 DIogb6L/+4vHTPxTGbI8jsB+SCUx/v1vUzp+KsTtkAy6FbGdn6V+P04aOqaGK2uL mmCNpYYgXCm8NI3XlJ5cfV7WcTTum77pc3RLlACQHi607YLR+sAPlPjUs4kq/ToZ vaAvtb6QdKxiSgNcC+WVvv8DWeAQ4ZPXDC6WFDJ2CbFOgy4fSEwID6x4H2VgTUvw 7wvPQObGNi7C08tQp7CyDkTW+leBwMGZj6PmeLpRngz2tGVXEhD5ZO40rYAh0coU 4SRkUTYe5TfJLOU2P/c9 =OWt/ -----END PGP SIGNATURE-----

Thomas Bächler

4:56 p.m.

Allan McRae schrieb:

...

...
This shouldn't generate new dependencies, as makepkg already needs to know which packages are needed for building the package. So its a specialization of the already provided dependencies. If it's not, it's a bug in the PKGBUILD.

Well, then... what is the point? Versioned dependencies already cover soname values if the package is listed as a dep in addition to is library.

Versioned dependencies don't cover shit. And they only work in one direction. You have bash depending on readline>=6. Now, what happens if readline changes their ABI again (not that they're going to do it soon, but hypothetically). How do you know when readline upgrades that it will be incompatible with bash? You don't know which version number will change the SONAME, the upstream project maintainers are pretty inconsistent about that. So you cannot simply add depends=(readline<7) because it's not known that the SONAME will change there (maybe readline is consistent here, others aren't), it might change earlier or later. The only way of ensuring compatibility is to have each package know which SONAME it provides and which one it needs. Without such a mechanism, it is easy to completely break the system, as witnessed by the many reports after the libjpeg and readline updates. This patch illustrates how this can be done easily and safely. Of course, additional checks have to be added: - Check whether the sodep is among the explicit depends of the package, make the build process fail otherwise (or include such a check in namcap). - Check whether an sodep is among optdepends, discard it in this case. - Maybe more Including those features in the initial patch is not a good idea, as nobody would actually read it then. I suggested to first submit this patch, discuss the approach and then go on to add additional patches to make this feature safe to use with our build system. My suggestion was also to add soprovides to every package, but only add sodepends explicitly in the PKGBUILD on critical packages so pacman won't be too slow in its dependency checks. My 2 cents here to make our packages more robust against failure.

Florian Pritz

4:58 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Allan McRae wrote:

...

...
This shouldn't generate new dependencies, as makepkg already needs to know which packages are needed for building the package. So its a specialization of the already provided dependencies. If it's not, it's a bug in the PKGBUILD.

Well, then... what is the point? Versioned dependencies already cover soname values if the package is listed as a dep in addition to is library.

Allan

http://www.archlinux.org/pipermail/arch-general/2009-August/006873.html The problem was that libarchive-2.7.0-1 needed liblzma.so.0 (which wasn't in the depends array). With this patch sodep-32-liblzma.so.0 would have been in the depends array -> no system breakage possible even if the maintainer doesn't know he forgot a dependency. - -- Florian Pritz -- {flo,bluewind}@server-speed.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIcBAEBAgAGBQJKhulKAAoJEG0WVcFM4cE+Lg0QAJLeQuYfDG15ctgXUFODjArT O3/HVLEnoxydieaG1go5F4k9igEwsAGL3MM7Z0qt/BtSe1gymZGB6ZfHRmtsJhbG +9hq4vO6q6PLnjot4riAXp3oyFhx6fNDcHc6WeLLiFXjm5Lz0ypN79ZU88Hd5q0U Jyw4DXOQKZQzQjWTafZkMg5L8V8e0zY2Zg8d9Ea0MIWgFyCxWiaWjmcrB/MKnanx pZStxL6QePXe85iq29zZNPBP8IV5V8gf98xZZWo71uTBNprtNkvs8il+Yq9cY4dN 90iLHjcoqKpg5yYl3UiEaHGv7xJPRXScr96K/wXty2xG3QzT6wqdHbryZ8i4DMUD U9BH0MIJBhx0mSuvz7u/88uyVHFZEe0ZCmwiPbw42nl3aN67tBUm4hOSpdgb4War elmZnjBrsnies2CCq25+nPr7UJLhF3ORr490T6yyT/5vQeOZ1McrVdGxrQhrTNY5 D5KCZFdx/dJD5CCiFErHVHAFWrSM/JyAra9EaRbeXZyBtnFxOTQKvd+uax8PgQ/q VncoUSlMUcHAYH1ksDWKnuZwkNYzQAY9SLYtcK0UCRDQfFX0bSAnXBvY4hyaky72 fZGZRlfJimUZgLuyRsnyoZRRecG3wMyA7HOV4F2JCMuz9DiJc5D/ySAh6I+HabJG Uc3FWXBQ3u5MwgT4yJOX =0tA5 -----END PGP SIGNATURE-----

Allan McRae

18 Aug 18 Aug

11:06 a.m.

I like to take a step back for a while and think about why a feature is needed in pacman/makepkg (especially from a not Arch perspective as pacman aims to be distro agnostic) So, the issue is sometimes library version mismatches/missingness break a users system. From a pacman/makepkg stand point, how does this occur? In a "stable release" distro, this should never occur. Library differences should only happen between releases which will require a user chaning repos and doing a full -Syu. From experience with Arch, this situation can occur In a rolling release when 1) a user installs a package with "pacman -Sy pkg" and version deps cause other packages to be updated 2) with the use of SyncFirst and non/wrong versioned deps. Case #2 occurred with pacman and was corrected by correcting the versioned dep. Also, it would not have occurred if -Wl,--as-needed was used. Automatic .so deps would have stopped it but so would have correctly versioned deps. So in that situation we already have the tools to deal with this. Case #1 is a case whether this could be useful. But has anyone thought about why this is happening more that it used to? Surely there were lots of people who did a "pacman -Sy pkg" in the past. The answer... excess versioned deps. If we had no versioned deps, only that package specifically being updated would be affected. In general people would not tend to do a "-Sy pkg" with a library and thus the lack of versioned deps only ever caused a minor inconvenience to the user who did the wrong thing. Adding sodeps would just make it difficult for people to do a "pacman -Sy pkg" and those people would end up doing a "-Sd pkg", which would have the same result as no versioned deps. I think a better approach is to have a very few system critical packages providing/depending on the relevant libs and stop over using versioned deps. Allan

Thomas Bächler

11:30 a.m.

Allan McRae schrieb:

...

Case #2 occurred with pacman and was corrected by correcting the versioned dep. Also, it would not have occurred if -Wl,--as-needed was used. Automatic .so deps would have stopped it but so would have correctly versioned deps. So in that situation we already have the tools to deal with this.

This is a rare corner case. Let's ignore it for now.

...

Case #1 is a case whether this could be useful. But has anyone thought about why this is happening more that it used to? Surely there were lots of people who did a "pacman -Sy pkg" in the past. The answer... excess versioned deps. If we had no versioned deps, only that package specifically being updated would be affected. In general people would not tend to do a "-Sy pkg" with a library and thus the lack of versioned deps only ever caused a minor inconvenience to the user who did the wrong thing.

Adding sodeps would just make it difficult for people to do a "pacman -Sy pkg" and those people would end up doing a "-Sd pkg", which would have the same result as no versioned deps.

It would simply fail or request to update tons of other packages. The problem here is, it fails silently. pacman does not realize that by upgrading dependency A for package B, it makes A incompatible with many other packages. There is no warning and there is no way a user can avoid such situations, other than -Syu'ing. Personally, I recommend to always do a full -Syu before messing with your system at all, but I realize that is not always possible.

...

I think a better approach is to have a very few system critical packages providing/depending on the relevant libs and stop over using versioned deps.

If we use fewer versioned deps, we solve a new problem by reverting to an old problem. First of all, I think that adding soname-provides to every package automtically won't hurt anyone (increases package built time slightly) and won't break anything. Even if we never use it, it's nice to have them in pacman -Qi. About versioned dependencies: I am unsure when we should do this: For system-critical packages, definitely, this will simply prevent pacman from breaking those, which is a good thing. For everything else - there's pros and cons, I don't want to complicate our package management too much. One idea is to have a sodepends=(libreadline.so) statement, which would only add a versioned depend on the right libreadline soname, but not on other libraries. Another idea is to implicitly add soname dependencies, but only for libraries owned by packages that are explicitly specified in the depends array (maybe print a warning for the rest). I want to explicitly ask you about the cons for this feature. What do you think is a hard reason NOT to do it?

Allan McRae

12:05 p.m.

Thomas Bächler wrote:

...

I want to explicitly ask you about the cons for this feature. What do you think is a hard reason NOT to do it?

1) The depends array in the PKGBUILD no longer represents the information in the package. 2) Lots more dependencies for packages. This will slow pacman down. Also, I believe that provides are slower than depends to resolve. I could be completely wrong there... but that would make this even worse. So it does "hurt" everyone. 3) I believe this should be implemented manually for a very small number of package sets (shells & readline being the main case), but I do not see a general need for this in a large number of packages. Note that this is not much harder that your option of adding a "sodepends" variable - it just requires adding a versioning on the library. 4) How would you negate this to allow a library to be an optdep other than just not using this feature? I am generally against adding extra syntax to PKGBUILDs unless there is a very strong reason to do so. #1 is my primary critisism. Every other option in makepkg that does something magically (removes files, compresses files, etc), has the files they act on defined specifically in makepkg.conf and thus are not really magical at all. We should be able to look at a PKGBUILD and see the information about a package. I know that all the libraries that are magically depends should have their package included in at least the makedepends, but then how do we tell what really are makedeps? Allan

Thomas Bächler

12:27 p.m.

Allan McRae schrieb:

...

Thomas Bächler wrote:

...
I want to explicitly ask you about the cons for this feature. What do you think is a hard reason NOT to do it?

1) The depends array in the PKGBUILD no longer represents the information in the package.

I want that solved as well. We should only add dependencies for soname if the corresponding packages are already in the depends array. For all others, print a warning or ignore them. Otherwise, (in addition to your concern about the PKGBUILD information being incomplete) optdepends will break.

...

2) Lots more dependencies for packages. This will slow pacman down. Also, I believe that provides are slower than depends to resolve. I could be completely wrong there... but that would make this even worse. So it does "hurt" everyone.

This is a valid point, and we need to investigate how much this will hurt. Still, I would prefer correctness over speed.

...

3) I believe this should be implemented manually for a very small number of package sets (shells & readline being the main case), but I do not see a general need for this in a large number of packages. Note that this is not much harder that your option of adding a "sodepends" variable - it just requires adding a versioning on the library.

This is not an argument, but just an opinion.

...

4) How would you negate this to allow a library to be an optdep other than just not using this feature? I am generally against adding extra syntax to PKGBUILDs unless there is a very strong reason to do so.

See my answer to #1. If something is not in depends, its libraries should be ignored for implicit so dependencies.

...

#1 is my primary critisism. Every other option in makepkg that does something magically (removes files, compresses files, etc), has the files they act on defined specifically in makepkg.conf and thus are not really magical at all. We should be able to look at a PKGBUILD and see the information about a package. I know that all the libraries that are magically depends should have their package included in at least the makedepends, but then how do we tell what really are makedeps?

My answer to #1 solves that issue completely, but it will slow down makepkg a lot.

Florian Pritz

10:05 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thomas Bächler wrote:

...

...
2) Lots more dependencies for packages. This will slow pacman down. Also, I believe that provides are slower than depends to resolve. I could be completely wrong there... but that would make this even worse. So it does "hurt" everyone.

This is a valid point, and we need to investigate how much this will hurt. Still, I would prefer correctness over speed.

I just repackaged 861 packages (everything that was installed and still in my cache) and timed pacman -U * (1.5gb of data) time until dependency check finished: without sodepends = 25 sec with sodepends = 27 sec Won't really hurt anyone. - -- Florian Pritz -- {flo,bluewind}@server-speed.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIcBAEBAgAGBQJKiyWcAAoJEG0WVcFM4cE+4YcQANR1Dvu3zpXkr8mUWhtUe2FC FA/9JMKz1fCfs+9WobWc5aIj2BcW6sEGAl50/N/dGRHb/8JLh5V8lBY4ADFtqb2o rNnl8lf9aMASnfaS7RIgkZrvrr1zkrQMbcUiyB9ub4xct4JtD+XnMyGYWmQaT0Ox Jqv8fN5celuxyEfeRFF/QWSmzD9dqmda03vsdXjv4s96YdKWTD8pORlpqPtEJSOl VkxY3c1MCsTHfX3PFMR054YSTk/cIlQuNxafV1qDAC+Kq1sNCEw9Jwxv00QsvmAx fbVSsODSBEp+VkE8jjSLse/mijXD6odMaWh2x/yCtWdM7ibe8GTC8nHSh0cYJx/q +gx6f7n/VLNvwDX/zL2XYvc1MDOyAQ+tZZ8GBkCIN9/64wIUX0TtAiaFSHqPGgQF 6A5AOGogMR//RBuGAW1tVFXrLnDalHj86Gc1Lwo/CjE7K5zMfzLzLre5FWrR6BJv vxeu6eGktotvUTM3bdjk55NctcGNWPBJ3imobkInrFpoWYPtIXYcxsfkb4ITaPsf 0CQMZZiN8sMh3k0wZmFuRoTrOliUi3b00JF5xuB31lQvwK73h3g9dOcDA8WD2Xfl SML7iOVVZykJjn6ENbDvr6eVCfEMsEdcgFON8bqwwNFkSwp4ckuNc/lOUrJJUGvn KNtb8zTyN2K1ZFEzPsqw =CW5W -----END PGP SIGNATURE-----

Matias Hernandez Arellano

10:31 p.m.

New subject: [pacman-dev] libalpm documentation

Florian Pritz escribió:

...

Thomas Bächler wrote:

...
...
2) Lots more dependencies for packages. This will slow pacman down. Also, I believe that provides are slower than depends to resolve. I could be completely wrong there... but that would make this even worse. So it does "hurt" everyone. This is a valid point, and we need to investigate how much this will hurt. Still, I would prefer correctness over speed.

I just repackaged 861 packages (everything that was installed and still in my cache) and timed pacman -U * (1.5gb of data) time until dependency check finished: without sodepends = 25 sec with sodepends = 27 sec

Won't really hurt anyone.

(First sorry for my english...) Is there any documentation for libalpm.h and libalpm_list.h? -- Matias Hernández (Msdark) www.msdark.archlinux.cl ArchLinux-CL - WikiAdmin MSN: matiasfh@gmail.com Twitter : www.twitter.com/msdark Plurk : www.plurk.com/msdark Linux User: #445065 http://keys.gnupg.net/pks/lookup?op=get&search=0xEA3EEC672189419C

Florian Pritz

28 Aug 28 Aug

3:56 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thomas Bächler wrote:

...

Allan McRae schrieb:

...
Thomas Bächler wrote:

...
I want to explicitly ask you about the cons for this feature. What do you think is a hard reason NOT to do it?

1) The depends array in the PKGBUILD no longer represents the information in the package.

I want that solved as well. We should only add dependencies for soname if the corresponding packages are already in the depends array. For all others, print a warning or ignore them. Otherwise, (in addition to your concern about the PKGBUILD information being incomplete) optdepends will break.

New patch fixes this. Please comment. - -- Florian Pritz -- {flo,bluewind}@server-speed.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIcBAEBAgAGBQJKl/48AAoJEG0WVcFM4cE+WvcQAL55MVvL8szpqwHubgnQsbuu IOhgfsmz2ZiP7jYC9IpuYa4IFd6W2idH0fqwUMAwPrZLM2qE9hDG0l+u2Usoqw0z Akrd0r2JOQKHOC6JtY6fu5j80PHJCZcnWzWJasnuhC+ZVdNznhq5Fqw4HoFLGXB6 EM5NN1HzSoFc6yYk3VgD3Wj2V6O14NTrMC1e3zIO43lSbsJpfgwPYoE/KJfx22NN X5KjERhDpzb3LsY5PKrHgYF521na30JhJPyu5jxBHmVFM5PN5Gh2ebFU210wkg6o 1GkOLfCQQFnSMYXEytg1CZFzdK8jEj44p1oKRHIG8dEzVNPVSjoB3JYqw1DdS/Ed +VaSDbQsWEG5jmkjoZgibVD1BmFsGPr4spP0zz5pPeGsTpkMze7QFka105ZM3rmc 4dPveVtZjXSHVQBtdr4RVYKtTiLBJ0nS4TRAJpMAC2BLo1vZM9hC00srYvn4Z03l 1M6OFn6uS3JhTxJw6fQdFw1rrKlEoQSNq2yCGOrHDJkCyxpOUEZ9V2IS7fZ5JqxT S6LMHnPxKshXXZeP3dewhPHxEb4MbxTofVXlVYn+LfWY4TwbwH+YUe68hHo867/9 yPg1Uw/HmHQHohu1GzhPcHVybhyP1pYVPW4a0ED7k2LHbXFai+4Eh0wKAnGzMwuR A+rU45K+yKs8bfrnPBlP =40Hc -----END PGP SIGNATURE-----

Florian Pritz

15 Aug 15 Aug

4:15 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Fixed problems mentioned so far. - -- Florian Pritz -- {flo,bluewind}@server-speed.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIcBAEBAgAGBQJKht81AAoJEG0WVcFM4cE+7a4P/jsSUrJJ1OjOLC+sRbtf87zI oU7Nb2wyLcvjXvtPdErnXUdxau1vYfgTVvjoRV6ARMqiYqgmsphtHnjgHfNfoWvC fDJpLhCiSa5vm+pSnzcAlZI4BV+xa/0QDEqAWneplbtmNS86KEnkvJPt+1qmtmDh R869YcpNTP/hzFP8PBdkcLGopdOruyy3+ldiEECxzbaI5msN5LTIWMTaRlH9Cfis +03BU2OU84lTapBn3JWXKcNktBAXurqOXEELqEjBZy7+GRZkOKRPPvj2ybfgu8zb bGdF3K6vkC3s3NvoGaX0NG7M3uOSM2NzXYH4NPeoAkFNUJvPvN1bfBefcTmrC9mt WKtZXQSzCiwcXseqvhWpWjwC5U+OyBIm/3DEycPKuq9tf2zMfQ1rStWnYzuuSu3C +A6U5QmMz+fe/NA9leSR4qPRCxHglqK05qS0ZYYGGN6jfLm8PGyEJovue/byMbhS sKMyqlPWhC7beHKi+9nVfmxJVdsgtYCzKSPj6wPWn8sxuVfvN1d6oCfpElM4Ve70 N1IW+XqG+EiYWjArnSG/LKgWuXOlAqsOT9NgyoDF/R5+SlDE3zNDaZPG4M4yYFQ6 OUDDB2v+E6+kiPvFk0bfDYmCispzT+FhPS1BO1xfPJMYbW8I806aQWpw6YFtLrRh 0r0xr22M6dK8KFfxb3JS =Ry89 -----END PGP SIGNATURE-----

5355

Age (days ago)

5368

Last active (days ago)

List overview

Download

19 comments

7 participants

participants (7)

Allan McRae
bluewind＠server-speed.net
Christoph Schied
Florian Pritz
Matias Hernandez Arellano
matthewbruenig＠gmail.com
Thomas Bächler