[arch-dev-public] bash 4.0 / readline 6.0 rebuilds
So bash 4.0 is out, and there's lots of neat little changes, but this also requires readline 6.0, which is a big old nasty soname change. Unless anyone has any objections, I'd like to do these upgrades tonight and put everything in testing if it all goes as plans. Any problems with that? Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
Aaron Griffin wrote:
So bash 4.0 is out, and there's lots of neat little changes, but this also requires readline 6.0, which is a big old nasty soname change.
Unless anyone has any objections, I'd like to do these upgrades tonight and put everything in testing if it all goes as plans.
Any problems with that?
Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
I thought there will be more than that, maybe covered by dependency chains. I have been using the attached script to find needed rebuilds and it works but is slow (takes under two hours to scan my entire cache being all [core] and [extra] plus part of [community]). Allan #! /bin/bash # Run in minimal chroot to avoid false positives due to dependancies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip # Copy script to <chrootdir>/root/tmp and packages to <chrootdir>/root/tmp/pkg # Usage: # ./rebuildlist <library> # <library> does not contain ".so" library=$1 if [ "x$library" == "x" ]; then echo "Usage $0 <library>" exit fi for pkg in $(ls pkg); do echo $pkg mkdir tmp cp pkg/$pkg tmp cd tmp tar -xf $pkg rm $pkg found=$(ldd $(find .) 2>/dev/null | grep "${library}.so" | wc -l) if [ $found -ne 0 ]; then echo $pkg >> ../rebuildlist.txt fi cd .. rm -rf tmp done
On Mon, Feb 23, 2009 at 2:53 PM, Allan McRae <allan@archlinux.org> wrote:
Aaron Griffin wrote:
So bash 4.0 is out, and there's lots of neat little changes, but this also requires readline 6.0, which is a big old nasty soname change.
Unless anyone has any objections, I'd like to do these upgrades tonight and put everything in testing if it all goes as plans.
Any problems with that?
Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
I thought there will be more than that, maybe covered by dependency chains. I have been using the attached script to find needed rebuilds and it works but is slow (takes under two hours to scan my entire cache being all [core] and [extra] plus part of [community]).
Allan
#! /bin/bash
# Run in minimal chroot to avoid false positives due to dependancies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
# Copy script to <chrootdir>/root/tmp and packages to <chrootdir>/root/tmp/pkg
# Usage: # ./rebuildlist <library> # <library> does not contain ".so"
library=$1 if [ "x$library" == "x" ]; then echo "Usage $0 <library>" exit fi
for pkg in $(ls pkg); do echo $pkg mkdir tmp cp pkg/$pkg tmp cd tmp tar -xf $pkg rm $pkg found=$(ldd $(find .) 2>/dev/null | grep "${library}.so" | wc -l) if [ $found -ne 0 ]; then echo $pkg >> ../rebuildlist.txt fi cd .. rm -rf tmp done
Hmm, is this intensive? Would it be bad to run it on gerolde? I don't have all of the packages cached, and don't have enough space on my x86_64 machine (slicehost slice) to run it from that side
Aaron Griffin wrote:
On Mon, Feb 23, 2009 at 2:53 PM, Allan McRae <allan@archlinux.org> wrote:
Aaron Griffin wrote:
So bash 4.0 is out, and there's lots of neat little changes, but this also requires readline 6.0, which is a big old nasty soname change.
Unless anyone has any objections, I'd like to do these upgrades tonight and put everything in testing if it all goes as plans.
Any problems with that?
Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
I thought there will be more than that, maybe covered by dependency chains. I have been using the attached script to find needed rebuilds and it works but is slow (takes under two hours to scan my entire cache being all [core] and [extra] plus part of [community]).
Allan
#! /bin/bash
# Run in minimal chroot to avoid false positives due to dependancies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
# Copy script to <chrootdir>/root/tmp and packages to <chrootdir>/root/tmp/pkg
# Usage: # ./rebuildlist <library> # <library> does not contain ".so"
library=$1 if [ "x$library" == "x" ]; then echo "Usage $0 <library>" exit fi
for pkg in $(ls pkg); do echo $pkg mkdir tmp cp pkg/$pkg tmp cd tmp tar -xf $pkg rm $pkg found=$(ldd $(find .) 2>/dev/null | grep "${library}.so" | wc -l) if [ $found -ne 0 ]; then echo $pkg >> ../rebuildlist.txt fi cd .. rm -rf tmp done
Hmm, is this intensive? Would it be bad to run it on gerolde? I don't have all of the packages cached, and don't have enough space on my x86_64 machine (slicehost slice) to run it from that side
Moderately intensive... probably best to not run on gerolde. I can run it once I get into work today (so results in ~4 hours). Allan
Aaron Griffin schrieb:
Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
Add the new wpa_supplicant to that list (wpa_cli now uses readline).
#! /bin/bash
# Run in minimal chroot to avoid false positives due to dependancies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
# Copy script to <chrootdir>/root/tmp and packages to <chrootdir>/root/tmp/pkg
# Usage: # ./rebuildlist <library> # <library> does not contain ".so"
library=$1 if [ "x$library" == "x" ]; then echo "Usage $0 <library>" exit fi
for pkg in $(ls pkg); do echo $pkg mkdir tmp cp pkg/$pkg tmp cd tmp tar -xf $pkg rm $pkg found=$(ldd $(find .) 2>/dev/null | grep "${library}.so" | wc -l) if [ $found -ne 0 ]; then echo $pkg >> ../rebuildlist.txt fi cd .. rm -rf tmp done
Don't use ldd here, as it also covers indirect dependencies. If libA depends on libB and libB depends on libC, then only libB needs updating when libC changes the SOname, as long as libA never calls libC directly. You should parse the output of readelf -d, which only shows direct dependencies and thus allows you to distinguish between direct and indirect dependencies ... at least as far as I understand it.
Thomas Bächler wrote:
Aaron Griffin schrieb:
Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
Add the new wpa_supplicant to that list (wpa_cli now uses readline).
#! /bin/bash
# Run in minimal chroot to avoid false positives due to dependancies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
# Copy script to <chrootdir>/root/tmp and packages to <chrootdir>/root/tmp/pkg
# Usage: # ./rebuildlist <library> # <library> does not contain ".so"
library=$1 if [ "x$library" == "x" ]; then echo "Usage $0 <library>" exit fi
for pkg in $(ls pkg); do echo $pkg mkdir tmp cp pkg/$pkg tmp cd tmp tar -xf $pkg rm $pkg found=$(ldd $(find .) 2>/dev/null | grep "${library}.so" | wc -l) if [ $found -ne 0 ]; then echo $pkg >> ../rebuildlist.txt fi cd .. rm -rf tmp done
Don't use ldd here, as it also covers indirect dependencies. If libA depends on libB and libB depends on libC, then only libB needs updating when libC changes the SOname, as long as libA never calls libC directly.
You should parse the output of readelf -d, which only shows direct dependencies and thus allows you to distinguish between direct and indirect dependencies ... at least as far as I understand it.
That does not matter when done in a very minimal chroot as directed at the top of the script. If libA depends on libB and libB depends on libC, and libB is not on the system, then "ldd libC" does not show libA. Allan
Allan McRae schrieb:
Don't use ldd here, as it also covers indirect dependencies. If libA depends on libB and libB depends on libC, then only libB needs updating when libC changes the SOname, as long as libA never calls libC directly.
You should parse the output of readelf -d, which only shows direct dependencies and thus allows you to distinguish between direct and indirect dependencies ... at least as far as I understand it.
That does not matter when done in a very minimal chroot as directed at the top of the script. If libA depends on libB and libB depends on libC, and libB is not on the system, then "ldd libC" does not show libA.
You're looking at this the wrong way: You want to know whether libA needs a rebuild because of the libC SONAME bump. So, you install libA and all its dependencies (libB and libC). Now ldd libA.so shows that it depends on libC (but the truth is that only libB requires libC, but libA does not), while readelf tells you libA does NOT need libC. Even better, you don't need to install all dependencies! You just need to extract libA and use readelf on it!
Thomas Bächler wrote:
Allan McRae schrieb:
Don't use ldd here, as it also covers indirect dependencies. If libA depends on libB and libB depends on libC, then only libB needs updating when libC changes the SOname, as long as libA never calls libC directly.
You should parse the output of readelf -d, which only shows direct dependencies and thus allows you to distinguish between direct and indirect dependencies ... at least as far as I understand it.
That does not matter when done in a very minimal chroot as directed at the top of the script. If libA depends on libB and libB depends on libC, and libB is not on the system, then "ldd libC" does not show libA.
You're looking at this the wrong way: You want to know whether libA needs a rebuild because of the libC SONAME bump. So, you install libA and all its dependencies (libB and libC). Now ldd libA.so shows that it depends on libC (but the truth is that only libB requires libC, but libA does not), while readelf tells you libA does NOT need libC.
Even better, you don't need to install all dependencies!
This is the point of this comment: # Run in minimal chroot to avoid false positives due to dependencies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip So, in your example, if you are testing if libA needs a rebuild due to libC, you only extract libA in your chroot, not libB. The ldd can not chain its way to libC. So ti ends up doing the same thing as readelf. Allan
Allan McRae schrieb:
This is the point of this comment: # Run in minimal chroot to avoid false positives due to dependencies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
So, in your example, if you are testing if libA needs a rebuild due to libC, you only extract libA in your chroot, not libB. The ldd can not chain its way to libC. So ti ends up doing the same thing as readelf.
Now I understand it. Still, if you use readelf it does not matter what the environment is, you could run it on any system which is not even Arch, or is the wrong architecture or anything.
Thomas Bächler wrote:
Allan McRae schrieb:
This is the point of this comment: # Run in minimal chroot to avoid false positives due to dependencies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
So, in your example, if you are testing if libA needs a rebuild due to libC, you only extract libA in your chroot, not libB. The ldd can not chain its way to libC. So ti ends up doing the same thing as readelf.
Now I understand it.
Still, if you use readelf it does not matter what the environment is, you could run it on any system which is not even Arch, or is the wrong architecture or anything.
Well, if that did not convince me, this does. I just noticed that "readelf --dynamic" appears to be a lot faster than "ldd". On my /usr/bin/*, readelf takes ~0.2sec while ldd takes ~12sec. I will test this out with an actual run of the script tomorrow. Allan
On Mon, Mar 2, 2009 at 2:43 AM, Allan McRae <allan@archlinux.org> wrote:
Thomas Bächler wrote:
Allan McRae schrieb:
This is the point of this comment: # Run in minimal chroot to avoid false positives due to dependencies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
So, in your example, if you are testing if libA needs a rebuild due to libC, you only extract libA in your chroot, not libB. The ldd can not chain its way to libC. So ti ends up doing the same thing as readelf.
Now I understand it.
Still, if you use readelf it does not matter what the environment is, you could run it on any system which is not even Arch, or is the wrong architecture or anything.
Well, if that did not convince me, this does. I just noticed that "readelf --dynamic" appears to be a lot faster than "ldd". On my /usr/bin/*, readelf takes ~0.2sec while ldd takes ~12sec. I will test this out with an actual run of the script tomorrow.
Allan
Status update on the rebuild?
On Fri, Mar 27, 2009 at 6:34 PM, Eric Bélanger <snowmaniscool@gmail.com> wrote:
On Mon, Mar 2, 2009 at 2:43 AM, Allan McRae <allan@archlinux.org> wrote:
Thomas Bächler wrote:
Allan McRae schrieb:
This is the point of this comment: # Run in minimal chroot to avoid false positives due to dependencies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
So, in your example, if you are testing if libA needs a rebuild due to libC, you only extract libA in your chroot, not libB. The ldd can not chain its way to libC. So ti ends up doing the same thing as readelf.
Now I understand it.
Still, if you use readelf it does not matter what the environment is, you could run it on any system which is not even Arch, or is the wrong architecture or anything.
Well, if that did not convince me, this does. I just noticed that "readelf --dynamic" appears to be a lot faster than "ldd". On my /usr/bin/*, readelf takes ~0.2sec while ldd takes ~12sec. I will test this out with an actual run of the script tomorrow.
Allan
Status update on the rebuild?
Well, mkinitcpio fails miserably under bash 4.0, so I definitely don't even want to push it to testing yet - and I think it's actually a bash issue (It errors saying that ;; is not a valid token in a certain case statement... ?). I can put up what I have so far somewhere, but I wanted to look at the mkinitcpio / bash interaction first before making people's systems unbootable (I did it to myself, hah). I've been real busy with work deadlines and pycon (which is neat, by the way) to deal with this now. I hope I'll have some time this weekend to play with it and get it all shaped up - maybe Dusty (who is here at pycon) and I can get the readline/bash rebuilds in shape on Saturday. Dusty, you interested? We can even do a "ArchLinux packaging" open session on it :)
On Fri, Mar 27, 2009 at 8:01 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On Fri, Mar 27, 2009 at 6:34 PM, Eric Bélanger <snowmaniscool@gmail.com> wrote:
On Mon, Mar 2, 2009 at 2:43 AM, Allan McRae <allan@archlinux.org> wrote:
Thomas Bächler wrote:
Allan McRae schrieb:
This is the point of this comment: # Run in minimal chroot to avoid false positives due to dependencies. # Chroot can be built with: # sudo mkarchroot <chrootdir>/root glibc coreutils findutils grep tar gzip
So, in your example, if you are testing if libA needs a rebuild due to libC, you only extract libA in your chroot, not libB. The ldd can not chain its way to libC. So ti ends up doing the same thing as readelf.
Now I understand it.
Still, if you use readelf it does not matter what the environment is, you could run it on any system which is not even Arch, or is the wrong architecture or anything.
Well, if that did not convince me, this does. I just noticed that "readelf --dynamic" appears to be a lot faster than "ldd". On my /usr/bin/*, readelf takes ~0.2sec while ldd takes ~12sec. I will test this out with an actual run of the script tomorrow.
Allan
Status update on the rebuild?
Well, mkinitcpio fails miserably under bash 4.0, so I definitely don't even want to push it to testing yet - and I think it's actually a bash issue (It errors saying that ;; is not a valid token in a certain case statement... ?).
I can put up what I have so far somewhere, but I wanted to look at the mkinitcpio / bash interaction first before making people's systems unbootable (I did it to myself, hah). I've been real busy with work deadlines and pycon (which is neat, by the way) to deal with this now.
I hope I'll have some time this weekend to play with it and get it all shaped up - maybe Dusty (who is here at pycon) and I can get the readline/bash rebuilds in shape on Saturday.
Dusty, you interested? We can even do a "ArchLinux packaging" open session on it :)
No problem. Take your time. I just asked because I wanted to update 2 packages from your list (fvwm-devel and rosegarden) and wanted to have that done at the same time as the rebuild. I'll just update them on Sunday night unless you manage to fix the mkinitcpio issue.
On Mon, Feb 23, 2009 at 1:37 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
So bash 4.0 is out, and there's lots of neat little changes, but this also requires readline 6.0, which is a big old nasty soname change.
Unless anyone has any objections, I'd like to do these upgrades tonight and put everything in testing if it all goes as plans.
Any problems with that?
Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
I was working on a rosegarden (and depends) update. It's ready. I'll just do the changes in trunk. It has to go in testing first, regardless of the rebuild, have I can't really test these things.
On Mon, Feb 23, 2009 at 3:08 PM, Eric Bélanger <snowmaniscool@gmail.com> wrote:
On Mon, Feb 23, 2009 at 1:37 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
So bash 4.0 is out, and there's lots of neat little changes, but this also requires readline 6.0, which is a big old nasty soname change.
Unless anyone has any objections, I'd like to do these upgrades tonight and put everything in testing if it all goes as plans.
Any problems with that?
Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
I was working on a rosegarden (and depends) update. It's ready. I'll just do the changes in trunk. It has to go in testing first, regardless of the rebuild, have I can't really test these things.
If you want, I have readline and bash rebuilt (as that part's a big pain - makepkg needs bash) - I could stick those in a temporary repo if anyone wants to split up these builds. I'm at work now, so can only do a few in the background when I get a chance
Am Mon, 23 Feb 2009 12:37:31 -0600 schrieb Aaron Griffin <aaronmgriffin@gmail.com>:
So bash 4.0 is out, and there's lots of neat little changes, but this also requires readline 6.0, which is a big old nasty soname change.
So far I'm the maintainer. I guess more likely unwanted because it was a long time orphan? If you think it should be in one hand along bash feel free to take it from me. I'm busy with other stuff.
Unless anyone has any objections, I'd like to do these upgrades tonight and put everything in testing if it all goes as plans.
Any problems with that?
Packages affected, according to the web interface: * abook * afterstep * archboot * bash * bc * cdcd * clisp * device-mapper * freeciv * fvwm * fvwm-devel * gftp * gnuchess * gnutls * gphoto2 * gutenprint * hugs98 * inetutils * jack-audio-connection-kit * lftp * libxml2 * lua * maxima * mysql-clients * ntp * pal * php * pilot-link * postgresql-libs * python24 * r * ratpoison * rosegarden * ruby * socat * sqlite3 * sqlite3 (testing) * swi-prolog * tunepimp * uml_utilities * unixodbc * wvstreams
sqlite3 should get some signoffs and could be moved to core first. -Andy
On Mon, Feb 23, 2009 at 3:37 PM, Andreas Radke <a.radke@arcor.de> wrote:
Am Mon, 23 Feb 2009 12:37:31 -0600 schrieb Aaron Griffin <aaronmgriffin@gmail.com>:
So bash 4.0 is out, and there's lots of neat little changes, but this also requires readline 6.0, which is a big old nasty soname change.
So far I'm the maintainer. I guess more likely unwanted because it was a long time orphan? If you think it should be in one hand along bash feel free to take it from me. I'm busy with other stuff.
If you want to do this, you're more then welcome to - I just say bash 4.0 was out, got excited, and noticed they also released a new readline at the same time... and I started doing that and just figured I might as well do the rebuilds too. If you want me to maintain readline, I can, as it's very closely tied to bash. I'll leave it up to you :)
Am Mon, 23 Feb 2009 15:43:20 -0600 schrieb Aaron Griffin <aaronmgriffin@gmail.com>:
If you want me to maintain readline, I can, as it's very closely tied to bash. I'll leave it up to you :)
Just orphaned readline. Go and adopt it for both arches. -Andy
On Mon, Feb 23, 2009 at 3:58 PM, Andreas Radke <a.radke@arcor.de> wrote:
Am Mon, 23 Feb 2009 15:43:20 -0600 schrieb Aaron Griffin <aaronmgriffin@gmail.com>:
If you want me to maintain readline, I can, as it's very closely tied to bash. I'll leave it up to you :)
Just orphaned readline. Go and adopt it for both arches.
Done. Thanks to Gerardo for running Allan's script with a minor speed update, I now have the following list: http://phraktured.net/readline-rebuilds Woohoo.
participants (5)
-
Aaron Griffin
-
Allan McRae
-
Andreas Radke
-
Eric Bélanger
-
Thomas Bächler