[arch-dev-public] Repository rebuilding
As some of us have noticed, there's a new toolchain in current now. One of the new features is GNU hashing for library symbols. This should speed up linking libraries together. To get full advantage of this hashing, the ideal thing would be to have everything built using GNU hashing on your system. I would suggest a full repo rebuild for this. Another reason for a full repo rebuild would be the bugs we get regarding old packages. Last week we had such a package which had been imported in the repositories 3 years ago and needs libstdc++5 to function now. I looked at it and it doesn't even build with the current compiler we have. This would be the ideal opportunity to get rid of packages that don't compile anymore, or to get them fixed. Compared to operation libtool-slay we had over a year ago, this operation is much more work, but we won't have to pass through testing: the rebuilds don't break anything and the packages remain compatible with eachother. What do we think of this idea?
Thursday 09 August 2007, Jan de Groot wrote: | As some of us have noticed, there's a new toolchain in current | now. One of the new features is GNU hashing for library symbols. | This should speed up linking libraries together. To get full | advantage of this hashing, the ideal thing would be to have | everything built using GNU hashing on your system. I would suggest | a full repo rebuild for this. you mean having --hash-style=gnu as standard for any ld? i read it would increase speed up to 50%, sounds great! is it backward compatible? no, right? how do debuggers deal with this? do they already know it? (or we need to update/fix them first?) | Another reason for a full repo rebuild would be the bugs we get | regarding old packages. Last week we had such a package which had | been imported in the repositories 3 years ago and needs libstdc++5 | to function now. I looked at it and it doesn't even build with the | current compiler we have. This would be the ideal opportunity to | get rid of packages that don't compile anymore, or to get them | fixed. ah! more cleanup! if you keep my apostrophe, i'm joining your late-summer-cleanup team *big smile* sounds interesting :D | Compared to operation libtool-slay we had over a year ago, this | operation is much more work, but we won't have to pass through | testing: the rebuilds don't break anything and the packages remain | compatible with eachother. means every pkg containing a lib has to be rebuilt? as it is not breaking anything, i'm for doing it! side-question: can we try to rebuild the world on a server with one of this fancy auto-build-tools we played some time ago and see how fine this works? would mean less uploading and an excellent test-case for the auto-build tool, right? greetings, damir -- .·´¯`·.¸.·´¯`·.¸¸.·´¯`·.¸.·´¯`·.¸.·´¯`·.¸.·´¯`·.¸¸.·´ ° ° ° ° ° ° ><((((º> ° ° ° ° ° <º)))>< <º)))><
On Thu, 2007-08-09 at 21:57 +0200, Damir Perisa wrote:
Thursday 09 August 2007, Jan de Groot wrote: | As some of us have noticed, there's a new toolchain in current | now. One of the new features is GNU hashing for library symbols. | This should speed up linking libraries together. To get full | advantage of this hashing, the ideal thing would be to have | everything built using GNU hashing on your system. I would suggest | a full repo rebuild for this.
you mean having
--hash-style=gnu
as standard for any ld?
i read it would increase speed up to 50%, sounds great! is it backward compatible? no, right? how do debuggers deal with this? do they already know it? (or we need to update/fix them first?)
--hash-style=both is what the new gcc compiles with by default. This is backwards compatible on runtime, though older binutils versions can't link to these libraries at least on amd64 (see various bugreports we got because some maintainers use testing to compile current stuff with)
i'm not sure if makeworld now works with makepkg3.x - Dan/Aaron, how is the state? and can it handle the whole distribution? maybe we should add makedistro to autoincrease the pkgrel and filtering packages that won't compile. well we should have the possibility to rebuild the whole distribution whenever we want. every other distribution does it for their "releases". we should also have the option to make afford of new features and optimisations the toolchain gives us. but i see big traffic problems for our mirrors and non flatrate users coming. that's one downside of a rolling release. Andy
On Thu, Aug 09, 2007 at 10:51:29PM +0200, Andreas Radke wrote:
i'm not sure if makeworld now works with makepkg3.x - Dan/Aaron, how is the state? and can it handle the whole distribution? maybe we should add makedistro to autoincrease the pkgrel and filtering packages that won't compile.
well we should have the possibility to rebuild the whole distribution whenever we want. every other distribution does it for their "releases". we should also have the option to make afford of new features and optimisations the toolchain gives us.
but i see big traffic problems for our mirrors and non flatrate users coming. that's one downside of a rolling release.
Andy
We'd probably want to make it coencide with a cd release. That way people could at least download new base images. Jason
On 8/9/07, Jan de Groot <jan@jgc.homeip.net> wrote:
As some of us have noticed, there's a new toolchain in current now. One of the new features is GNU hashing for library symbols. This should speed up linking libraries together. To get full advantage of this hashing, the ideal thing would be to have everything built using GNU hashing on your system. I would suggest a full repo rebuild for this.
Another reason for a full repo rebuild would be the bugs we get regarding old packages. Last week we had such a package which had been imported in the repositories 3 years ago and needs libstdc++5 to function now. I looked at it and it doesn't even build with the current compiler we have. This would be the ideal opportunity to get rid of packages that don't compile anymore, or to get them fixed.
Compared to operation libtool-slay we had over a year ago, this operation is much more work, but we won't have to pass through testing: the rebuilds don't break anything and the packages remain compatible with eachother.
What do we think of this idea?
I'm for it! You seem to know what you're doing here, and I trust that machine *point* will still boot. Perhaps we should schedule a weekend for this, and just get it all done - I'm willing to provide man power here, and I'm sure other people would too. Yay!
On Thu, 09 Aug 2007 21:16:00 +0200 Jan de Groot <jan@jgc.homeip.net> wrote:
As some of us have noticed, there's a new toolchain in current now. One of the new features is GNU hashing for library symbols. This should speed up linking libraries together. To get full advantage of this hashing, the ideal thing would be to have everything built using GNU hashing on your system. I would suggest a full repo rebuild for this.
Another reason for a full repo rebuild would be the bugs we get regarding old packages.
And to get correct license information pushed out to the actual packages. If we could combine this incentive with making sure all license fields are correct, every package in our repos would quickly have the license info and that would be cool. -- Travis
2007/8/10, Travis Willard <travis@archlinux.org>:
On Thu, 09 Aug 2007 21:16:00 +0200 Jan de Groot <jan@jgc.homeip.net> wrote:
As some of us have noticed, there's a new toolchain in current now. One of the new features is GNU hashing for library symbols. This should speed up linking libraries together. To get full advantage of this hashing, the ideal thing would be to have everything built using GNU hashing on your system. I would suggest a full repo rebuild for this.
Another reason for a full repo rebuild would be the bugs we get regarding old packages.
And to get correct license information pushed out to the actual packages. If we could combine this incentive with making sure all license fields are correct, every package in our repos would quickly have the license info and that would be cool.
And don't forget about md5sums! ;-) There were a lot of them missing last time I remember. -- Roman Kyrylych (Роман Кирилич)
Roman Kyrylych wrote:
2007/8/10, Travis Willard <travis@archlinux.org>:
On Thu, 09 Aug 2007 21:16:00 +0200 Jan de Groot <jan@jgc.homeip.net> wrote:
As some of us have noticed, there's a new toolchain in current now. One of the new features is GNU hashing for library symbols. This should speed up linking libraries together. To get full advantage of this hashing, the ideal thing would be to have everything built using GNU hashing on your system. I would suggest a full repo rebuild for this.
Another reason for a full repo rebuild would be the bugs we get regarding old packages. And to get correct license information pushed out to the actual packages. If we could combine this incentive with making sure all license fields are correct, every package in our repos would quickly have the license info and that would be cool.
And don't forget about md5sums! ;-) There were a lot of them missing last time I remember.
I suggest to start with that stuff when some of us are on Froscon. Meaning we can collaboratively fix things as they break. Plus the devs who are there can dedicate themselves much more as it is their free time anyway. :) I'd be helping with x86_64 mostly though, as I think the team is smaller still. Cheers, -A
Friday 10 August 2007, Alexander Baldeck wrote: | I'd be helping with x86_64 mostly though, as I think the team is | smaller still. as soon as my new laptop arrives, i will most probably changing teams. this old one has a broken screen and the new one is a x61t sxga+. so as soon as i manage to make it work with arch64, i will join x86_64 while still being active for i686 (but less than usual). ... all this only if i do not get a crazy idea to switch away from linux kernel and start using opensolaris or something else on 64bit ;) still waiting from lenovo/ibm for shippment... they said 3-5 weeks but now its already 6 weeks. - D -- .·´¯`·.¸.·´¯`·.¸¸.·´¯`·.¸.·´¯`·.¸.·´¯`·.¸.·´¯`·.¸¸.·´ ° ° ° ° ° ° ><((((º> ° ° ° ° ° <º)))>< <º)))><
On 8/10/07, Damir Perisa <damir.perisa@solnet.ch> wrote:
Friday 10 August 2007, Alexander Baldeck wrote: | I'd be helping with x86_64 mostly though, as I think the team is | smaller still.
as soon as my new laptop arrives, i will most probably changing teams.
"changing teams" ? I never really thought of the guys with x86_64 hardware as another team.... to me it's like saying "the people with wireless hardware are a different team"
Friday 10 August 2007, Aaron Griffin wrote: | On 8/10/07, Damir Perisa <damir.perisa@solnet.ch> wrote: | > Friday 10 August 2007, Alexander Baldeck wrote: | > | I'd be helping with x86_64 mostly though, as I think the team | > | is smaller still. | > | > as soon as my new laptop arrives, i will most probably changing | > teams. | | "changing teams" ? I never really thought of the guys with x86_64 | hardware as another team.... to me it's like saying "the people | with wireless hardware are a different team" they are, aren't they? *smile* teams in the sense of package management. if somebody makes a mistake with a pkg on x86_64, i was up until now not affected... when you are possibly affected, you start becomming involved :) - D -- dcop amarok player nowPlaying: Gotan Project - La Del Ruso
On 10/08/07, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On 8/10/07, Damir Perisa <damir.perisa@solnet.ch> wrote:
Friday 10 August 2007, Alexander Baldeck wrote: | I'd be helping with x86_64 mostly though, as I think the team is | smaller still.
as soon as my new laptop arrives, i will most probably changing teams.
"changing teams" ? I never really thought of the guys with x86_64 hardware as another team.... to me it's like saying "the people with wireless hardware are a different team"
I don't like for the fact that some members of certain teams seem to have the interests of their team first and foremost, with the other team a distant second...
Friday 10 August 2007, Phil Dillon-Thiselton wrote: | I don't like for the fact that some members of certain teams seem | to have the interests of their team first and foremost, with the | other team a distant second... actually... me neither! therefore please let me introduce myself to all you guys i do not know yet so good because of passivity and inactivity. i'm Damir Perisa, package manager for i686 for some years, i try to keep i18n one of archs projects and actually i would rather be dancing tango or being somewhere taking pictures than spending time on computers but sometimes you have to do also the dry work to keep things running :) ... i'm at the moment writing my master thesis in molecular biology what explains my increased passivity over the last year or so... welcome all you new guys! never met you at one of the last meetings, because ....well... i myself was missing, sorry. looking forward to unite the arch-itectures* and teams and increase my time for arch soon (qt4, kde4, qt4-immodule, scientific pkgs cleanup feature enhancement and other things on my roadmap) about uniting things and suchalike: actually i'm quite confused about the state and the processes in x68_64 ... so i have some homework in front of me... :) best regards, Damir * play of words on "arch-enemies" -- dcop amarok player nowPlaying: Gotan Project - La Del Ruso
On 10/08/07, Damir Perisa <damir.perisa@solnet.ch> wrote:
Friday 10 August 2007, Phil Dillon-Thiselton wrote: | I don't like for the fact that some members of certain teams seem | to have the interests of their team first and foremost, with the | other team a distant second...
actually... me neither!
therefore please let me introduce myself to all you guys i do not know yet so good because of passivity and inactivity.
i'm Damir Perisa, package manager for i686 for some years, i try to keep i18n one of archs projects and actually i would rather be dancing tango or being somewhere taking pictures than spending time on computers but sometimes you have to do also the dry work to keep things running :) ... i'm at the moment writing my master thesis in molecular biology what explains my increased passivity over the last year or so... welcome all you new guys! never met you at one of the last meetings, because ....well... i myself was missing, sorry.
Hallo. I was missing as well.
looking forward to unite the arch-itectures* and teams and increase my time for arch soon (qt4, kde4, qt4-immodule, scientific pkgs cleanup feature enhancement and other things on my roadmap)
As you may have seen from the clean up wiki I have a vested interest in some of those science pkgs so keep us informed as to your thinking :-)
about uniting things and suchalike: actually i'm quite confused about the state and the processes in x68_64 ... so i have some homework in front of me... :)
best regards, Damir
* play of words on "arch-enemies"
-- dcop amarok player nowPlaying: Gotan Project - La Del Ruso
_______________________________________________ arch-dev-public mailing list arch-dev-public@archlinux.org http://archlinux.org/mailman/listinfo/arch-dev-public
Friday 10 August 2007, Phil Dillon-Thiselton wrote: | As you may have seen from the clean up wiki I have a vested | interest in some of those science pkgs so keep us informed as to | your thinking | | :-) cool! can you point me to somewhere in the wiki? forgive my laziness... one thought in advance: i have a private repo (on my own computer, for me and colleagues at university) of quite some scientific apps that would fit nicely online. looking forward... actually looking forward to eat something now ;) (sorry, still funny mood) thanx in advance for the right pointers... greetings, damir -- dcop amarok player nowPlaying: LeAnn Rimes - Crazy
On Fri, 2007-08-10 at 12:34 +0200, Alexander Baldeck wrote:
I suggest to start with that stuff when some of us are on Froscon. Meaning we can collaboratively fix things as they break. Plus the devs who are there can dedicate themselves much more as it is their free time anyway. :)
I'd be helping with x86_64 mostly though, as I think the team is smaller still.
Good point, though I wonder if bumping pkgrels is all we will do at froscon. About the smaller team: these days I build for both architectures. Yes, it takes a while to build and upload, but as I use AMD64 on my main machine now, I can compile for both architectures now.
I suggest to start with that stuff when some of us are on Froscon. Meaning we can collaboratively fix things as they break. Plus the devs who are there can dedicate themselves much more as it is their free time anyway. :)
I'd be helping with x86_64 mostly though, as I think the team is smaller still.
Cheers,
-A
crap. i won't travel there to do dumb rebuilds by hand. therefor we have makeworld. doing this by hand is the most stupid thing i've ever heard. as we all know pacman can handle subdezimal numbers ;-) - just script it to add a .1 behind every pkgrel and then run makeworld over it. Andy
On 8/10/07, Andreas Radke <a.radke@arcor.de> wrote:
as we all know pacman can handle subdezimal numbers ;-) - just script it to add a .1 behind every pkgrel and then run makeworld over it.
Why not just increment the pkgrel? I don't understand the significance
On Fri, Aug 10, 2007 at 12:48:12PM -0500, Aaron Griffin wrote:
On 8/10/07, Andreas Radke <a.radke@arcor.de> wrote:
as we all know pacman can handle subdezimal numbers ;-) - just script it to add a .1 behind every pkgrel and then run makeworld over it.
Why not just increment the pkgrel? I don't understand the significance
Me too. Where is the semantic difference between moving to -2 and -1.1 or 1.01? Jürgen
Friday 10 August 2007, Jürgen Hötzel wrote: | Me too. Where is the semantic difference between moving to -2 and | -1.1 or 1.01? speaking about labels, marks (and probably also the seldom occuring apostrophe), why not rebuilding everything by adding .1hash to pkgrel while doing the makeworld? this would be an indication for the rebuilt distro... a hallmark in history... the 8th world wonder. - D -- dcop amarok player nowPlaying: Gotan Project - La Del Ruso
Am Fri, 10 Aug 2007 12:48:12 -0500 schrieb "Aaron Griffin" <aaronmgriffin@gmail.com>:
Why not just increment the pkgrel? I don't understand the significance
we already have packages with subdecimal versions in both architectures. it's just a question of shell scripting what's the easy way to increase either the pkgrel or add ".1" Andy
On 8/10/07, Andreas Radke <a.radke@arcor.de> wrote:
we already have packages with subdecimal versions in both architectures. it's just a question of shell scripting what's the easy way to increase either the pkgrel or add ".1"
it's trivial in both cases. either way, it doesn't seem right to try and optimize a process based on limitations of a language. Still, I never liked decimal pkgrels. It's the "release" number, or number of times ArchLinux has released or rebuilt the package. It doesn't make sense anymore when you say you released a package 0.3 times. What was the rationale behind using decimal pkgrels? Because I really have no idea where it started.
Friday 10 August 2007, Aaron Griffin wrote: | What was the rationale behind using decimal pkgrels? Because I | really have no idea where it started. partial rebuilds: rebuilds only for one arch were increased only by 0.1 not to enforce rebuilding of the pkg for another arch. - D -- dcop amarok player nowPlaying: Gotan Project - La Del Ruso
On Fri, Aug 10, 2007 at 02:17:44PM -0500, Aaron Griffin wrote:
On 8/10/07, Andreas Radke <a.radke@arcor.de> wrote:
we already have packages with subdecimal versions in both architectures. it's just a question of shell scripting what's the easy way to increase either the pkgrel or add ".1"
it's trivial in both cases. either way, it doesn't seem right to try and optimize a process based on limitations of a language.
Still, I never liked decimal pkgrels. It's the "release" number, or number of times ArchLinux has released or rebuilt the package. It doesn't make sense anymore when you say you released a package 0.3 times.
What was the rationale behind using decimal pkgrels? Because I really have no idea where it started.
That's my fault. We started it with testing and tur releases: 1t1 and 1s1. That way if we were testing revisions that would be released in official as -2, the testing package would be -2t1. Someone figured that users would be confused if packages went from -1 to -3. I didn't think so. After that it was i686 and x86_64. The x86_64 people sometimes had to make changes to PKGBUILDs so that they would build on their machines. When i686 releases -3, x86_64 has to modify it so they release -3.1. That way we'd know that the x86_64 -3.1 was actually just a modification of the i686 package to work with x86_64 instead of being a totally different release -4. There have been a number of cases where people have been confused between ppc, x86_64, and i686 thinking that one release was actually equivalent to another release in a different architecture. We were using this to keep track. What would you think if x86_64 had a -4 version of a package that you have as -3 in i686? Would you think that i686 should update to -4? Or would you check the PKGBUILDs of the two of them to see if one or the other needs an update? Jason
Friday 10 August 2007, Jason Chu wrote: | [...] The x86_64 people sometimes had to make changes to [...] are x86_64 people compatible to i686 people? /me wonders... - D ps sorry guys last one for today -- dcop amarok player nowPlaying: Gotan Project - Vuelvo Al Sur
On 8/10/07, Jason Chu <jason@archlinux.org> wrote:
What would you think if x86_64 had a -4 version of a package that you have as -3 in i686? Would you think that i686 should update to -4? Or would you check the PKGBUILDs of the two of them to see if one or the other needs an update?
Well, there's a few things here that make this point either moot, or an edge case: a) most people don't know the versions that another arch uses. It's really that simple. If I `pacman -S foo` and get 1.5-4, it doesn't tell me what version x86_64 has. The front page likewise only shows i686 packages (sadly...). b) if people get confused then they don't understand the significance of the pkgrel, and it's thus our responsibility to make it more clear. Also, Juergen, thanks for the script - I was gonna do it when I got home from work as I knew it was that simple. You just beat me 8)
Friday 10 August 2007, Aaron Griffin wrote: | Still, I never liked decimal pkgrels. It's the "release" number, | or number of times ArchLinux has released or rebuilt the package. | It doesn't make sense anymore when you say you released a package | 0.3 times. at least it is decimals... imagine we would work in multiple dimensions and have pkgrel as a multidimensional matrix :) well.. 0.3 times means that in 3 out of 10 times it is rebuilt LOL sorry... it was a long day and i find it _really_ funny. - D -- dcop amarok player nowPlaying: Gotan Project - La Del Ruso
On Fri, Aug 10, 2007 at 09:39:01PM +0200, Damir Perisa wrote:
Friday 10 August 2007, Aaron Griffin wrote: | Still, I never liked decimal pkgrels. It's the "release" number, | or number of times ArchLinux has released or rebuilt the package. | It doesn't make sense anymore when you say you released a package | 0.3 times.
at least it is decimals... imagine we would work in multiple dimensions and have pkgrel as a multidimensional matrix :)
well.. 0.3 times means that in 3 out of 10 times it is rebuilt LOL
sorry... it was a long day and i find it _really_ funny.
Hmm... most of the discussion on this is list is about "'" and "." ;)) me musing ;)) Jürgen
Friday 10 August 2007, Jürgen Hötzel wrote: | Hmm... most of the discussion on this is list is about "'" and "." | ;)) me musing ;)) LOL!!!!! -- dcop amarok player nowPlaying: STUPID AMAROK CANNOT OUTPUT TO JACK
Am Fri, 10 Aug 2007 14:17:44 -0500 schrieb "Aaron Griffin" <aaronmgriffin@gmail.com>:
What was the rationale behind using decimal pkgrels? Because I really have no idea where it started.
a decimal pkgrel so far shows that a fix or rebuild was only done for one architecture. the one where the pkg is still perfect stays without a decimal pkgrel and saves them a unnecessary build/up-/download. Andy
On Fri, Aug 10, 2007 at 09:10:15PM +0200, Andreas Radke wrote:
Am Fri, 10 Aug 2007 12:48:12 -0500 schrieb "Aaron Griffin" <aaronmgriffin@gmail.com>:
Why not just increment the pkgrel? I don't understand the significance
we already have packages with subdecimal versions in both architectures. it's just a question of shell scripting what's the easy way to increase either the pkgrel or add ".1"
Hmm... i wouldn't call this advanced shell scripting: find . -name PKGBUILD -print0|xargs -0 bumppkgbuild
On Fri, Aug 10, 2007 at 06:48:03PM +0200, Andreas Radke wrote:
as we all know pacman can handle subdezimal numbers ;-) - just script it to add a .1 behind every pkgrel and then run makeworld over it.
Wow. You sound like you need a good KISS. *muuuah* -S
Am Donnerstag, 9. August 2007 21:16:00 schrieb Jan de Groot:
I would suggest a full repo rebuild for this.
Before doing this: Would it be possible to extend our todo list to have a complete/incomplete state for each architecture? -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
participants (12)
-
Aaron Griffin
-
Alexander Baldeck
-
Andreas Radke
-
Damir Perisa
-
Jan de Groot
-
Jason Chu
-
Jürgen Hötzel
-
Phil Dillon-Thiselton
-
Pierre Schmitz
-
Roman Kyrylych
-
Simo Leone
-
Travis Willard