[arch-dev-public] The "Great Python Rebuild of 2010" begins
Hi, I have pushed the new python (3.1) and python2 (2.7) packages to the new [staging] repo so rebuilds can start there. Remember the staging repo should never be used outside a build chroot... If you do, everything python related will break on your system. TUs will not be able to start uploading rebuilds yet as they do not have a staging equivalent, but we will sort something out for them once we get all the db-scripts/devtools changes tested for the main one first. The package pool is going to save us a heap of issues when this gets moved so it is worth the wait. The key things to remember with the rebuild are: - build in clean chroots. Most packages will detect "python2" or "python-2.7" once they see no "python" and use that. Some will need simple patching. - change your dependencies to python2 - check files for #!'s with "/usr/bin/python" or "/usr/bin/env python". If you have those you will need to do a sed to change them to python2. If you come across a package that you can not easily fix, add it to http://wiki.archlinux.org/index.php/DeveloperWiki:Python_Todo_List#Packages_... and I will take a look at it. Have fun, Allan
On Tue, Aug 17, 2010 at 10:46 AM, Allan McRae <allan@archlinux.org> wrote:
Hi,
I have pushed the new python (3.1) and python2 (2.7) packages to the new [staging] repo so rebuilds can start there. Remember the staging repo should never be used outside a build chroot... If you do, everything python related will break on your system.
TUs will not be able to start uploading rebuilds yet as they do not have a staging equivalent, but we will sort something out for them once we get all the db-scripts/devtools changes tested for the main one first. The package pool is going to save us a heap of issues when this gets moved so it is worth the wait.
The key things to remember with the rebuild are:
- build in clean chroots. Most packages will detect "python2" or "python-2.7" once they see no "python" and use that. Some will need simple patching.
- change your dependencies to python2
- check files for #!'s with "/usr/bin/python" or "/usr/bin/env python". If you have those you will need to do a sed to change them to python2.
Perhaps the "universal sed statement" should go on the wiki? Just an initial stab: find $srcdir -name '*.py' -print0 | xargs sed -i -e 's|#!/usr/bin/python#!/usr/bin/python2#' -e 's|#!/usr/bin/env python#!/usr/bin/env python2#' -Dan
On 18/08/10 01:58, Dan McGee wrote:
On Tue, Aug 17, 2010 at 10:46 AM, Allan McRae<allan@archlinux.org> wrote:
Hi,
I have pushed the new python (3.1) and python2 (2.7) packages to the new [staging] repo so rebuilds can start there. Remember the staging repo should never be used outside a build chroot... If you do, everything python related will break on your system.
TUs will not be able to start uploading rebuilds yet as they do not have a staging equivalent, but we will sort something out for them once we get all the db-scripts/devtools changes tested for the main one first. The package pool is going to save us a heap of issues when this gets moved so it is worth the wait.
The key things to remember with the rebuild are:
- build in clean chroots. Most packages will detect "python2" or "python-2.7" once they see no "python" and use that. Some will need simple patching.
- change your dependencies to python2
- check files for #!'s with "/usr/bin/python" or "/usr/bin/env python". If you have those you will need to do a sed to change them to python2.
Perhaps the "universal sed statement" should go on the wiki? Just an initial stab:
find $srcdir -name '*.py' -print0 | xargs sed -i -e 's|#!/usr/bin/python#!/usr/bin/python2#' -e 's|#!/usr/bin/env python#!/usr/bin/env python2#'
I think you got delimiter confusion there! I find that mostly the *.py files will be in one directory so a find is not necessary. Allan
On 08/17/2010 06:46 PM, Allan McRae wrote:
Hi,
I have pushed the new python (3.1) and python2 (2.7) packages to the new [staging] repo so rebuilds can start there. Remember the staging repo should never be used outside a build chroot... If you do, everything python related will break on your system.
TUs will not be able to start uploading rebuilds yet as they do not have a staging equivalent, but we will sort something out for them once we get all the db-scripts/devtools changes tested for the main one first. The package pool is going to save us a heap of issues when this gets moved so it is worth the wait.
The key things to remember with the rebuild are:
- build in clean chroots. Most packages will detect "python2" or "python-2.7" once they see no "python" and use that. Some will need simple patching.
- change your dependencies to python2
- check files for #!'s with "/usr/bin/python" or "/usr/bin/env python". If you have those you will need to do a sed to change them to python2.
If you come across a package that you can not easily fix, add it to http://wiki.archlinux.org/index.php/DeveloperWiki:Python_Todo_List#Packages_... and I will take a look at it.
Have fun, Allan
just a heads up, you actually need to remove python and namcap (for now) from chroot. -- Ionuț
On 18/08/10 01:46, Allan McRae wrote:
Hi,
I have pushed the new python (3.1) and python2 (2.7) packages to the new [staging] repo so rebuilds can start there. Remember the staging repo should never be used outside a build chroot... If you do, everything python related will break on your system.
TUs will not be able to start uploading rebuilds yet as they do not have a staging equivalent, but we will sort something out for them once we get all the db-scripts/devtools changes tested for the main one first. The package pool is going to save us a heap of issues when this gets moved so it is worth the wait.
The key things to remember with the rebuild are:
- build in clean chroots. Most packages will detect "python2" or "python-2.7" once they see no "python" and use that. Some will need simple patching.
- change your dependencies to python2
- check files for #!'s with "/usr/bin/python" or "/usr/bin/env python". If you have those you will need to do a sed to change them to python2.
If you come across a package that you can not easily fix, add it to http://wiki.archlinux.org/index.php/DeveloperWiki:Python_Todo_List#Packages_... and I will take a look at it.
Just a reminder for people to look at their packages and rebuild them for this. We seem to of stalled majorly... Allan
On Mon, 30 Aug 2010 00:48:06 +1000, Allan McRae <allan@archlinux.org> wrote:
Just a reminder for people to look at their packages and rebuild them for this. We seem to of stalled majorly...
Do you think a community-staging would be helpful? I could commit the needed changes to dbscripts and devtools; I'd like to do a new release soon anyway. -- Pierre Schmitz, https://users.archlinux.de/~pierre
On Sun, 29 Aug 2010 18:05:47 +0200, Pierre Schmitz <pierre@archlinux.de> wrote:
On Mon, 30 Aug 2010 00:48:06 +1000, Allan McRae <allan@archlinux.org> wrote:
Just a reminder for people to look at their packages and rebuild them for this. We seem to of stalled majorly...
Do you think a community-staging would be helpful? I could commit the needed changes to dbscripts and devtools; I'd like to do a new release soon anyway.
FYI: support for community-staging was added to devtools 0.9.10 and dbscripts. -- Pierre Schmitz, https://users.archlinux.de/~pierre
On 09/09/10 23:22, Pierre Schmitz wrote:
FYI: support for community-staging was added to devtools 0.9.10 and dbscripts.
Is someone going to do the necessary setup for this on sigurd (create directory) and gerolde (add to rsync)? Anyway, the current status is, 1 package remaining for [core] and 69 remaining for [extra]. There are a few packages flagged as having issues with the transition, so I will spend my time fixing those. It would be great if everyone could do a push to get [extra] finished. It is coming up to a month since this started and I have noticed that people are doing "double updates" to keep [extra] rolling while rebuilds are in [staging] so we are beginning to create more work by having this sitting around wainting to be finished. Allan
On Sat, 11 Sep 2010 01:10:28 +1000, Allan McRae <allan@archlinux.org> wrote:
On 09/09/10 23:22, Pierre Schmitz wrote:
FYI: support for community-staging was added to devtools 0.9.10 and dbscripts.
Is someone going to do the necessary setup for this on sigurd (create directory) and gerolde (add to rsync)?
Obviously not. Who is in charge of sigurd? Loui and Dan? (http://wiki.archlinux.org/index.php/DeveloperWiki:Sigurd_%28TU%29) -- Pierre Schmitz, https://users.archlinux.de/~pierre
On Sat, 11 Sep 2010 17:59:42 +0200 Pierre Schmitz <pierre@archlinux.de> wrote:
On Sat, 11 Sep 2010 01:10:28 +1000, Allan McRae <allan@archlinux.org> wrote:
On 09/09/10 23:22, Pierre Schmitz wrote:
FYI: support for community-staging was added to devtools 0.9.10 and dbscripts.
Is someone going to do the necessary setup for this on sigurd (create directory) and gerolde (add to rsync)?
Obviously not. Who is in charge of sigurd? Loui and Dan? (http://wiki.archlinux.org/index.php/DeveloperWiki:Sigurd_%28TU%29)
[root@sigurd ~]# grep wheel /etc/group wheel:x:10:root,aaron,dan,dieter,loui,tbaechler I could help, though I don't know what you're talking about :) Dieter
On Sat, 11 Sep 2010 18:13:21 +0200, Dieter Plaetinck <dieter@plaetinck.be> wrote:
On Sat, 11 Sep 2010 17:59:42 +0200 Pierre Schmitz <pierre@archlinux.de> wrote:
On Sat, 11 Sep 2010 01:10:28 +1000, Allan McRae <allan@archlinux.org> wrote:
On 09/09/10 23:22, Pierre Schmitz wrote:
FYI: support for community-staging was added to devtools 0.9.10 and dbscripts.
Is someone going to do the necessary setup for this on sigurd (create directory) and gerolde (add to rsync)?
Obviously not. Who is in charge of sigurd? Loui and Dan? (http://wiki.archlinux.org/index.php/DeveloperWiki:Sigurd_%28TU%29)
[root@sigurd ~]# grep wheel /etc/group wheel:x:10:root,aaron,dan,dieter,loui,tbaechler
I could help, though I don't know what you're talking about :)
Dieter
This should do it: install -d -m775 -o root -g tusers /srv/ftp/community-staging/os/{i686,x86_64} -- Pierre Schmitz, https://users.archlinux.de/~pierre
On 09/11/2010 07:24 PM, Pierre Schmitz wrote:
On Sat, 11 Sep 2010 18:13:21 +0200, Dieter Plaetinck <dieter@plaetinck.be> wrote:
On Sat, 11 Sep 2010 17:59:42 +0200 Pierre Schmitz<pierre@archlinux.de> wrote:
On Sat, 11 Sep 2010 01:10:28 +1000, Allan McRae<allan@archlinux.org> wrote:
On 09/09/10 23:22, Pierre Schmitz wrote:
FYI: support for community-staging was added to devtools 0.9.10 and dbscripts.
Is someone going to do the necessary setup for this on sigurd (create directory) and gerolde (add to rsync)?
Obviously not. Who is in charge of sigurd? Loui and Dan? (http://wiki.archlinux.org/index.php/DeveloperWiki:Sigurd_%28TU%29)
[root@sigurd ~]# grep wheel /etc/group wheel:x:10:root,aaron,dan,dieter,loui,tbaechler
I could help, though I don't know what you're talking about :)
Dieter
This should do it: install -d -m775 -o root -g tusers /srv/ftp/community-staging/os/{i686,x86_64}
Thomas just did it. the infrastructure is set up -- Ionuț
On 11/09/10 01:10, Allan McRae wrote:
Anyway, the current status is, 1 package remaining for [core] and 69 remaining for [extra]. There are a few packages flagged as having issues with the transition, so I will spend my time fixing those.
We are down to: [core] - 1 package [extra] - 42 packages [community] - 29 packages Total: 72 packages That is great work by the TUs in the past week to do all those rebuilds in [community]. Can everyone please look at their dashboard and check which of their packages need rebuilt. I would like to move this to [testing]/[community-testing] in the next week or so. Allan
Allan McRae <allan@archlinux.org> wrote:
Can everyone please look at their dashboard and check which of their packages need rebuilt. I would like to move this to [testing]/[community-testing] in the next week or so.
Should we make an announcement now on [aur-general] to tell people they should start changing dependencies to python2 and paths to /usr/bin/python2 ? I think most changes can be done in advance, it would make the transition less painful. -- Rémy
Status update: [extra] - openoffice variants need done (Andy?) - koffice (kword) needs done - avahi and vte are wainting on bug fixes so no point rebuilding until they are push to the main repos [community] - 12 packages to go So we are almost there people! I will be moving this rebuild from [staging] to [testing] regardless of the status of the remaining rebuilds. Allan
On 02/10/10 10:20, Allan McRae wrote:
Status update:
[extra] - openoffice variants need done (Andy?) - koffice (kword) needs done - avahi and vte are wainting on bug fixes so no point rebuilding until they are push to the main repos
[community] - 12 packages to go
So we are almost there people! I will be moving this rebuild from [staging] to [testing] regardless of the status of the remaining rebuilds.
um... once GNOME moves from [testing]...
Allan McRae <allan@archlinux.org> wrote:
Just a reminder for people to look at their packages and rebuild them for this. We seem to of stalled majorly...
I have a small fix to push to SVN (install script for twisted still uses old paths). Should I push it myself or send it to someone?
On 30/08/10 02:08, Rémy Oudompheng wrote:
I have a small fix to push to SVN (install script for twisted still uses old paths). Should I push it myself or send it to someone?
Feel free to push it yourself. Allan
I have a bunch of packages waiting for [staging] on gerolde: can someone try them and upload them if they're OK? Bittorrent and asymptote seem to works correctly,
2010/8/17 Allan McRae <allan@archlinux.org>:
If you come across a package that you can not easily fix, add it to http://wiki.archlinux.org/index.php/DeveloperWiki:Python_Todo_List#Packages_... and I will take a look at it.
I don't seem to have permissions to edit the wikipage. My account is "Remyoudompheng". There are problems with bazaar, as explained in https://bugs.launchpad.net/bzr/+bug/612096 bazaar is absolutely unusable as is (I mean the bug arises for every action). -- Rémy
Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
I don't seem to have permissions to edit the wikipage. My account is "Remyoudompheng".
There are problems with bazaar, as explained in https://bugs.launchpad.net/bzr/+bug/612096 bazaar is absolutely unusable as is (I mean the bug arises for every action).
Another status report: namcap, python-vpython, mercurial seem to work, gvim and subversion build, but I don't know how to test them. I've uploaded them to my directory. python-numeric seems to be broken, I don't know if it is maintained despite being deprecated. I will probably be able to use [staging] daily soon. -- Rémy.
On 31 August 2010 14:43, Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
I don't seem to have permissions to edit the wikipage. My account is "Remyoudompheng".
There are problems with bazaar, as explained in https://bugs.launchpad.net/bzr/+bug/612096 bazaar is absolutely unusable as is (I mean the bug arises for every action).
Another status report: namcap, python-vpython, mercurial seem to work, gvim and subversion build, but I don't know how to test them. I've uploaded them to my directory. python-numeric seems to be broken, I don't know if it is maintained despite being deprecated. I will probably be able to use [staging] daily soon.
Just a simple run with each provided executable should do. There is no need for any thorough testing, because one person can only do so much. I believe problems should be reported by users when using [testing]. As [staging] should only be used in a chroot, you can either: 1) install python[2] with makechrootpkg, install the program(s) with_out_ the -c option and chroot into the chroot 2) create a usable chroot with everything needed for building, chroot into it and makepkg/pacman as needed You have to set up proper binds and copy relevant files. You can also choose to just break your local system/python stuff and pacman -U python[2] as and when needed. -- GPG/PGP ID: B42DDCAD
Allan McRae <allan@archlinux.org> wrote:
- build in clean chroots. Most packages will detect "python2" or "python-2.7" once they see no "python" and use that. Some will need simple patching.
CMake has builtin detection functions for python with only work for versions up to 2.6. Some patching or update should be done there. -- Rémy
Allan McRae <allan@archlinux.org> wrote:
TUs will not be able to start uploading rebuilds yet as they do not have a staging equivalent, but we will sort something out for them once we get all the db-scripts/devtools changes tested for the main one first. The package pool is going to save us a heap of issues when this gets moved so it is worth the wait.
The current python package already contains a /usr/bin/python2 link. This means that packages which only need a patch to point to python2 instead of python can be rebuilt directly without the need for a [staging] package. For example, glib2 (where python is only used for a script), or youtube-dl (which *is* a script). Similarly, a number of packages which are already in [staging] could probably be moved to [extra] or [testing] without problems: AFAIK, problems would only occur when linking libpython2.7.so or putting files in /usr/lib/python2.7. Should we warn TUs and people on AUR about this fact? It could make the transition smoother. I have generated the attached quick-and-dirty list of safe packages from the wiki. There could be false positives, of course. -- Rémy
Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
Similarly, a number of packages which are already in [staging] could probably be moved to [extra] or [testing] without problems
This would require adding provides=('python2') to the current python 2.6 package. Can it be done safely ?
On 09/03/2010 06:01 PM, Rémy Oudompheng wrote:
Rémy Oudompheng<remyoudompheng@gmail.com> wrote:
Similarly, a number of packages which are already in [staging] could probably be moved to [extra] or [testing] without problems
This would require adding provides=('python2') to the current python 2.6 package. Can it be done safely ?
lets don't rush things and made harder the move of packaging. we have a list, we stick on it, same list can be used when doing the move to extra after staging became more stable -- Ionuț
On 04/09/10 01:01, Rémy Oudompheng wrote:
Rémy Oudompheng<remyoudompheng@gmail.com> wrote:
Similarly, a number of packages which are already in [staging] could probably be moved to [extra] or [testing] without problems
This would require adding provides=('python2') to the current python 2.6 package. Can it be done safely ?
Check the current provides! :P Allan
participants (8)
-
Allan McRae
-
Dan McGee
-
Dieter Plaetinck
-
Ionuț Bîru
-
Pierre Schmitz
-
Ray Rashif
-
Rémy Oudompheng
-
Rémy Oudompheng