Re: [arch-dev-public] [arch-dev] Packaging inconsistencies of python modules
Public list, no need to be private.
On Tue, Apr 5, 2011 at 6:54 AM, Ray Rashif
When I got ready to split python-scipy today (sorry angvp it appears to have taken me more than just "a day" to get to it), I realised I didn't know - or rather was confused about - how/what to name the python3 package. That is a direct result of not being confident about the naming scheme. If there ever has been any discussion with a conclusion, I'd like to be reminded of it. Otherwise..
I remember very early on (during the python transition), the general consensus was:
python2-foo (for any 2.x project or module) foo (for the more popular and bigger 3.x modules which are independent projects by their own like PyQt) python-foo (for the average 3.x module)
So for eg. PyQt we would or were supposed to have (pardon me if I'm wrong):
* python2-pyqt * pyqt
And for NumPy:
* python2-numpy * numpy
Or failing that, at least:
* python2-numpy * python-numpy
However, PyQt is now:
* python2-qt * python-qt
A search for pyqt brings up only 'pyqt3' and 'eric'. I've added the term 'PyQt' to the desc to work around this in trunk.
Moving on, NumPy is:
* python2-numpy (which provides python-numpy..!?) * python3-numpy
I'd like some clarification as to whether all these were intentional.
If not, either way, I'd like to propose the topmost examples as a convention. I understand that it is tedious to go back and forth with the renaming and replacements, but if you all agree, I will help with as many of these packages as I can.
This is great, other than the fact that it totally screws up your upgrade paths and it is all too easy to leave a system with the wrong modules installed (3 vs 2). I wish I had spoken up sooner way back when we did the migration but I wasn't thinking at all and this didn't come to mind. As Stéphane said, we have a problem here in those periods where a package becomes python 3 compatible, is split, but all packages depending on it are not rebuilt. Even this would not satisfy all our issues as many packages optdepend on these modules, and/or are just installed for someone's script X on their system that we aren't tracking deps for in the first place. These two reasons make the rename/rebuild idea not fully work, in my opinion. I'm facing this exact situation with python-pip and python-virtualenv right now- both just became python 3 compatible, and as soon as I rename them I am going to break a lot of people's stuff. It would be fine if this all happened when we did the "great rebuild" as there would only be one time of breakage, but the situation right now is untenable- *every single time I see a python module being updated* I have to be scared as I can't tell whether the one being installed is 2 or 3, and whether I'm suddenly going to lose database connectivity or something. -Dan P.S. Pacman doesn't help here at the moment- https://bugs.archlinux.org/task/23410
At first I didn't want to bring this up openly, as a precaution, but I
suppose it's alright.
On 5 April 2011 21:37, Dan McGee
As Stéphane said, we have a problem here in those periods where a package becomes python 3 compatible, is split, but all packages depending on it are not rebuilt. Even this would not satisfy all our issues as many packages optdepend on these modules, and/or are just installed for someone's script X on their system that we aren't tracking deps for in the first place. These two reasons make the rename/rebuild idea not fully work, in my opinion.
Yes, true. We're doing this now: python-foo -> python2-foo: conflicts=('python-foo<$newver') python-foo *new*: provides=('python3-foo') python3-foo -> python-foo: provides=('python3-foo') conflicts=('python3-foo') replaces=('python3-foo') If things depend on the old 'python-foo', we rebuild them. If things like make/opt depends are missed, people file bug reports, and we ammend as appropriate. Two key points: (1) The breakage comes only when doing an upgrade, and the new 'python-foo' gets into the system not because of any renaming or replacement, but because $currentver < $newver. (2) Thus, the uninformed user (which is basically all of us at such a point in time) gets an unwanted "update"; a python2 module gets replaced by a python3 one. However, this is "solved" by (re)installing the renamed package. Optionally, by this convention, all such updates could warrant an announcement each. Alternatively, prepare a group of such updates, and post an announcement to inform users to upgrade the system with their important packages specified in the list: pacman -Syu python2-importantfoo python2-importantbar Granted, that is very troublesome. Someone (or many) will not be prepared. But it's the only way. If we now begin to introduce python3-* stuff to work around the problem, it all really becomes a mess. We can, of course, initiate another move to have 'python3' instead of 'python' (I don't think this one would be difficult since we already do provision there). Or, well, following NumPy, leave 'python' as-is and make it a convention for modules to use a 'python3-' prefix, even though it wouldn't look consistent with the python package. I have no doubt people will start asking why we don't just rename 'python' to 'python3', then. -- GPG/PGP ID: 8AADBB10
I think the correct approach is the one that has been started: python2-foo -> python-2.x package python-foo -> python-3.x package I am against using python3-foo instead of python-foo... We just need to bite the bullet and get this entirely fixed in our repos. I did not encourage getting this done during the initial python change over as there was enough things to do, but I did not realise that it would becoming a mess now... Do I hear a volunteer to start creating the TODO list? While we are at it, we should make sure all python-3.x packages use the "#!/usr/bin/python3" or "#!/usr/bin/env python3" in their shebangs (similar to what we do for python-2.x packages). That is the upstream recommended way of packaging. Allan
On Wed, 2011-04-06 at 08:20 +1000, Allan McRae wrote:
I think the correct approach is the one that has been started:
python2-foo -> python-2.x package python-foo -> python-3.x package
I am against using python3-foo instead of python-foo...
We just need to bite the bullet and get this entirely fixed in our repos.
So naming scheme is more important than smooth upgrade paths? You can fix everything in the repos, but you can't make a smooth upgrade path without leaving lots of unused python3 libraries on systems where python2-depending apps are installed. With numpy, I chose the easy and smooth way, and that's the way of adding provides and using python2 and python3 naming. As a sidenote, I think it will be very funny to see python4 getting released in the future. Then we'll have to rename all python packages to python3-* and name the new ones python-* again.
On 07/04/11 02:17, Jan de Groot wrote:
On Wed, 2011-04-06 at 08:20 +1000, Allan McRae wrote:
I think the correct approach is the one that has been started:
python2-foo -> python-2.x package python-foo -> python-3.x package
I am against using python3-foo instead of python-foo...
We just need to bite the bullet and get this entirely fixed in our repos.
So naming scheme is more important than smooth upgrade paths? You can fix everything in the repos, but you can't make a smooth upgrade path without leaving lots of unused python3 libraries on systems where python2-depending apps are installed. With numpy, I chose the easy and smooth way, and that's the way of adding provides and using python2 and python3 naming.
Consistent naming is more important to me. That way it is easy to find the package you want. At the moment "pacman -S python-numpy" installs a python2 version, which is inconsistent with (almost?) every other python module providing python-2.x and python-3.x versions. If this is all done in one go, then we can do a news announcement ad advise that "pacman -Qqtd" will show the unnecessary packages.
As a sidenote, I think it will be very funny to see python4 getting released in the future. Then we'll have to rename all python packages to python3-* and name the new ones python-* again.
When discussing the policy for a /usr/bin/python2 symlink, there were some comments from the main python developers that indicate this will not be an issue for many, many, many years... But anyway, if python-4 does eventuate, we will know well in advance and can transition package names properly using provides/replaces which would then be removed on python-4 release. That ship sailed long ago for python-3.x. Allan
participants (4)
-
Allan McRae
-
Dan McGee
-
Jan de Groot
-
Ray Rashif