Hi all,
I want to propose adding all active Python versions to [community], not just the latest one. This would only entail adding the interpreter itself, no other packages.
Having access to interpreters for older active versions is really helpful for Python developers. This allows them to easily run test suites against older versions. It is very common for developers to maintain software against a couple major releases. Tools like tox or nox are able to automate testing against multiple Python versions, just needing the interpreter.
The current active Python releases are: - 3.9 - 3.8 - 3.7 - 3.6 - 2.7
The list can be found here[1].
So, I propose introducing 2 new packages: - python3.7 - python3.6
And when we update the python package to 3.9: - python3.8
Does anyone have any big issue with this? What are your thoughts?
[1] https://www.python.org/downloads/
Cheers, Filipe Laíns
On 2020-11-21 14:34:24 (+0000), Filipe Laíns via arch-dev-public wrote:
Hi all,
I want to propose adding all active Python versions to [community], not just the latest one. This would only entail adding the interpreter itself, no other packages.
Having access to interpreters for older active versions is really helpful for Python developers. This allows them to easily run test suites against older versions. It is very common for developers to maintain software against a couple major releases. Tools like tox or nox are able to automate testing against multiple Python versions, just needing the interpreter.
The current active Python releases are:
- 3.9
- 3.8
- 3.7
- 3.6
- 2.7
The list can be found here[1].
So, I propose introducing 2 new packages:
- python3.7
- python3.6
And when we update the python package to 3.9:
- python3.8
Does anyone have any big issue with this? What are your thoughts?
I guess a downside is the increase in maintenance burden to provide this and it potentially leading to things not being ported/ fixed (ergo fragmentation) as maintainers might want to rely on several interpreters being around in the future (to build packages against). However, as you are specifically stating you would like to only have the interpreters added, maybe that's fine?
An alternative (in a per-user setup) can be to use pyenv [1]. It works reasonably well with tox etc. and I have used it in the past successfully to develop against several python interpreter versions.
Best, David
On Sat, Nov 21, 2020 at 04:47:27PM +0100, David Runge wrote:
On 2020-11-21 14:34:24 (+0000), Filipe Laíns via arch-dev-public wrote:
Hi all,
I want to propose adding all active Python versions to [community], not just the latest one. This would only entail adding the interpreter itself, no other packages.
An alternative (in a per-user setup) can be to use pyenv [1]. It works reasonably well with tox etc. and I have used it in the past successfully to develop against several python interpreter versions.
I'm personally not very excited for the idea of adding more python interpreters. I'm a bit concerned that we today say "no packages" but in the future we relax a bit and end up with python36-$pkgname, as it's the pragmatic option as opposed to blocking entire rebuilds or package updates.
What is the downsides of utilizing something like pyenv? There are user repositories providing older python interpreters as well if people need it.
On Sat, 2020-11-21 at 16:59 +0100, Morten Linderud via arch-dev-public wrote:
On Sat, Nov 21, 2020 at 04:47:27PM +0100, David Runge wrote:
On 2020-11-21 14:34:24 (+0000), Filipe Laíns via arch-dev-public wrote:
Hi all,
I want to propose adding all active Python versions to [community], not just the latest one. This would only entail adding the interpreter itself, no other packages.
An alternative (in a per-user setup) can be to use pyenv [1]. It works reasonably well with tox etc. and I have used it in the past successfully to develop against several python interpreter versions.
I'm personally not very excited for the idea of adding more python interpreters. I'm a bit concerned that we today say "no packages" but in the future we relax a bit and end up with python36-$pkgname, as it's the pragmatic option as opposed to blocking entire rebuilds or package updates.
We can add a guideline blocking this.
What is the downsides of utilizing something like pyenv? There are user repositories providing older python interpreters as well if people need it.
pyenv forces users to compile the Python interpreter themselves, which can take a long time with --enable-optimizations. None of the user repos available builds with optimizations, or has signed packages AFAIK. Of course they could in the future, but I think having the packages in the repos is much better in terms of security and usability. I run one of the user which provides these packages and I don't see myself fixing any of the issues I pointed out due to technical limitations.
Cheers, Filipe Laíns
On 11/21/20 11:58 AM, Filipe Laíns via arch-dev-public wrote:
pyenv forces users to compile the Python interpreter themselves, which can take a long time with --enable-optimizations. None of the user repos available builds with optimizations, or has signed packages AFAIK. Of course they could in the future, but I think having the packages in the repos is much better in terms of security and usability. I run one of the user which provides these packages and I don't see myself fixing any of the issues I pointed out due to technical limitations.
Incidentally... if you don't see yourself fixing the --enable-optimizations or gpg --detach-sign problem for your own builds, I'm morbidly curious as to the technical limitations and would like to know why you think these technical limitations will vanish once you're uploading them to repos.archlinux.org specifically.
Am Sat, 21 Nov 2020 14:34:24 +0000 schrieb Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org:
Does anyone have any big issue with this? What are your thoughts?
[1] https://www.python.org/downloads/
Cheers, Filipe Laíns
-1
Arch is yours. Whoever needs more and older releases on the system - just do it yourself! In the past we said "use abs and AUR - Arch is the base to make it your own".
I don't want to see users raising bugs that something doesn't work with an older version of python. And I don't want to see these requests pop up every now and then to add even more stuff in different versions. It's sad enough we still have python2 and gtk2 around. To have gcc9 around and other duplicates is not what I want to see growing in Arch. I don't want to see our distribution transformed into another Debian.
-Andy
On Sat, 2020-11-21 at 16:58 +0100, Andreas Radke via arch-dev-public wrote:
Am Sat, 21 Nov 2020 14:34:24 +0000 schrieb Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org:
Does anyone have any big issue with this? What are your thoughts?
[1] https://www.python.org/downloads/
Cheers, Filipe Laíns
-1
Arch is yours. Whoever needs more and older releases on the system - just do it yourself! In the past we said "use abs and AUR - Arch is the base to make it your own".
This argument can be used to deny adding any package to the repos. You need this library, tool, etc.? Just add it yourself.
Why are we packaging software that is used by far less people but we can't package these Python interpreters which are being actively missed by people?
I don't want to see users raising bugs that something doesn't work with an older version of python. And I don't want to see these requests pop up every now and then to add even more stuff in different versions.
We already have multiple versions of Java, Ruby, Javascript, etc. hell, even Python. I don't think having people opening bugs because they are deliberately using an older version of Python is a big problem. It hasn't been for any of the other languages, I don't think it will be here. I think this is an hypothetical that doesn't really materialize into reality.
It's sad enough we still have python2 and gtk2 around. To have gcc9 around and other duplicates is not what I want to see growing in Arch.
What you call sad I call a bad UX. Why do we need to force users to compile active releases of the Python interpreters themselves, which can take a long time if they are building with optimization, or to resort to pre-built repos with much lower security standards than us, when there are people willing to maintain them?
I can't understand how it's sad to help out users by not forcing them to resort the sort of things I mentioned above, as long as we are not struggling to do so. I like helping people, that's why I am a packager, that is the opposite of sad for me, so I really can't understand this.
I don't want to see our distribution transformed into another Debian.
That is not what is happening.
Cheers, Filipe Laíns
On Sat, 21 Nov 2020 16:59:21 +0000 Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org wrote:
On Sat, 2020-11-21 at 16:58 +0100, Andreas Radke via arch-dev-public wrote:
Am Sat, 21 Nov 2020 14:34:24 +0000 schrieb Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org:
Does anyone have any big issue with this? What are your thoughts?
[1] https://www.python.org/downloads/
Cheers, Filipe Laíns
-1
Arch is yours. Whoever needs more and older releases on the system - just do it yourself! In the past we said "use abs and AUR - Arch is the base to make it your own".
This argument can be used to deny adding any package to the repos. You need this library, tool, etc.? Just add it yourself.
Why are we packaging software that is used by far less people but we can't package these Python interpreters which are being actively missed by people?
I don't want to see users raising bugs that something doesn't work with an older version of python. And I don't want to see these requests pop up every now and then to add even more stuff in different versions.
We already have multiple versions of Java, Ruby, Javascript, etc. hell, even Python. I don't think having people opening bugs because they are deliberately using an older version of Python is a big problem. It hasn't been for any of the other languages, I don't think it will be here. I think this is an hypothetical that doesn't really materialize into reality.
It's sad enough we still have python2 and gtk2 around. To have gcc9 around and other duplicates is not what I want to see growing in Arch.
What you call sad I call a bad UX. Why do we need to force users to compile active releases of the Python interpreters themselves, which can take a long time if they are building with optimization, or to resort to pre-built repos with much lower security standards than us, when there are people willing to maintain them?
I can't understand how it's sad to help out users by not forcing them to resort the sort of things I mentioned above, as long as we are not struggling to do so. I like helping people, that's why I am a packager, that is the opposite of sad for me, so I really can't understand this.
It's more concerning to me that you can't understand this argument than anything else so far. Arch keeps old things around in the repos when they're required by other things in the repos. It's a necessary evil, not something to be actively encouraged.
I don't want to see our distribution transformed into another Debian.
That is not what is happening.
Cheers, Filipe Laíns
On Sat, 2020-11-21 at 11:24 -0600, Doug Newgard via arch-dev-public wrote:
On Sat, 21 Nov 2020 16:59:21 +0000 Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org wrote:
On Sat, 2020-11-21 at 16:58 +0100, Andreas Radke via arch-dev- public wrote:
Am Sat, 21 Nov 2020 14:34:24 +0000 schrieb Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org:
Does anyone have any big issue with this? What are your thoughts?
[1] https://www.python.org/downloads/
Cheers, Filipe Laíns
-1
Arch is yours. Whoever needs more and older releases on the system - just do it yourself! In the past we said "use abs and AUR - Arch is the base to make it your own".
This argument can be used to deny adding any package to the repos. You need this library, tool, etc.? Just add it yourself.
Why are we packaging software that is used by far less people but we can't package these Python interpreters which are being actively missed by people?
I don't want to see users raising bugs that something doesn't work with an older version of python. And I don't want to see these requests pop up every now and then to add even more stuff in different versions.
We already have multiple versions of Java, Ruby, Javascript, etc. hell, even Python. I don't think having people opening bugs because they are deliberately using an older version of Python is a big problem. It hasn't been for any of the other languages, I don't think it will be here. I think this is an hypothetical that doesn't really materialize into reality.
It's sad enough we still have python2 and gtk2 around. To have gcc9 around and other duplicates is not what I want to see growing in Arch.
What you call sad I call a bad UX. Why do we need to force users to compile active releases of the Python interpreters themselves, which can take a long time if they are building with optimization, or to resort to pre-built repos with much lower security standards than us, when there are people willing to maintain them?
I can't understand how it's sad to help out users by not forcing them to resort the sort of things I mentioned above, as long as we are not struggling to do so. I like helping people, that's why I am a packager, that is the opposite of sad for me, so I really can't understand this.
It's more concerning to me that you can't understand this argument than anything else so far. Arch keeps old things around in the repos when they're required by other things in the repos. It's a necessary evil, not something to be actively encouraged.
I understand that. I am not asking to put all releases of Python on the repos, only the active ones, which people are using.
I don't want to see our distribution transformed into another Debian.
That is not what is happening.
Cheers, Filipe Laíns
On 21/11/2020 18.48, Filipe Laíns via arch-dev-public wrote:
I understand that. I am not asking to put all releases of Python on the repos, only the active ones, which people are using.
I presume you can back it up with numbers how widely 3.7 and 3.6 are used by Arch users. All I can see is that both have less than 30 votes in AUR. Even if I take into account how irrelevant AUR votes are, I assume the problem is exaggerated.
Why are we packaging software that is used by far less people but we can't package these Python interpreters which are being actively missed by people?
Our approach of "YOLO push random new packages to repos" is something I never agreed with, for the record. Unfortunately it's a kingdom with almost no rules.
BP
On Sat, 2020-11-21 at 19:11 +0100, Bartłomiej Piotrowski via arch-dev-public wrote: On 21/11/2020 18.48, Filipe Laíns via arch-dev-public wrote:
I understand that. I am not asking to put all releases of Python on the repos, only the active ones, which people are using.
I presume you can back it up with numbers how widely 3.7 and 3.6 are used by Arch users. All I can see is that both have less than 30 votes in AUR. Even if I take into account how irrelevant AUR votes are, I assume the problem is exaggerated.
I have some metrics that might be relevant.
The people that are missing these interpreters generally use pyenv to make up for it, which is not an optimal solution, but the most common when the interpreters are missing.
Looking at some package stats for pyenv[1], we see a usage of 3,4%. I think that is a big enough number.
pyenv does have another usage, which is managing custom Python interpreters, which is mainly used in CPython development, but I'd say that only a small percentage of the users use it uniquely for that.
We can also look directly at the python36 and python37 packages reported, which users might be getting from the AUR or user repos.
python37 has 1.18% [2] python36 has 1.63% [3]
The main reason why the packages might be required, as I explained in the original email, is for package testing. It puts every maintainer and most contributors to projects that target Python 3.6 or 3.7 in need of an interpreter. Most open source Python project target at least a few different Python versions.
These metrics are not perfect by any means, but I do think they present relevant enough data to show an existent need.
Why are we packaging software that is used by far less people but we can't package these Python interpreters which are being actively missed by people?
Our approach of "YOLO push random new packages to repos" is something I never agreed with, for the record. Unfortunately it's a kingdom with almost no rules.
BP
[1] https://pkgstats.archlinux.de/api/packages/pyenv [2] https://pkgstats.archlinux.de/api/packages/python36 [3] https://pkgstats.archlinux.de/api/packages/python37
Cheers, Filipe Laíns
Whoops, my email client messed up the formatting :/
Here's the reply without the quotes:
Looking at some package stats for pyenv[1], we see a usage of 3,4%. I think that is a big enough number.
pyenv does have another usage, which is managing custom Python interpreters, which is mainly used in CPython development, but I'd say that only a small percentage of the users use it uniquely for that.
We can also look directly at the python36 and python37 packages reported, which users might be getting from the AUR or user repos.
python37 has 1.18% [2] python36 has 1.63% [3]
The main reason why the packages might be required, as I explained in the original email, is for package testing. It puts every maintainer and most contributors to projects that target Python 3.6 or 3.7 in need of an interpreter. Most open source Python project target at least a few different Python versions.
These metrics are not perfect by any means, but I do think they present relevant enough data to show an existent need.
[1] https://pkgstats.archlinux.de/api/packages/pyenv [2] https://pkgstats.archlinux.de/api/packages/python36 [3] https://pkgstats.archlinux.de/api/packages/python37
Cheers, Filipe Laíns
On 11/21/20 11:59 AM, Filipe Laíns via arch-dev-public wrote:
On Sat, 2020-11-21 at 16:58 +0100, Andreas Radke via arch-dev-public wrote:
Am Sat, 21 Nov 2020 14:34:24 +0000 schrieb Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org:
Does anyone have any big issue with this? What are your thoughts?
[1] https://www.python.org/downloads/
Cheers, Filipe Laíns
-1
Arch is yours. Whoever needs more and older releases on the system - just do it yourself! In the past we said "use abs and AUR - Arch is the base to make it your own".
This argument can be used to deny adding any package to the repos. You need this library, tool, etc.? Just add it yourself.
Why are we packaging software that is used by far less people but we can't package these Python interpreters which are being actively missed by people?
I don't want to see users raising bugs that something doesn't work with an older version of python. And I don't want to see these requests pop up every now and then to add even more stuff in different versions.
We already have multiple versions of Java, Ruby, Javascript, etc. hell,
Your analysis is correct, it is indeed hell. I'm not sure why that is an argument in favor of doing even more of it though.
Now, if you were proposing to get rid of some of this, I could get behind that.
even Python.
I don't know if maybe you've missed the fact that we've spent like a year now aggressively upgrading or removing leaf packages depending on python2 in search of that glorious Promised Future where we can finally remove the python2 package itself?
And we're doing that even for a major incompatible release that most people argue is actually a different language.
I'm not sure how this suddenly became an argument to package a ton of point releases that aren't even incompatible.
I don't think having people opening bugs because they are deliberately using an older version of Python is a big problem. It hasn't been for any of the other languages, I don't think it will be here. I think this is an hypothetical that doesn't really materialize into reality.
You're proposing something far beyond the scope of what we do for other languages, and ignoring that for other languages we only do it if driven to in desperation because another official repository package depends on it.
We don't introduce leaf packages of ancient versions of currently packaged interpreters just for fun. I don't even think anyone else does that either.
It's sad enough we still have python2 and gtk2 around. To have gcc9 around and other duplicates is not what I want to see growing in Arch.
What you call sad I call a bad UX. Why do we need to force users to compile active releases of the Python interpreters themselves, which can take a long time if they are building with optimization, or to resort to pre-built repos with much lower security standards than us, when there are people willing to maintain them?
I can't understand how it's sad to help out users by not forcing them to resort the sort of things I mentioned above, as long as we are not struggling to do so. I like helping people, that's why I am a packager, that is the opposite of sad for me, so I really can't understand this.
I have... no idea what the problem is supposed to be here? I'm desperately trying to understand what the actual point of your proposal is.
You want to package old versions of the python3 interpreter "but not modules", because "people would like to use it for tox".
Old versions of stuff that people need for who knows what weird reason is *the textbook reason* why the AUR exists. How long it takes to compile stuff in the AUR is not our problem, and we don't have a rationale to upload gcc-{4,5,6,7,8,9} because actual users need to use a lot of diverse compilers "to test that it still works on really old compilers".
Likewise, if people want to test support for old versions of python3, they have options which don't involve uploading trash into [community].
- Do what normal people do, and build AUR packages they need - Ask FFY00 to provide signed repos - Use travis CI, which doesn't use distro packages but provides a diverse set of hand-compiled python versions, which is apparently some kind of gold standard for doing regression testing on lots of different python releases ;) - Use pyenv
Apparently none of these are viable options in your mind, and the most Arch-like of them, option #1 "use the AUR" you believe to be problematic because it takes a long time to build with optimizations... even though you yourself said those packages don't enable optimizations and don't take a long time???
(If you're using python-$old exclusively for regression tests in tox, you don't need a super optimized version.)
I don't want to see our distribution transformed into another Debian.
That is not what is happening.
Yeah, even Debian doesn't do this. Travis-ci.com does, though.
I don't see any reason why [community] needs to grow to support the needs of exclusively Continuous Integration testsuite matrixes.
On Sat, 2020-11-21 at 20:24 -0500, Eli Schwartz via arch-dev-public wrote:
Your analysis is correct, it is indeed hell. I'm not sure why that is an argument in favor of doing even more of it though.
Now, if you were proposing to get rid of some of this, I could get behind that.
It was not an analysis, "hell" was an used here as an idiom.
even Python.
I don't know if maybe you've missed the fact that we've spent like a year now aggressively upgrading or removing leaf packages depending on python2 in search of that glorious Promised Future where we can finally remove the python2 package itself?
I did not miss that. I absolutely want to see the python2 modules removed, not _necessarily_ the python2 interpreter.
And we're doing that even for a major incompatible release that most people argue is actually a different language.
I'm not sure how this suddenly became an argument to package a ton of point releases that aren't even incompatible.
Except they are... Python does not follow semver, only patch releases are supposed to be compatible. They do try to keep breakage to a minimum, but it definitely exists. One very obvious example is the introduction of "async" and "await" as reserved keywords. There are a bunch of other backwards incompatible changes, so yes, they are incompatible releases, to the point you could call them different languages.
I don't think having people opening bugs because they are deliberately using an older version of Python is a big problem. It hasn't been for any of the other languages, I don't think it will be here. I think this is an hypothetical that doesn't really materialize into reality.
You're proposing something far beyond the scope of what we do for other languages, and ignoring that for other languages we only do it if driven to in desperation because another official repository package depends on it.
We don't introduce leaf packages of ancient versions of currently packaged interpreters just for fun. I don't even think anyone else does that either.
That was not of the point of me bringing up other languages. The whole point of me bringing up other languages was to show that Andreas' concern is unlikely to materialize into reality.
It's sad enough we still have python2 and gtk2 around. To have gcc9 around and other duplicates is not what I want to see growing in Arch.
What you call sad I call a bad UX. Why do we need to force users to compile active releases of the Python interpreters themselves, which can take a long time if they are building with optimization, or to resort to pre-built repos with much lower security standards than us, when there are people willing to maintain them?
I can't understand how it's sad to help out users by not forcing them to resort the sort of things I mentioned above, as long as we are not struggling to do so. I like helping people, that's why I am a packager, that is the opposite of sad for me, so I really can't understand this.
I have... no idea what the problem is supposed to be here? I'm desperately trying to understand what the actual point of your proposal is.
You want to package old versions of the python3 interpreter "but not modules", because "people would like to use it for tox".
People can use it tox or to create virtual environments directly. The main point of the proposal is that people have use for the old interpreters and I would like to provide them.
Old versions of stuff that people need for who knows what weird reason is *the textbook reason* why the AUR exists. How long it takes to compile stuff in the AUR is not our problem, and we don't have a rationale to upload gcc-{4,5,6,7,8,9} because actual users need to use a lot of diverse compilers "to test that it still works on really old compilers".
AFAIK, the textbook reason why the AUR exists is to provide software that isn't in the official repos. It being older versions should be completely incidental, but not the "textbook reason".
Likewise, if people want to test support for old versions of python3, they have options which don't involve uploading trash into [community].
I like how you refereed to the currently active Python releases as "trash", will keep that in mind.
- Do what normal people do, and build AUR packages they need
- Ask FFY00 to provide signed repos
- Use travis CI, which doesn't use distro packages but provides a
diverse set of hand-compiled python versions, which is apparently some kind of gold standard for doing regression testing on lots of different python releases ;)
- Use pyenv
Apparently none of these are viable options in your mind, and the most Arch-like of them, option #1 "use the AUR" you believe to be problematic because it takes a long time to build with optimizations... even though you yourself said those packages don't enable optimizations and don't take a long time???
(If you're using python-$old exclusively for regression tests in tox, you don't need a super optimized version.)
Tests can take a long time to run, multiply that by the interpreters you need to run them against. But that is only part of the reason.
I don't want to see our distribution transformed into another Debian.
That is not what is happening.
Yeah, even Debian doesn't do this. Travis-ci.com does, though.
I don't see any reason why [community] needs to grow to support the needs of exclusively Continuous Integration testsuite matrixes.
I agree, it doesn't need to grow support for this specific need. What I think it should grow support to is software that is being actively missed by arch users, which in this case are the Python interpreters for active Python releases.
I don't understand what are the drawbacks of this. Having more packages in the repos, I guess?
As long as there is someone willing to maintain the packages, why would we block it? I doesn't really have any effect on the rest of the people. If the person start having trouble maintaining them, they can be dropped. It doesn't affect you and helps other people.
Cheers, Filipe Laíns
On Sun, 22 Nov 2020 15:05:04 +0000 Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org wrote:
As long as there is someone willing to maintain the packages, why would we block it? I doesn't really have any effect on the rest of the people. If the person start having trouble maintaining them, they can be dropped. It doesn't affect you and helps other people.
Cheers, Filipe Laíns
Why did you start the discussion if you plan on ignoring other people's opinions anyway?
Doug
On Sun, 2020-11-22 at 09:13 -0600, Doug Newgard via arch-dev-public wrote:
Why did you start the discussion if you plan on ignoring other people's opinions anyway?
Doug
I am not ignoring Andreas' or Eli's opinion, I understand that they don't want these packages in the repos. What I am asking is why they don't want them there?
I have addressed the concern Andreas raised. I am not sure if satisfactorily to him as he has not replied yet.
Eli, so far, has presented me with the current status quo, stating this is not we currently do. Which was never what I was arguing, that's why I have started this thread, in hope of changing that. From his reply, the only part I see as actual reasoning against the proposed change is this:
I don't see any reason why [community] needs to grow to support the
needs of exclusively Continuous Integration testsuite matrixes.
Which I have addressed. Please let me know if I've missed any of his points.
Cheers, Filipe Laíns
On Sun, 22 Nov 2020 15:32:22 +0000 Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org wrote:
On Sun, 2020-11-22 at 09:13 -0600, Doug Newgard via arch-dev-public wrote:
Why did you start the discussion if you plan on ignoring other people's opinions anyway?
Doug
I am not ignoring Andreas' or Eli's opinion, I understand that they don't want these packages in the repos. What I am asking is why they don't want them there?
I don't appreciate you trimming the paragraph I was replying to. I makes my comments look completely different without context.
The paragraph specifically said that you see no reason to block anything as long as someone is willing to maintain it. That means that other's opinions in this disussion don't matter, unless I misunderstood?
Doug
On Sun, 2020-11-22 at 09:43 -0600, Doug Newgard via arch-dev-public wrote:
On Sun, 22 Nov 2020 15:32:22 +0000 Filipe Laíns via arch-dev-public arch-dev-public@archlinux.org wrote:
On Sun, 2020-11-22 at 09:13 -0600, Doug Newgard via arch-dev-public wrote:
Why did you start the discussion if you plan on ignoring other people's opinions anyway?
Doug
I am not ignoring Andreas' or Eli's opinion, I understand that they don't want these packages in the repos. What I am asking is why they don't want them there?
I don't appreciate you trimming the paragraph I was replying to. I makes my comments look completely different without context.
Sorry, I only replied to the specific reply because my email client is messing up the format and I have not fixed it yet. I did not meant to take it out of context.
The paragraph specifically said that you see no reason to block anything as long as someone is willing to maintain it. That means that other's opinions in this disussion don't matter, unless I misunderstood?
Doug
That was not meant that way. Similarly to your reply, the context matters.
The comment was meant as "If this helps people and does not impact you, why would you block it?", I said this because Eli seems hell bent on not having the packages in the repos. Sorry if it came out the wrong way.
Cheers, Filipe Laíns
On 11/22/20 10:05 AM, Filipe Laíns wrote:
On Sat, 2020-11-21 at 20:24 -0500, Eli Schwartz via arch-dev-public wrote:
Your analysis is correct, it is indeed hell. I'm not sure why that is an argument in favor of doing even more of it though.
Now, if you were proposing to get rid of some of this, I could get behind that.
It was not an analysis, "hell" was an used here as an idiom.
You were unintentionally accurate.
even Python.
I don't know if maybe you've missed the fact that we've spent like a year now aggressively upgrading or removing leaf packages depending on python2 in search of that glorious Promised Future where we can finally remove the python2 package itself?
I did not miss that. I absolutely want to see the python2 modules removed, not _necessarily_ the python2 interpreter.
Now I'm starting to see part of the reason why people are having a difference of opinions here.
I think I probably speak for a bunch of people when I say I'd like to see the interpreter dropped due to not being needed by any packages *and* being an end of life interpreter. The interpreter is only useful inasmuch as people use it to run software, so if we don't have any...
I expect by the time we finally remove all reverse dependencies for it, the gap between the python2 EOL and its removal from the distro will be quite a bit biger than it currently is.
And we're doing that even for a major incompatible release that most people argue is actually a different language.
I'm not sure how this suddenly became an argument to package a ton of point releases that aren't even incompatible.
Except they are... Python does not follow semver, only patch releases are supposed to be compatible. They do try to keep breakage to a minimum, but it definitely exists. One very obvious example is the introduction of "async" and "await" as reserved keywords. There are a bunch of other backwards incompatible changes, so yes, they are incompatible releases, to the point you could call them different languages.
This is the first time I ever heard anyone describe "backwards incompatible changes means it is now a different programming language".
I had backwards incompatible changes in a patchlevel bugfix in the python 3.8.x release line causing immediate traceback and SystemExit on a program's startup, does that mean every bugfix is now a different programming language?
(I mean this quite literally. Python bugfixed a function not working as expected under certain situations. The program it broke was working around the failure by manually doing $thing, the traceback happened when you try to do $thing twice.)
I don't think having people opening bugs because they are deliberately using an older version of Python is a big problem. It hasn't been for any of the other languages, I don't think it will be here. I think this is an hypothetical that doesn't really materialize into reality.
You're proposing something far beyond the scope of what we do for other languages, and ignoring that for other languages we only do it if driven to in desperation because another official repository package depends on it.
We don't introduce leaf packages of ancient versions of currently packaged interpreters just for fun. I don't even think anyone else does that either.
That was not of the point of me bringing up other languages. The whole point of me bringing up other languages was to show that Andreas' concern is unlikely to materialize into reality.
You think no one reports bugs for python-* packages that we should provide the python2 version? I promise you, you're wrong. I've closed those bugs, so I know they exist.
You specifically brought up Java and Javascript, languages where each application is forced to vendor their entire dependency list rather than using system modules. For this reason, people don't report the kind of bugs Andreas is concerned about for Java/nodejs...
So I assumed you could not possibly be talking about Andreas' concern, and I responded somewhat tangentially myself. We can, of course, talk about Andreas' concern, but given I've now submitted the authoritative facts on the matter, I'm not sure what else there is to say.
It does indisputably happen right now. The logical inference is, it will happen *more*, if we add more runtimes for people to submit bugtracker tickets for.
You want to package old versions of the python3 interpreter "but not modules", because "people would like to use it for tox".
People can use it tox or to create virtual environments directly. The main point of the proposal is that people have use for the old interpreters and I would like to provide them.
The main point of my objection is that their use isn't to run software, it's to regression test it -- something more suited to Continuous Integration which already targets a wide range of operating systems *and* interpreter versions, and which can be used to block merging.
Old versions of stuff that people need for who knows what weird reason is *the textbook reason* why the AUR exists. How long it takes to compile stuff in the AUR is not our problem, and we don't have a rationale to upload gcc-{4,5,6,7,8,9} because actual users need to use a lot of diverse compilers "to test that it still works on really old compilers".
AFAIK, the textbook reason why the AUR exists is to provide software that isn't in the official repos. It being older versions should be completely incidental, but not the "textbook reason".
The AUR exists to provide software which is not in the official repos for practical OR ideological reasons.
Old stuff is ideological reasons.
Likewise, if people want to test support for old versions of python3, they have options which don't involve uploading trash into [community].
I like how you refereed to the currently active Python releases as "trash", will keep that in mind.
I like how you reinterpret my reference to Python releases which are in LTS "only security fixes are permitted and even then, we won't release Windows or macOS binaries, just source code for you to compile yourself" mode as "currently active".
They are not active. Active would imply that on https://devguide.python.org/#branchstatus they are marked as "bugfix" or "feature", indicating the Python developers will actually fix bugs in them and release multi-platform installers in addition to source code.
So python >= 3.8 I would call "not trash", but you want to upload python 3.6 and python 3.7, which I would call trash at this point.
They are expired versions. They went moldy, and the only people who should be using them are people who have been storing their entire computing environment in the freezer in order to eke out another year or two of emergency use.
- Do what normal people do, and build AUR packages they need
- Ask FFY00 to provide signed repos
- Use travis CI, which doesn't use distro packages but provides a
diverse set of hand-compiled python versions, which is apparently some kind of gold standard for doing regression testing on lots of different python releases ;)
- Use pyenv
Apparently none of these are viable options in your mind, and the most Arch-like of them, option #1 "use the AUR" you believe to be problematic because it takes a long time to build with optimizations... even though you yourself said those packages don't enable optimizations and don't take a long time???
(If you're using python-$old exclusively for regression tests in tox, you don't need a super optimized version.)
Tests can take a long time to run, multiply that by the interpreters you need to run them against. But that is only part of the reason.
Why are you nitpicking the offhand comment I made, rather than my actual recommendations?
Again, this is a non-issue if you stick the packages in an unofficial user repository and enable optimizations. And users are welcome to make the tradeoff of less time compiling in exchange for slightly longer test runtimes. Building a package is a one-time cost anyway, which you could do overnight.
As long as there is someone willing to maintain the packages, why would we block it? I doesn't really have any effect on the rest of the people. If the person start having trouble maintaining them, they can be dropped. It doesn't affect you and helps other people.
One reason is, for the same reason we ideologically don't want gcc-{4,5,6,7,8,9} in the repos, no matter how "little effect" it has on other people. We don't wish to promote using old things.
Sometimes we include *one* old version of gcc in desperation, because CUDA cannot be convinced to use current gcc and cannot be fixed because it's closed-source. Even this bothers Andreas (and me, and actually, it bothers Sven too even though he's the one that uploaded gcc9 to begin with) but we put up with it because it's a dependency of something else, and we only package the one old version needed and upgrade it as soon as cuda supports something more modern.
We wouldn't have gcc9 merely "for the public service convenience of anyone who needs it".
On Sun, 2020-11-22 at 11:45 -0500, Eli Schwartz via arch-dev-public wrote:
On 11/22/20 10:05 AM, Filipe Laíns wrote:
On Sat, 2020-11-21 at 20:24 -0500, Eli Schwartz via arch-dev-public wrote:
Your analysis is correct, it is indeed hell. I'm not sure why that is an argument in favor of doing even more of it though.
Now, if you were proposing to get rid of some of this, I could get behind that.
It was not an analysis, "hell" was an used here as an idiom.
You were unintentionally accurate.
even Python.
I don't know if maybe you've missed the fact that we've spent like a year now aggressively upgrading or removing leaf packages depending on python2 in search of that glorious Promised Future where we can finally remove the python2 package itself?
I did not miss that. I absolutely want to see the python2 modules removed, not _necessarily_ the python2 interpreter.
Now I'm starting to see part of the reason why people are having a difference of opinions here.
I think I probably speak for a bunch of people when I say I'd like to see the interpreter dropped due to not being needed by any packages *and* being an end of life interpreter. The interpreter is only useful inasmuch as people use it to run software, so if we don't have any...
I expect by the time we finally remove all reverse dependencies for it, the gap between the python2 EOL and its removal from the distro will be quite a bit biger than it currently is.
And we're doing that even for a major incompatible release that most people argue is actually a different language.
I'm not sure how this suddenly became an argument to package a ton of point releases that aren't even incompatible.
Except they are... Python does not follow semver, only patch releases are supposed to be compatible. They do try to keep breakage to a minimum, but it definitely exists. One very obvious example is the introduction of "async" and "await" as reserved keywords. There are a bunch of other backwards incompatible changes, so yes, they are incompatible releases, to the point you could call them different languages.
This is the first time I ever heard anyone describe "backwards incompatible changes means it is now a different programming language".
You literally mentioned that in your reply. Though I assume you meant more significant change than what is being discussed. If the languages semantics are different one could argue it is actually a different language. Anyway, this does not really matter for the issue being discussed.
I had backwards incompatible changes in a patchlevel bugfix in the python 3.8.x release line causing immediate traceback and SystemExit on a program's startup, does that mean every bugfix is now a different programming language?
(I mean this quite literally. Python bugfixed a function not working as expected under certain situations. The program it broke was working around the failure by manually doing $thing, the traceback happened when you try to do $thing twice.)
I don't think having people opening bugs because they are deliberately using an older version of Python is a big problem. It hasn't been for any of the other languages, I don't think it will be here. I think this is an hypothetical that doesn't really materialize into reality.
You're proposing something far beyond the scope of what we do for other languages, and ignoring that for other languages we only do it if driven to in desperation because another official repository package depends on it.
We don't introduce leaf packages of ancient versions of currently packaged interpreters just for fun. I don't even think anyone else does that either.
That was not of the point of me bringing up other languages. The whole point of me bringing up other languages was to show that Andreas' concern is unlikely to materialize into reality.
You think no one reports bugs for python-* packages that we should provide the python2 version? I promise you, you're wrong. I've closed those bugs, so I know they exist.
That is not "users raising bugs that something doesn't work with an older version of python", and is not what I argued against.
You specifically brought up Java and Javascript, languages where each application is forced to vendor their entire dependency list rather than using system modules. For this reason, people don't report the kind of bugs Andreas is concerned about for Java/nodejs...
They are not necessarily forced, but whatever, just ignore that.
So I assumed you could not possibly be talking about Andreas' concern, and I responded somewhat tangentially myself. We can, of course, talk about Andreas' concern, but given I've now submitted the authoritative facts on the matter, I'm not sure what else there is to say.
It does indisputably happen right now. The logical inference is, it will happen *more*, if we add more runtimes for people to submit bugtracker tickets for.
Can you show me an handful of examples? And not about people asking us to provide Python 2 versions of packages, but rather about what Andreas mentioned.
You want to package old versions of the python3 interpreter "but not modules", because "people would like to use it for tox".
People can use it tox or to create virtual environments directly. The main point of the proposal is that people have use for the old interpreters and I would like to provide them.
The main point of my objection is that their use isn't to run software, it's to regression test it -- something more suited to Continuous Integration which already targets a wide range of operating systems *and* interpreter versions, and which can be used to block merging.
There is use, you just keep ignoring it.
Old versions of stuff that people need for who knows what weird reason is *the textbook reason* why the AUR exists. How long it takes to compile stuff in the AUR is not our problem, and we don't have a rationale to upload gcc-{4,5,6,7,8,9} because actual users need to use a lot of diverse compilers "to test that it still works on really old compilers".
AFAIK, the textbook reason why the AUR exists is to provide software that isn't in the official repos. It being older versions should be completely incidental, but not the "textbook reason".
The AUR exists to provide software which is not in the official repos for practical OR ideological reasons.
Old stuff is ideological reasons.
Likewise, if people want to test support for old versions of python3, they have options which don't involve uploading trash into [community].
I like how you refereed to the currently active Python releases as "trash", will keep that in mind.
I like how you reinterpret my reference to Python releases which are in LTS "only security fixes are permitted and even then, we won't release Windows or macOS binaries, just source code for you to compile yourself" mode as "currently active".
They are not active. Active would imply that on https://devguide.python.org/#branchstatus%C2%A0they are marked as "bugfix" or "feature", indicating the Python developers will actually fix bugs in them and release multi-platform installers in addition to source code.
So python >= 3.8 I would call "not trash", but you want to upload python 3.6 and python 3.7, which I would call trash at this point.
Once again, nice.
They are expired versions. They went moldy, and the only people who should be using them are people who have been storing their entire computing environment in the freezer in order to eke out another year or two of emergency use.
I really wish you would stop making this kind of statements. It's not nice to anyone, actually, the opposite.
- Do what normal people do, and build AUR packages they need
- Ask FFY00 to provide signed repos
- Use travis CI, which doesn't use distro packages but provides a
diverse set of hand-compiled python versions, which is apparently some kind of gold standard for doing regression testing on lots of different python releases ;)
- Use pyenv
Apparently none of these are viable options in your mind, and the most Arch-like of them, option #1 "use the AUR" you believe to be problematic because it takes a long time to build with optimizations... even though you yourself said those packages don't enable optimizations and don't take a long time???
I want to have packages built with optimizations but doing that takes a long time, that's why none of the current options offers that. What is difficult to understand?
(If you're using python-$old exclusively for regression tests in tox, you don't need a super optimized version.)
Tests can take a long time to run, multiply that by the interpreters you need to run them against. But that is only part of the reason.
Why are you nitpicking the offhand comment I made, rather than my actual recommendations?
Because I felt like I had explained that already, I don't want to be reiterating the same point every time. I did it now, won't do it again if you come with the same questions.
Again, this is a non-issue if you stick the packages in an unofficial user repository and enable optimizations. And users are welcome to make the tradeoff of less time compiling in exchange for slightly longer test runtimes. Building a package is a one-time cost anyway, which you could do overnight.
Yeah, none of that provide a nice user experience, which is what I am trying to fix.
As long as there is someone willing to maintain the packages, why would we block it? I doesn't really have any effect on the rest of the people. If the person start having trouble maintaining them, they can be dropped. It doesn't affect you and helps other people.
One reason is, for the same reason we ideologically don't want gcc-{4,5,6,7,8,9} in the repos, no matter how "little effect" it has on other people. We don't wish to promote using old things.
Now, that is a proper argument. I think it is reasonable to deny this based on that, although not very nice for users. Personally, I do not think it is a big deal. I don't think us providing Java 8 or Ruby 2.6 in the repos is actively promoting people to use that software. People need to use a custom package, with custom paths, which I think actively discourages them from using it. What it does is enabling them to use it, which is good for the people who need it, but not for the people who don't but choose to use it anyway, those people are already being discouraged as I point out above.
So yeah, personally I do not think that is significant enough to block us from having these interpreters, which would in fact help people. But others might not agree, so it is up to the team.
Sometimes we include *one* old version of gcc in desperation, because CUDA cannot be convinced to use current gcc and cannot be fixed because it's closed-source. Even this bothers Andreas (and me, and actually, it bothers Sven too even though he's the one that uploaded gcc9 to begin with) but we put up with it because it's a dependency of something else, and we only package the one old version needed and upgrade it as soon as cuda supports something more modern.
I understand that, however I think most of those bothers come from us having to maintain another GCC, not from not wanting to provide it.
We wouldn't have gcc9 merely "for the public service convenience of anyone who needs it".
*You* wouldn't. If someone wants to do it and it doesn't negatively impact anyone else, I am neutral.
Cheers, Filipe Laíns
Remember, write the angry email first, wait a half hour, then re-write. Please keep the IRC flamewars to IRC where some of us can avoid it without losing contact with important discussion. :)
On 21/11/2020 15:34, Filipe Laíns via arch-dev-public wrote:
Hi all,
I want to propose adding all active Python versions to [community], not just the latest one. This would only entail adding the interpreter itself, no other packages.
Having access to interpreters for older active versions is really helpful for Python developers. This allows them to easily run test suites against older versions. It is very common for developers to maintain software against a couple major releases. Tools like tox or nox are able to automate testing against multiple Python versions, just needing the interpreter.
The current active Python releases are:
- 3.9
- 3.8
- 3.7
- 3.6
- 2.7
The list can be found here[1].
So, I propose introducing 2 new packages:
- python3.7
- python3.6
And when we update the python package to 3.9:
- python3.8
I am not super in favor of providing an old version, as we strive to provide the latest and greatest. However some things come to mind:
* Does the Python foundation still maintain older Python versions and do they provide proper security updates?
* This seems to be purely for Development right? Users would use this Python version to bootstrap a virtualenv I guess?
Greetings,
Jelle
On Thu, 2020-11-26 at 21:21 +0100, Jelle van der Waa wrote:
On 21/11/2020 15:34, Filipe Laíns via arch-dev-public wrote:
Hi all,
I want to propose adding all active Python versions to [community], not just the latest one. This would only entail adding the interpreter itself, no other packages.
Having access to interpreters for older active versions is really helpful for Python developers. This allows them to easily run test suites against older versions. It is very common for developers to maintain software against a couple major releases. Tools like tox or nox are able to automate testing against multiple Python versions, just needing the interpreter.
The current active Python releases are: - 3.9 - 3.8 - 3.7 - 3.6 - 2.7
The list can be found here[1].
So, I propose introducing 2 new packages: - python3.7 - python3.6
And when we update the python package to 3.9: - python3.8
I am not super in favor of providing an old version, as we strive to provide the latest and greatest. However some things come to mind:
We would still provide the latest and greatest :P
- Does the Python foundation still maintain older Python versions and do
they provide proper security updates?
It depends on the version. Python 3.6 and 3.7 are in security maintenance status.
I am okay limiting my proposal to interpreters which are getting security updates, which at the time being doesn't change anything regarding the versions affected.
It is also not clear in the proposal, but I have mentioned in the replies. I only want the interpreters there as long as they do not give us any issues. If a interpreter starts needing older dependencies or otherwise becomes a maintenance burden, it would be removed.
- This seems to be purely for Development right? Users would use this
Python version to bootstrap a virtualenv I guess?
The target is development, yes. You are correct, users, or tools, could use the interpreter to bootstrap a virtual environment using either virtualenv, which supports using external interpreters, or the builtin venv module.
Greetings,
Jelle
Cheers, Filipe Laíns
arch-dev-public@lists.archlinux.org