[arch-dev-public] Add active Python versions to the repos

[Eli Schwartz]

On 11/22/20 10:05 AM, Filipe Laíns wrote:
> On Sat, 2020-11-21 at 20:24 -0500, Eli Schwartz via arch-dev-public wrote:
>> Your analysis is correct, it is indeed hell. I'm not sure why that is an
>> argument in favor of doing even more of it though.
>>
>> Now, if you were proposing to get rid of some of this, I could get
>> behind that.
> 
> It was not an analysis, "hell" was an used here as an idiom.

https://i.imgur.com/axJmn.gif

You were unintentionally accurate.

>>> even Python.
>>
>> I don't know if maybe you've missed the fact that we've spent like a
>> year now aggressively upgrading or removing leaf packages depending on
>> python2 in search of that glorious Promised Future where we can finally
>> remove the python2 package itself?
> 
> I did not miss that. I absolutely want to see the python2 modules removed, not
> _necessarily_ the python2 interpreter.

Now I'm starting to see part of the reason why people are having a
difference of opinions here.

I think I probably speak for a bunch of people when I say I'd like to
see the interpreter dropped due to not being needed by any packages
*and* being an end of life interpreter. The interpreter is only useful
inasmuch as people use it to run software, so if we don't have any...

I expect by the time we finally remove all reverse dependencies for it,
the gap between the python2 EOL and its removal from the distro will be
quite a bit biger than it currently is.

>> And we're doing that even for a major incompatible release that most
>> people argue is actually a different language.
>>
>> I'm not sure how this suddenly became an argument to package a ton of
>> point releases that aren't even incompatible.
> 
> Except they are... Python does not follow semver, only patch releases are
> supposed to be compatible. They do try to keep breakage to a minimum, but it
> definitely exists. One very obvious example is the introduction of "async" and
> "await" as reserved keywords. There are a bunch of other backwards incompatible
> changes, so yes, they are incompatible releases, to the point you could call
> them different languages.

This is the first time I ever heard anyone describe "backwards
incompatible changes means it is now a different programming language".

I had backwards incompatible changes in a patchlevel bugfix in the
python 3.8.x release line causing immediate traceback and SystemExit on
a program's startup, does that mean every bugfix is now a different
programming language?

(I mean this quite literally. Python bugfixed a function not working as
expected under certain situations. The program it broke was working
around the failure by manually doing $thing, the traceback happened when
you try to do $thing twice.)

>>> I don't think having people opening bugs because they are
>>> deliberately using an older version of Python is a big problem. It
>>> hasn't been for any of the other languages, I don't think it will be
>>> here.
>>> I think this is an hypothetical that doesn't really materialize into
>>> reality.
>>
>> You're proposing something far beyond the scope of what we do for other
>> languages, and ignoring that for other languages we only do it if driven
>> to in desperation because another official repository package depends on it.
>>
>> We don't introduce leaf packages of ancient versions of currently
>> packaged interpreters just for fun. I don't even think anyone else does
>> that either.
> 
> That was not of the point of me bringing up other languages. The whole point of
> me bringing up other languages was to show that Andreas' concern is unlikely to
> materialize into reality.

You think no one reports bugs for python-* packages that we should
provide the python2 version? I promise you, you're wrong. I've closed
those bugs, so I know they exist.

You specifically brought up Java and Javascript, languages where each
application is forced to vendor their entire dependency list rather than
using system modules. For this reason, people don't report the kind of
bugs Andreas is concerned about for Java/nodejs...

So I assumed you could not possibly be talking about Andreas' concern,
and I responded somewhat tangentially myself. We can, of course, talk
about Andreas' concern, but given I've now submitted the authoritative
facts on the matter, I'm not sure what else there is to say.

It does indisputably happen right now. The logical inference is, it will
happen *more*, if we add more runtimes for people to submit bugtracker
tickets for.

>> You want to package old versions of the python3 interpreter "but not
>> modules", because "people would like to use it for tox".
> 
> People can use it tox or to create virtual environments directly. The main point
> of the proposal is that people have use for the old interpreters and I would
> like to provide them.

The main point of my objection is that their use isn't to run software,
it's to regression test it -- something more suited to Continuous
Integration which already targets a wide range of operating systems
*and* interpreter versions, and which can be used to block merging.

>> Old versions of stuff that people need for who knows what weird reason
>> is *the textbook reason* why the AUR exists. How long it takes to
>> compile stuff in the AUR is not our problem, and we don't have a
>> rationale to upload gcc-{4,5,6,7,8,9} because actual users need to use a
>> lot of diverse compilers "to test that it still works on really old
>> compilers".
> 
> AFAIK, the textbook reason why the AUR exists is to provide software that isn't
> in the official repos. It being older versions should be completely incidental,
> but not the "textbook reason".

The AUR exists to provide software which is not in the official repos
for practical OR ideological reasons.

Old stuff is ideological reasons.

>> Likewise, if people want to test support for old versions of python3,
>> they have options which don't involve uploading trash into [community].
> 
> I like how you refereed to the currently active Python releases as "trash", will
> keep that in mind.

I like how you reinterpret my reference to Python releases which are in
LTS "only security fixes are permitted and even then, we won't release
Windows or macOS binaries, just source code for you to compile yourself"
mode as "currently active".

They are not active. Active would imply that on
https://devguide.python.org/#branchstatus they are marked as "bugfix" or
"feature", indicating the Python developers will actually fix bugs in
them and release multi-platform installers in addition to source code.

So python >= 3.8 I would call "not trash", but you want to upload python
3.6 and python 3.7, which I would call trash at this point.

They are expired versions. They went moldy, and the only people who
should be using them are people who have been storing their entire
computing environment in the freezer in order to eke out another year or
two of emergency use.

>> - Do what normal people do, and build AUR packages they need
>> - Ask FFY00 to provide signed repos
>> - Use travis CI, which doesn't use distro packages but provides a
>>   diverse set of hand-compiled python versions, which is apparently some
>>   kind of gold standard for doing regression testing on lots of
>>   different python releases ;)
>> - Use pyenv
>>
>> Apparently none of these are viable options in your mind, and the most
>> Arch-like of them, option #1 "use the AUR" you believe to be problematic
>> because it takes a long time to build with optimizations... even though
>> you yourself said those packages don't enable optimizations and don't
>> take a long time???
>>
>> (If you're using python-$old exclusively for regression tests in tox,
>> you don't need a super optimized version.)
> 
> Tests can take a long time to run, multiply that by the interpreters you need to
> run them against. But that is only part of the reason.

Why are you nitpicking the offhand comment I made, rather than my actual
recommendations?

Again, this is a non-issue if you stick the packages in an unofficial
user repository and enable optimizations. And users are welcome to make
the tradeoff of less time compiling in exchange for slightly longer test
runtimes. Building a package is a one-time cost anyway, which you could
do overnight.

> As long as there is someone willing to maintain the packages, why would we block
> it? I doesn't really have any effect on the rest of the people. If the person
> start having trouble maintaining them, they can be dropped. It doesn't affect
> you and helps other people.

One reason is, for the same reason we ideologically don't want
gcc-{4,5,6,7,8,9} in the repos, no matter how "little effect" it has on
other people. We don't wish to promote using old things.

Sometimes we include *one* old version of gcc in desperation, because
CUDA cannot be convinced to use current gcc and cannot be fixed because
it's closed-source. Even this bothers Andreas (and me, and actually, it
bothers Sven too even though he's the one that uploaded gcc9 to begin
with) but we put up with it because it's a dependency of something else,
and we only package the one old version needed and upgrade it as soon as
cuda supports something more modern.

We wouldn't have gcc9 merely "for the public service convenience of
anyone who needs it".

-- 
Eli Schwartz
Bug Wrangler and Trusted User
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x84818A6819AF4A9B.asc
Type: application/pgp-keys
Size: 21716 bytes
Desc: not available
URL: <https://lists.archlinux.org/pipermail/arch-dev-public/attachments/20201122/f86cb6de/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.archlinux.org/pipermail/arch-dev-public/attachments/20201122/f86cb6de/attachment-0001.sig>