[arch-dev-public] Add active Python versions to the repos

[Filipe Laíns]

On Sun, 2020-11-22 at 11:45 -0500, Eli Schwartz via arch-dev-public wrote:
> On 11/22/20 10:05 AM, Filipe Laíns wrote:
> > On Sat, 2020-11-21 at 20:24 -0500, Eli Schwartz via arch-dev-public wrote:
> > > Your analysis is correct, it is indeed hell. I'm not sure why that is an
> > > argument in favor of doing even more of it though.
> > > 
> > > Now, if you were proposing to get rid of some of this, I could get
> > > behind that.
> > 
> > It was not an analysis, "hell" was an used here as an idiom.
> 
> https://i.imgur.com/axJmn.gif
> 
> You were unintentionally accurate.
> 
> > > > even Python.
> > > 
> > > I don't know if maybe you've missed the fact that we've spent like a
> > > year now aggressively upgrading or removing leaf packages depending on
> > > python2 in search of that glorious Promised Future where we can finally
> > > remove the python2 package itself?
> > 
> > I did not miss that. I absolutely want to see the python2 modules removed,
> > not
> > _necessarily_ the python2 interpreter.
> 
> Now I'm starting to see part of the reason why people are having a
> difference of opinions here.
> 
> I think I probably speak for a bunch of people when I say I'd like to
> see the interpreter dropped due to not being needed by any packages
> *and* being an end of life interpreter. The interpreter is only useful
> inasmuch as people use it to run software, so if we don't have any...
> 
> I expect by the time we finally remove all reverse dependencies for it,
> the gap between the python2 EOL and its removal from the distro will be
> quite a bit biger than it currently is.
> 
> > > And we're doing that even for a major incompatible release that most
> > > people argue is actually a different language.
> > > 
> > > I'm not sure how this suddenly became an argument to package a ton of
> > > point releases that aren't even incompatible.
> > 
> > Except they are... Python does not follow semver, only patch releases are
> > supposed to be compatible. They do try to keep breakage to a minimum, but it
> > definitely exists. One very obvious example is the introduction of "async"
> > and
> > "await" as reserved keywords. There are a bunch of other backwards
> > incompatible
> > changes, so yes, they are incompatible releases, to the point you could call
> > them different languages.
> 
> This is the first time I ever heard anyone describe "backwards
> incompatible changes means it is now a different programming language".

You literally mentioned that in your reply. Though I assume you meant more
significant change than what is being discussed. If the languages semantics are
different one could argue it is actually a different language. Anyway, this does
not really matter for the issue being discussed.

> I had backwards incompatible changes in a patchlevel bugfix in the
> python 3.8.x release line causing immediate traceback and SystemExit on
> a program's startup, does that mean every bugfix is now a different
> programming language?
> 
> (I mean this quite literally. Python bugfixed a function not working as
> expected under certain situations. The program it broke was working
> around the failure by manually doing $thing, the traceback happened when
> you try to do $thing twice.)
> 
> > > > I don't think having people opening bugs because they are
> > > > deliberately using an older version of Python is a big problem. It
> > > > hasn't been for any of the other languages, I don't think it will be
> > > > here.
> > > > I think this is an hypothetical that doesn't really materialize into
> > > > reality.
> > > 
> > > You're proposing something far beyond the scope of what we do for other
> > > languages, and ignoring that for other languages we only do it if driven
> > > to in desperation because another official repository package depends on
> > > it.
> > > 
> > > We don't introduce leaf packages of ancient versions of currently
> > > packaged interpreters just for fun. I don't even think anyone else does
> > > that either.
> > 
> > That was not of the point of me bringing up other languages. The whole point
> > of
> > me bringing up other languages was to show that Andreas' concern is unlikely
> > to
> > materialize into reality.
> 
> You think no one reports bugs for python-* packages that we should
> provide the python2 version? I promise you, you're wrong. I've closed
> those bugs, so I know they exist.

That is not "users raising bugs that something doesn't work with an older
version of python", and is not what I argued against.

> You specifically brought up Java and Javascript, languages where each
> application is forced to vendor their entire dependency list rather than
> using system modules. For this reason, people don't report the kind of
> bugs Andreas is concerned about for Java/nodejs...

They are not necessarily forced, but whatever, just ignore that.

> So I assumed you could not possibly be talking about Andreas' concern,
> and I responded somewhat tangentially myself. We can, of course, talk
> about Andreas' concern, but given I've now submitted the authoritative
> facts on the matter, I'm not sure what else there is to say.
> 
> It does indisputably happen right now. The logical inference is, it will
> happen *more*, if we add more runtimes for people to submit bugtracker
> tickets for.

Can you show me an handful of examples? And not about people asking us to
provide Python 2 versions of packages, but rather about what Andreas mentioned.

> > > You want to package old versions of the python3 interpreter "but not
> > > modules", because "people would like to use it for tox".
> > 
> > People can use it tox or to create virtual environments directly. The main
> > point
> > of the proposal is that people have use for the old interpreters and I would
> > like to provide them.
> 
> The main point of my objection is that their use isn't to run software,
> it's to regression test it -- something more suited to Continuous
> Integration which already targets a wide range of operating systems
> *and* interpreter versions, and which can be used to block merging.

There is use, you just keep ignoring it.

> > > Old versions of stuff that people need for who knows what weird reason
> > > is *the textbook reason* why the AUR exists. How long it takes to
> > > compile stuff in the AUR is not our problem, and we don't have a
> > > rationale to upload gcc-{4,5,6,7,8,9} because actual users need to use a
> > > lot of diverse compilers "to test that it still works on really old
> > > compilers".
> > 
> > AFAIK, the textbook reason why the AUR exists is to provide software that
> > isn't
> > in the official repos. It being older versions should be completely
> > incidental,
> > but not the "textbook reason".
> 
> The AUR exists to provide software which is not in the official repos
> for practical OR ideological reasons.
> 
> Old stuff is ideological reasons.
> 
> > > Likewise, if people want to test support for old versions of python3,
> > > they have options which don't involve uploading trash into [community].
> > 
> > I like how you refereed to the currently active Python releases as "trash",
> > will
> > keep that in mind.
> 
> I like how you reinterpret my reference to Python releases which are in
> LTS "only security fixes are permitted and even then, we won't release
> Windows or macOS binaries, just source code for you to compile yourself"
> mode as "currently active".
> 
> They are not active. Active would imply that on
> https://devguide.python.org/#branchstatus they are marked as "bugfix" or
> "feature", indicating the Python developers will actually fix bugs in
> them and release multi-platform installers in addition to source code.
> 
> So python >= 3.8 I would call "not trash", but you want to upload python
> 3.6 and python 3.7, which I would call trash at this point.

Once again, nice.

> They are expired versions. They went moldy, and the only people who
> should be using them are people who have been storing their entire
> computing environment in the freezer in order to eke out another year or
> two of emergency use.

I really wish you would stop making this kind of statements. It's not nice to
anyone, actually, the opposite.

> > > - Do what normal people do, and build AUR packages they need
> > > - Ask FFY00 to provide signed repos
> > > - Use travis CI, which doesn't use distro packages but provides a
> > >   diverse set of hand-compiled python versions, which is apparently some
> > >   kind of gold standard for doing regression testing on lots of
> > >   different python releases ;)
> > > - Use pyenv
> > > 
> > > Apparently none of these are viable options in your mind, and the most
> > > Arch-like of them, option #1 "use the AUR" you believe to be problematic
> > > because it takes a long time to build with optimizations... even though
> > > you yourself said those packages don't enable optimizations and don't
> > > take a long time???

I want to have packages built with optimizations but doing that takes a long
time, that's why none of the current options offers that. What is difficult to
understand?

> > > (If you're using python-$old exclusively for regression tests in tox,
> > > you don't need a super optimized version.)
> > 
> > Tests can take a long time to run, multiply that by the interpreters you
> > need to
> > run them against. But that is only part of the reason.
> 
> Why are you nitpicking the offhand comment I made, rather than my actual
> recommendations?

Because I felt like I had explained that already, I don't want to be reiterating
the same point every time. I did it now, won't do it again if you come with the
same questions.

> Again, this is a non-issue if you stick the packages in an unofficial
> user repository and enable optimizations. And users are welcome to make
> the tradeoff of less time compiling in exchange for slightly longer test
> runtimes. Building a package is a one-time cost anyway, which you could
> do overnight.

Yeah, none of that provide a nice user experience, which is what I am trying to
fix.

> > As long as there is someone willing to maintain the packages, why would we
> > block
> > it? I doesn't really have any effect on the rest of the people. If the
> > person
> > start having trouble maintaining them, they can be dropped. It doesn't
> > affect
> > you and helps other people.
> 
> One reason is, for the same reason we ideologically don't want
> gcc-{4,5,6,7,8,9} in the repos, no matter how "little effect" it has on
> other people. We don't wish to promote using old things.

Now, that is a proper argument. I think it is reasonable to deny this based on
that, although not very nice for users.
Personally, I do not think it is a big deal. I don't think us providing Java 8
or Ruby 2.6 in the repos is actively promoting people to use that software.
People need to use a custom package, with custom paths, which I think actively
discourages them from using it. What it does is enabling them to use it, which
is good for the people who need it, but not for the people who don't but choose
to use it anyway, those people are already being discouraged as I point out
above.

So yeah, personally I do not think that is significant enough to block us from
having these interpreters, which would in fact help people. But others might not
agree, so it is up to the team.

> Sometimes we include *one* old version of gcc in desperation, because
> CUDA cannot be convinced to use current gcc and cannot be fixed because
> it's closed-source. Even this bothers Andreas (and me, and actually, it
> bothers Sven too even though he's the one that uploaded gcc9 to begin
> with) but we put up with it because it's a dependency of something else,
> and we only package the one old version needed and upgrade it as soon as
> cuda supports something more modern.

I understand that, however I think most of those bothers come from us having to
maintain another GCC, not from not wanting to provide it.

> We wouldn't have gcc9 merely "for the public service convenience of
> anyone who needs it".

*You* wouldn't. If someone wants to do it and it doesn't negatively impact
anyone else, I am neutral.

Cheers,
Filipe Laíns
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <https://lists.archlinux.org/pipermail/arch-dev-public/attachments/20201122/d82753c9/attachment.sig>