On Fri, Jan 10, 2020, 16:43 Christian Rebischke via arch-dev-public < arch-dev-public@archlinux.org> wrote:

Hi everybody,

I would like to propose that we create todos for rebuilds of language specific packages.

We had two major rebuilds in the last months: python3.8 and ruby2.7.

Can we agree that we create a todo before such rebuilds? The advantages outweigh the disadvantages. We would gain:

Hi, I'm not sure I understand. Can you clearly state the problems we've encountered due to not doing this? What downsides do you see to your proposal? Can you think of any alternative solutions? * More people help rebuilding the packages.

Solving the wrong problem, IMO. This is largely toil and should be automated away. Foutrelis already has such a system that we've used for rebuilds in the past. We could/should instead work towards making this more generally available on the build machines. * Every maintainer gets informed about the rebuild.

As a maintainer, I don't care that you're rebuilding my package to keep up with library changes. Rather, I'm thankful to whomever did this for me. Why would a language rebuild differ from any other soname bump? * Maintainers have the possibility to test the packages.

Why is this not currently possible? Couldn't we just use testing prior to pushing out to the repos (something we've done in the past and continue to do)? What about packages which aren't using the lowlevel API and are simply interpreted code. Those don't get rebuilt, but they're potentially impacted by the language version bump. They'll never be called out on a rebuild lost because we generate those based on ELF dependencies. dR If tools exist for creating todos, I would like to ask the persons with

such tools to make them available for everybody (if not already happened).

Greetings, shibumi

On 01/11/20 at 09:04am, Anatol Pomozov via arch-dev-public wrote:

Hello

On Sat, Jan 11, 2020 at 6:58 AM Dave Reisner wrote:

...

On Fri, Jan 10, 2020, 16:43 Christian Rebischke via arch-dev-public < arch-dev-public@archlinux.org> wrote:

...

Hi everybody,

I would like to propose that we create todos for rebuilds of language specific packages.

We had two major rebuilds in the last months: python3.8 and ruby2.7.

Can we agree that we create a todo before such rebuilds? The advantages outweigh the disadvantages. We would gain:

Hi,

I'm not sure I understand. Can you clearly state the problems we've encountered due to not doing this? What downsides do you see to your proposal? Can you think of any alternative solutions?

* More people help rebuilding the packages.

...

Solving the wrong problem, IMO. This is largely toil and should be automated away.

I agree with every statement that Dave made. Especially with this one.

Yes, we should make this rebuilder official in some way and properly document it so multiple people can use it. Some languages however require special treatment such as Haskell and require rebuilds from GHC => Haskell-foo => Haskell-bar etc which can become complicated. For example if I want to update a dependency of taskell I will have to rebuild all depending programs/libs so tooling which makes that easier is welcome to me. I know Felix has some stuff but we should make this more visible. I would propose to document how to rebuild a library/program on the Package Guidelines pages which I plan to do for Haskell after talking with Felix.

We need to automate as much our daily routine as possible. Scripting/automation is the only way to keep the increasing complexity of the system under control.

Yes!

...

As a maintainer, I don't care that you're rebuilding my package to keep up with library changes. Rather, I'm thankful to whomever did this for me.

Why would a language rebuild differ from any other soname bump?

For Haskell this is the case, but I am not sure if this is unique.

...

* Maintainers have the possibility to test the packages.

Did you have any problems with testing the recent language rebuilds?

We have never tested anything in [staging] where rebuilds happen since that's not possible at all since it's always half broken. Testing happens in [testing]. -- Jelle van der Waa

On 1/10/20 4:42 PM, Christian Rebischke via arch-dev-public wrote:

Hi everybody,

I would like to propose that we create todos for rebuilds of language specific packages.

We had two major rebuilds in the last months: python3.8 and ruby2.7.

Can we agree that we create a todo before such rebuilds? The advantages outweigh the disadvantages. We would gain:

* More people help rebuilding the packages.'

What help is needed? If this is just about having more people sed the pkgrel variable with "$pkgrel + 1", then try to build it, more people doesn't actually help. We have automated rebuilders which are very capable in this regard.

* Every maintainer gets informed about the rebuild.

I agree with you that this is indeed a problem, and I would like to propose a pretty simple solution. Let's post on arch-dev-public to give people a heads-up. This means even if your package failed to be detected for rebuilding and would never appear on any TODO, you as a maintainer know that it happened and can manually rebuild your package.

* Maintainers have the possibility to test the packages.

At least for the python rebuilds, the process of rebuilding the ecosystem is long and painfully drawn out, *because* packages with failing testsuites cannot be rebuilt automatically and go onto a TODO list of broken packages. Given this thread started because we just rebuilt ruby, can I assume that PKGBUILDs for ruby packages are in the general habit of not containing check() functions for running unittests? Either because upstream does not have unittests or because they are not being run? If packages have upstream unittests but don't run them, then the maintainer of the package has been derelict in his or her duty. If packages do NOT have upstream unittests, then this is unfortunate, and I don't currently have an answer for what we should do. :(

If tools exist for creating todos, I would like to ask the persons with such tools to make them available for everybody (if not already happened).

It's a website submission form that expects you to write some explanatory message, then fill in a newline-separated list of pkgnames. Any rebuilder must by definition have the latter, even if that rebuilder is "I scrolled through archweb and did it all manually by flipping back and forth between my terminal and my browser". No "tool for creating todos" need exist. Ask instead about tools for enumerating language dependencies. ... For python, it's pretty simple. pkgfile -d '/usr/lib/python3.8/' For ruby, it's also pretty simple: pkgfile -d '/usr/lib/ruby/' Also sogrep to see if any packages link to the shared library libpython3.8 or libruby, but hey, that's what we already do for rebuilds of only 3 or 4 packages. -- Eli Schwartz Bug Wrangler and Trusted User

On 13/01/2020 18.28, Christian Rebischke via arch-dev-public wrote:

[testing] has been used and I wanted to test my packages, but the push from [testing] to [community] happened before my scheduled date for this.

Have you raised this on the TU or staff channels before the move happened? Otherwise I don't find it surprising no one guessed your personal testing plan. Documented procedures are surely nice, proper communication is even better. BP