Re: [aur-general] Cycled dependencies in AUR
Hello. Today I tried to build ruby-rubocop [1] package and it's in a strange state. A lot of packages for the package are cross-dependent for the check, for example ruby-ruby-progressbar [2] and ruby-fuubar [3]. The proposal of how to solve it from Mario [4] is to skip the check() phase, see comment [5]. The same is valid for package ruby-rubocop-performance [6] that is dependency for rubocop check and requires it to be built IMHO the state when the packages aren't able to be built by the simple AUR helper command like `yay -S ruby-rubocop` should be considered as room for improvement. If I'm wrong, please say so. But I'd like to have AUR packages in an easily installable state and not passing additional arguments like `--nocheck` for makepackage commands. [1] - https://aur.archlinux.org/packages/ruby-rubocop [2] - https://aur.archlinux.org/packages/ruby-ruby-progressbar/ [3] - https://aur.archlinux.org/packages/ruby-fuubar/ [4] - https://aur.archlinux.org/account/supermario [5] - https://aur.archlinux.org/packages/ruby-rubocop-rspec/#comment-823420 [6] - https://aur.archlinux.org/packages/ruby-rubocop-performance/ Best regards, Mikhail f. Shiryaev p.s. Mario, sorry for sending you two messages.
On Tue, 24 Aug 2021, at 15:12, Mikhail f. Shiryaev via aur-general wrote:
Hello.
Today I tried to build ruby-rubocop [1] package and it's in a strange state. A lot of packages for the package are cross-dependent for the check, for example ruby-ruby-progressbar [2] and ruby-fuubar [3]. The proposal of how to solve it from Mario [4] is to skip the check() phase, see comment [5].
The same is valid for package ruby-rubocop-performance [6] that is dependency for rubocop check and requires it to be built
IMHO the state when the packages aren't able to be built by the simple AUR helper command like `yay -S ruby-rubocop` should be considered as room for improvement.
If I'm wrong, please say so. But I'd like to have AUR packages in an easily installable state and not passing additional arguments like `--nocheck` for makepackage commands.
[1] - https://aur.archlinux.org/packages/ruby-rubocop [2] - https://aur.archlinux.org/packages/ruby-ruby-progressbar/ [3] - https://aur.archlinux.org/packages/ruby-fuubar/ [4] - https://aur.archlinux.org/account/supermario [5] - https://aur.archlinux.org/packages/ruby-rubocop-rspec/#comment-823420 [6] - https://aur.archlinux.org/packages/ruby-rubocop-performance/
Best regards, Mikhail f. Shiryaev
p.s. Mario, sorry for sending you two messages.
There's not really much that can be done to work around this, since the dependency also exists upstream. Potentially, AUR helpers can detect these scenarios and do a first pass with `--nocheck`, and then a second rebuild. Note that this isn't the only instance of a circular dependency like this. Another known similar situation is rust, which requires rust to be built [1]. Again,the package only reflects upstream's situation. BTW: You might also be interested in quarry[2]. [1]: https://archlinux.org/packages/extra/x86_64/rust/ [2]: https://wiki.archlinux.org/title/Ruby#Quarry -- Hugo Osvaldo Barrera
On 2021-08-24 19:11, Hugo Osvaldo Barrera via aur-general wrote:
On Tue, 24 Aug 2021, at 15:12, Mikhail f. Shiryaev via aur-general wrote:
The same is valid for package ruby-rubocop-performance [6] that is dependency for rubocop check and requires it to be built
There's not really much that can be done to work around this, since the dependency also exists upstream. Potentially, AUR helpers can detect these scenarios and do a first pass with `--nocheck`, and then a second rebuild.
I don't know of *any* that do that yet, and even instructing users to do this by hand is difficult. As a *packager* building packages for my own package repo it takes quite a bit of brain power and trial/error to sort out what order to build. I tihnk I'd have to agree with Mikail here that this is leaves room for improvement, and AUR packages should make concessions for this.
Note that this isn't the only instance of a circular dependency like this. Another known similar situation is rust, which requires rust to be built [1]. Again,the package only reflects upstream's situation.
There are lots of other instances of this, but I'm not sure Rust is really the same. Rust has a bootstrap issue with *building* itself, but that is ⓐ not an issue for end users because a binary package for it exists in the repos and the bother of bootstrapping it is on packagers, and ⓑ is not at all the same as a test suite that introduces a circular loop. Incidentally I just rehashed one of these in Python the other day (again). The upstream projects in the Python font tooling world regularly assume their test suites are run with Tox and/or something that can just `pip` out for a wheel of whatever they need to run. There are at least 3 different loops over about 15 packages. My solution to the loop I was dealing with (besides opening an upstream bug report[1]) was to look around the circular loop for where something was only a *checkdepends* and disable the checks by default for that package. This is less than ideal too, but I think it's better than piles of users frustrated because they can't able to build anything and otherwise fully functional packages claiming they aren't able to satisfy dependencies. I experimented with adding code to leave the tests but keep them from running, such as adding this: BUILDENV+=(!check) This works for local builds with `makepkg` but it doesn't work for chroot builds and it doesn't work for most AUR helpers. Frustratingly the `options=()` field can't touch this either. I suppose I could get more aggressive with a custom env var check to inject the check deps and function if found so at least I as the packages maintainer can build it with checks, but that feels ugly. I too feel like there should be a better way to handle this. I don't know what it is. Caleb [1] https://github.com/typemytype/booleanOperations/issues/64
On 8/24/21 11:12 AM, Mikhail f. Shiryaev via aur-general wrote:
Hello.
Today I tried to build ruby-rubocop [1] package and it's in a strange state. A lot of packages for the package are cross-dependent for the check, for example ruby-ruby-progressbar [2] and ruby-fuubar [3]. The proposal of how to solve it from Mario [4] is to skip the check() phase, see comment [5].
The same is valid for package ruby-rubocop-performance [6] that is dependency for rubocop check and requires it to be built
IMHO the state when the packages aren't able to be built by the simple AUR helper command like `yay -S ruby-rubocop` should be considered as room for improvement.
If I'm wrong, please say so. But I'd like to have AUR packages in an easily installable state and not passing additional arguments like `--nocheck` for makepackage commands.
I've been debating with myself for a little while now on how to best to respond to this. On the one hand I totally get it that it's annoying to not be able to just `$helper -S package` and need to either manually edit the pkgbuild, or call --nocheck, or whatever. On the other hand running the test suite is extremely valuable to pick up breakage between both gem versions and ruby versions. Fortunately, for ruby packages the dependency cycle is usually only for checkdepends so once you've built and installed the package the first time upgrades should be smooth even with the check. It's possible that this is actually only something that we care about for the official repositories and so I'm out of place by including them on these AUR packages. I admit that I haven't seen circular dependencies elsewhere on the AUR, but I also don't install that many packages so maybe they just exist on packages that I haven't seen or don't use. Anyway, if the consensus (or even just some TU telling me to do it) is to get rid of the check() functions and checkdepends then I of course will comply -- I'll just be a little sad about it :)
[1] - https://aur.archlinux.org/packages/ruby-rubocop [2] - https://aur.archlinux.org/packages/ruby-ruby-progressbar/ [3] - https://aur.archlinux.org/packages/ruby-fuubar/ [4] - https://aur.archlinux.org/account/supermario [5] - https://aur.archlinux.org/packages/ruby-rubocop-rspec/#comment-823420 [6] - https://aur.archlinux.org/packages/ruby-rubocop-performance/
Best regards, Mikhail f. Shiryaev
p.s. Mario, sorry for sending you two messages.
No worries :) I think I may have done the same to you since I needed to subscribe this address to the list in order to respond, sorry!
Cheers, Mario
On 8/25/21 4:03 AM, Mario Finelli via aur-general wrote:
I've been debating with myself for a little while now on how to best to respond to this. On the one hand I totally get it that it's annoying to not be able to just `$helper -S package` and need to either manually edit the pkgbuild, or call --nocheck, or whatever. On the other hand running the test suite is extremely valuable to pick up breakage between both gem versions and ruby versions. Fortunately, for ruby packages the dependency cycle is usually only for checkdepends so once you've built and installed the package the first time upgrades should be smooth even with the check.
It's possible that this is actually only something that we care about for the official repositories and so I'm out of place by including them on these AUR packages. I admit that I haven't seen circular dependencies elsewhere on the AUR, but I also don't install that many packages so maybe they just exist on packages that I haven't seen or don't use.
Anyway, if the consensus (or even just some TU telling me to do it) is to get rid of the check() functions and checkdepends then I of course will comply -- I'll just be a little sad about it :)
I have also a few ruby-* packages in AUR and I'm aware that they also bring in cyclic check dependencies, especially in the ruby-rails world and their subpackages. I'm in general also a fan of having tests for packages since they do serve a purpose. But in my dealings with these packages I have an increasingly harder time to manage the cycles and keeping my local repo up-to-date. Especially when a new rails version comes out. If there is a consensus from this thread to avoid such cycles then I will be happy to update my packages and just comment out the check() related lines. That way if people want to run them later they can re-enable them when the inspect the PKGBUILD file (which they should do anyway during install). Best regards Segaja
On Wed, Aug 25, 2021 at 9:55 AM Andreas 'Segaja' Schleifer via aur-general < aur-general@lists.archlinux.org> wrote:
On 8/25/21 4:03 AM, Mario Finelli via aur-general wrote:
I've been debating with myself for a little while now on how to best to respond to this. On the one hand I totally get it that it's annoying to not be able to just `$helper -S package` and need to either manually edit the pkgbuild, or call --nocheck, or whatever. On the other hand running the test suite is extremely valuable to pick up breakage between both gem versions and ruby versions. Fortunately, for ruby packages the dependency cycle is usually only for checkdepends so once you've built and installed the package the first time upgrades should be smooth even with the check.
It's possible that this is actually only something that we care about for the official repositories and so I'm out of place by including them on these AUR packages. I admit that I haven't seen circular dependencies elsewhere on the AUR, but I also don't install that many packages so maybe they just exist on packages that I haven't seen or don't use.
Anyway, if the consensus (or even just some TU telling me to do it) is to get rid of the check() functions and checkdepends then I of course will comply -- I'll just be a little sad about it :)
I have also a few ruby-* packages in AUR and I'm aware that they also bring in cyclic check dependencies, especially in the ruby-rails world and their subpackages.
I'm in general also a fan of having tests for packages since they do serve a purpose. But in my dealings with these packages I have an increasingly harder time to manage the cycles and keeping my local repo up-to-date. Especially when a new rails version comes out.
If there is a consensus from this thread to avoid such cycles then I will be happy to update my packages and just comment out the check() related lines. That way if people want to run them later they can re-enable them when the inspect the PKGBUILD file (which they should do anyway during install).
Best regards
Segaja
Thank you for your clarification everybody. I got the reasons behind the decision and they make sense, for sure! Regards, Mikhail
Hi, I'm the TU that recommended this. On 24.08.21 17:12, Mikhail f. Shiryaev via aur-general wrote:
Hello.
Today I tried to build ruby-rubocop [1] package and it's in a strange state. A lot of packages for the package are cross-dependent for the check, for example ruby-ruby-progressbar [2] and ruby-fuubar [3]. The proposal of how to solve it from Mario [4] is to skip the check() phase, see comment [5].
The same is valid for package ruby-rubocop-performance [6] that is dependency for rubocop check and requires it to be built
IMHO the state when the packages aren't able to be built by the simple AUR helper command like `yay -S ruby-rubocop` should be considered as room for improvement.
I'm not sure about this. The AUR was always a tricky environment. There is a fine line between proper packages with documented dependencies and tests or an easy way to install packages. The "easy" way is always to just install the gem artifact from rubygems.org within a PKGBUILD and ignore all dependencies/tests. This will just break on every ruby upgrade and you need to ensure somehow you've all dependencies. Sometimes it's the best to build stuff locally and maintain the related packages in a private repo or to patch the PKGBUILDs and ignore the deps. This was also done in a preparation to move the packages into community. We already have other strange depenency circles, like ruby-faraday-* has them.
If I'm wrong, please say so. But I'd like to have AUR packages in an easily installable state and not passing additional arguments like `--nocheck` for makepackage commands.
[1] - https://aur.archlinux.org/packages/ruby-rubocop [2] - https://aur.archlinux.org/packages/ruby-ruby-progressbar/ [3] - https://aur.archlinux.org/packages/ruby-fuubar/ [4] - https://aur.archlinux.org/account/supermario [5] - https://aur.archlinux.org/packages/ruby-rubocop-rspec/#comment-823420 [6] - https://aur.archlinux.org/packages/ruby-rubocop-performance/
Best regards, Mikhail f. Shiryaev
p.s. Mario, sorry for sending you two messages.
participants (6)
-
Andreas 'Segaja' Schleifer
-
Caleb Maclennan
-
Hugo Osvaldo Barrera
-
Mario Finelli
-
Mikhail f. Shiryaev
-
Tim Meusel