[arch-dev-public] cleanup dead Xorg packages
I've created a bug[1] to follow a long overdue Xorg cleanup to drop legacy and dead code from our packages. This will require fixing makedepends in a 2nd step. I will do the most work over the next days/weeks. This all didn't fit well into a single ToDo list. If you find more really dead Xorg packages add it please to the bug tracker with a link to its removal. Use the mailing list for further discussion if needed when you think a certain feature isn't widely used and should be dropped as well. -Andy [1] https://bugs.archlinux.org/task/64892
Packages have been rebuilt and prepared to remove obsolete libdmx and libxxf86dga. Xorgproto legacy support has been removed and wherever it was added to be a runtime dependency it is now a build time dependency. Some packages will need additional xorgproto makedependency added when missing some header now that the libraries won't pull it in anymore. That behavior is desired. I'm still in the process of fixing the move from legacy proto headers to xorgproto headers. I'm going to commit the changes to trunk for future builds. There's no package replacing libdmx and libxxf86dga so manual intervention will be needed. So here's a small news draft: "Xorg cleanup requires manual intervention "In the process of Xorg cleanup [1] the update requires manual intervention when you hit this message: :: installing xorgproto (2019.2-2) breaks dependency 'inputproto' required by lib32-libxi :: installing xorgproto (2019.2-2) breaks dependency 'dmxproto' required by libdmx :: installing xorgproto (2019.2-2) breaks dependency 'xf86dgaproto' required by libxxf86dga when updating, use: `pacman -Rdd libdmx libxxf86dga && pacman -Syu` to perform the upgrade. After the update it will be safe to also remove the "xorgproto" package. [1] https://bugs.archlinux.org/task/64892" Note: One single package "Nexuiz" couldn't be fixed to work without libxxf86dga. The package maintainer has been informed to look out for a fix or remove the package and maintain it in AUR where libxxf86dga can be added. -Andy
On Thu, Dec 19, 2019, 23:35 Andreas Radke via arch-dev-public < arch-dev-public@archlinux.org> wrote:
when updating, use: `pacman -Rdd libdmx libxxf86dga && pacman -Syu`
Shouldn't we suggest -Rc instead of -Rdd here?
Em dezembro 19, 2019 19:45 Jan Alexander Steffens via arch-dev-public escreveu:
On Thu, Dec 19, 2019, 23:35 Andreas Radke via arch-dev-public < arch-dev-public@archlinux.org> wrote:
when updating, use: `pacman -Rdd libdmx libxxf86dga && pacman -Syu`
Shouldn't we suggest -Rc instead of -Rdd here?
I've had to run -Rc here. But I'm not sure we can force users to do that, and -Rc might have unintended consequences. I'm not sure what's the best approach is, but we will have manual intervention regardless, it seems. Regards, Giancarlo Razzolini
On 20/12/19 8:35 am, Andreas Radke via arch-dev-public wrote:
Packages have been rebuilt and prepared to remove obsolete libdmx and libxxf86dga. Xorgproto legacy support has been removed and wherever it was added to be a runtime dependency it is now a build time dependency. Some packages will need additional xorgproto makedependency added when missing some header now that the libraries won't pull it in anymore. That behavior is desired. I'm still in the process of fixing the move from legacy proto headers to xorgproto headers. I'm going to commit the changes to trunk for future builds.
There's no package replacing libdmx and libxxf86dga so manual intervention will be needed. So here's a small news draft:
"Xorg cleanup requires manual intervention
"In the process of Xorg cleanup [1] the update requires manual intervention when you hit this message:
:: installing xorgproto (2019.2-2) breaks dependency 'inputproto' required by lib32-libxi :: installing xorgproto (2019.2-2) breaks dependency 'dmxproto' required by libdmx :: installing xorgproto (2019.2-2) breaks dependency 'xf86dgaproto' required by libxxf86dga
when updating, use: `pacman -Rdd libdmx libxxf86dga && pacman -Syu`
to perform the upgrade. After the update it will be safe to also remove the "xorgproto" package.
This why does xorgproto not provides=('inputproto' .... )? Then all we need to do is announce, update and remove. Allan
On 12/19/19 6:05 PM, Allan McRae via arch-dev-public wrote:
There's no package replacing libdmx and libxxf86dga so manual intervention will be needed. So here's a small news draft:
"Xorg cleanup requires manual intervention
"In the process of Xorg cleanup [1] the update requires manual intervention when you hit this message:
:: installing xorgproto (2019.2-2) breaks dependency 'inputproto' required by lib32-libxi :: installing xorgproto (2019.2-2) breaks dependency 'dmxproto' required by libdmx :: installing xorgproto (2019.2-2) breaks dependency 'xf86dgaproto' required by libxxf86dga
when updating, use: `pacman -Rdd libdmx libxxf86dga && pacman -Syu`
to perform the upgrade. After the update it will be safe to also remove the "xorgproto" package.
This why does xorgproto not provides=('inputproto' .... )? Then all we need to do is announce, update and remove.
Allan
Well, this is our new fancy all-the-everything-in-one package, but the inputproto headers / inputproto.pc won't be provided anymore... I think conflicts would be better fitted here, but the bigger question I have is why do we need to be so quick to remove things which apparently work, even if they're legacy/dead, *before* we fix the downstream users? It's hardly difficult to deprecate packages before pulling the plug on them, in cases like this. -- Eli Schwartz Bug Wrangler and Trusted User
With this move I've "fixed" libx11 no more depending at runtime on xorgproto package. I think no headers belong to an end user system and the libx11 library itself doesn't depend on it. But we also ship libx11-devel part inside the package and this indead depends on xorgproto headers. The libx11 .pc file clearly wants to have the headers installed. In the past it was enough to include libx11 to also pull in the proto headers at build time. This is now broken. Some devs call libx11 broken though only its -devel part is. After some discussion on IRC these solution are possible: a) revert to make libx11 depend again on xorgproto headers. This is the pragmatic way and would not need any further work. It just installs header files to the user system that aren't needed in any way there. So we did in the past and I don't really like it as it's not correct to me. b) stay with changed libx11 and add xorgproto to packages that check for any of its headers. This needs to be done to an amount of ~300 packages when hitting build errors over the next time. c) go an unusual way here and split libx11 into libx11, libx11-devel depending on xorgproto and maybe even libx11-xcb. This is the way distros go that support splitting libraries. It's probably the technical correct solution but will also require packages to makedepend on libx11-devel and save us no work. Other distributions have chosen what they prefer. That a decision that needs to be done downstream. I'd like to have either solution b) or c) in Arch to have a clear and more "transient" build time dependency. I guess it may help us also in the future when moving some day away from Xorg to its successor. But if majority wants solution a) back I'm fine reverting this change. Please vote. -Andy
El 21/12/19 a las 9:41 Andreas Radke via arch-dev-public escribió:
After some discussion on IRC these solution are possible:
a) revert to make libx11 depend again on xorgproto headers. This is the pragmatic way and would not need any further work. It just installs header files to the user system that aren't needed in any way there. So we did in the past and I don't really like it as it's not correct to me.
b) stay with changed libx11 and add xorgproto to packages that check for any of its headers. This needs to be done to an amount of ~300 packages when hitting build errors over the next time.
c) go an unusual way here and split libx11 into libx11, libx11-devel depending on xorgproto and maybe even libx11-xcb. This is the way distros go that support splitting libraries. It's probably the technical correct solution but will also require packages to makedepend on libx11-devel and save us no work.
I'm fine with either (a) or (b), both can be seen as correct for some definition of "dependencies". But please not (c), it doesn't really fix anything and it's an unnecessary divergence from our usual practice of not having split devel packages.
Downstream consumers of libx11 shouldn't have to know and account for libx11's headers/pkg-config files referencing xorgproto. A libx11-devel package would depend on xorgproto. Since there's no separate -devel package, the dependency stays with the regular libx11 package. You already called (a) the pragmatic approach and it seems like a no-brainer at 1.5 MiB installed size. If we go with (c), it might be worthwhile to mass edit packages in /trunk to add the build dep. Doing that manually for hundreds of packages is a huge time sink and quite annoying during rebuilds.
On 21/12/19 7:31 pm, Evangelos Foutras via arch-dev-public wrote:
Downstream consumers of libx11 shouldn't have to know and account for libx11's headers/pkg-config files referencing xorgproto. A libx11-devel package would depend on xorgproto. Since there's no separate -devel package, the dependency stays with the regular libx11 package.
This matches my opinion on the matter. A
On Sat, Dec 21, 2019 at 12:18 PM Allan McRae via arch-dev-public < arch-dev-public@archlinux.org> wrote:
On 21/12/19 7:31 pm, Evangelos Foutras via arch-dev-public wrote:
Downstream consumers of libx11 shouldn't have to know and account for libx11's headers/pkg-config files referencing xorgproto. A libx11-devel package would depend on xorgproto. Since there's no separate -devel package, the dependency stays with the regular libx11 package.
This matches my opinion on the matter.
Ditto. (a) and (c) are the technically correct solutions, and I prefer (a). Unless we want to start a distro-wide effort of splitting out -devel packages, that is.
First of all, thanks for doing the cleanup! :) On 2019-12-21 09:41:46 (+0100), Andreas Radke via arch-dev-public wrote:
Please vote.
I'd go for b) as to me it seems the more correct approach (and doesn't require introducing further packages). Additionally, it is reflected in the package guidelines [1]. I think, that the overhead of changing the packages is of course a reality, but also part of being a packager and changes like these don't happen every day. As reference: I've done a similar thing for the lv2 [2] package (which is a makedepend for all lv2 plugins and hosts, but not required at runtime) and ladspa [3] (which is a makedepend for all ladspa plugins, but not required at runtime). In the specific case of lv2 similar rules apply, as we don't have -devel packages (which is great) for e.g. suil or sratom (that require lv2 to build other things). However, upstream is currently about to change the way host applications are supposed to be build by moving all the disjointed libraries into a new conglomerate library (and is en route to introduce breaking changes to lv2). I'd rather stay overly explicit and pull in dependencies where they are needed (e.g. makedepends), which makes it more transparent to change in case of upstream change, than to rely on transitive dependencies. Best, David P.S.: I'd also love to see the provided shared libraries be exposed in the provides array of libx11 (i.e. libX11.so and libX11-xcb.so) [4]. [1] https://wiki.archlinux.org/index.php/Arch_package_guidelines#Package_depende... [2] https://www.archlinux.org/packages/community/x86_64/lv2/ [3] https://www.archlinux.org/packages/extra/x86_64/ladspa/ [4] https://wiki.archlinux.org/index.php/Arch_package_guidelines#Package_relatio... -- https://sleepmap.de
On Sat, Dec 21, 2019 at 12:42 PM David Runge <dave@sleepmap.de> wrote:
I'd go for b) as to me it seems the more correct approach (and doesn't require introducing further packages). Additionally, it is reflected in the package guidelines [1].
Unfortunately I think the guideline isn't very clear on what it means. It is thus: Build system of program A wants libraries B and C. Library B depends on C. You should add both B and C to A's depends. But if the build system of program A wants only library B, you should not add C to A's depends. We now have many packages that want libx11 but say nothing about *proto, yet they now need xorgproto as a makedepend. Even worse, this extends further downstream, and packages building against GTK now also need a makedepend on xorgproto. I made a rough estimation and end up at ~3500 packages that are now broken. Grepping for 'Requires:.*\wproto' through my local pkg-config files, these packages directly reference a *proto and should have a dep on xorgproto: libice libx11 libxau libxaw libxcomposite libxcursor libxdamage libxdmcp libxext libxfixes libxfont2 libxft libxi libxinerama libxkbfile libxmu libxrandr libxrender libxres libxss libxt libxtst libxv libxxf86vm
On Sat, 21 Dec 2019 at 18:13, Jan Alexander Steffens via arch-dev-public <arch-dev-public@archlinux.org> wrote:
We now have many packages that want libx11 but say nothing about *proto, yet they now need xorgproto as a makedepend. Even worse, this extends further downstream, and packages building against GTK now also need a makedepend on xorgproto.
I also noticed tumbler was deciding to disable its poppler plugin at build time; adding xorgproto to makedepends fixed it. So this can cause silent change of build options too, in the case of auto-detected features. @Andreas: Can you go ahead and add xorgproto back to libx11? Better to have 1.5 MiB of headers installed than add seemingly unrelated xorgproto build dep to packages failing to build or have features suddenly going missing from packages.
Am Sat, 21 Dec 2019 19:47:39 +0200 schrieb Evangelos Foutras via arch-dev-public <arch-dev-public@archlinux.org>:
@Andreas: Can you go ahead and add xorgproto back to libx11? Better to have 1.5 MiB of headers installed than add seemingly unrelated xorgproto build dep to packages failing to build or have features suddenly going missing from packages.
Yes, I'm going to add it back. Sorry for the noise. -Andy
On December 21, 2019 9:41:46 AM GMT+01:00, Andreas Radke via arch-dev-public <arch-dev-public@archlinux.org> wrote:
With this move I've "fixed" libx11 no more depending at runtime on xorgproto package. I think no headers belong to an end user system and the libx11 library itself doesn't depend on it. But we also ship libx11-devel part inside the package and this indead depends on xorgproto headers. The libx11 .pc file clearly wants to have the headers installed. In the past it was enough to include libx11 to also pull in the proto headers at build time. This is now broken. Some devs call libx11 broken though only its -devel part is.
After some discussion on IRC these solution are possible:
a) revert to make libx11 depend again on xorgproto headers. This is the pragmatic way and would not need any further work. It just installs header files to the user system that aren't needed in any way there. So we did in the past and I don't really like it as it's not correct to me.
b) stay with changed libx11 and add xorgproto to packages that check for any of its headers. This needs to be done to an amount of ~300 packages when hitting build errors over the next time.
c) go an unusual way here and split libx11 into libx11, libx11-devel depending on xorgproto and maybe even libx11-xcb. This is the way distros go that support splitting libraries. It's probably the technical correct solution but will also require packages to makedepend on libx11-devel and save us no work.
Other distributions have chosen what they prefer. That a decision that needs to be done downstream.
I'd like to have either solution b) or c) in Arch to have a clear and more "transient" build time dependency. I guess it may help us also in the future when moving some day away from Xorg to its successor. But if majority wants solution a) back I'm fine reverting this change.
Please vote.
-Andy
I'm voting for b) or c). If there aren't really any further deps, personally to me it doesn't really matter too much to combine them, however for compile additions I prefer independence. btw we also have xorg-server-devel Cheers Levente
On 12/21/19 3:41 AM, Andreas Radke via arch-dev-public wrote:
With this move I've "fixed" libx11 no more depending at runtime on xorgproto package. I think no headers belong to an end user system and the libx11 library itself doesn't depend on it. But we also ship libx11-devel part inside the package and this indead depends on xorgproto headers. The libx11 .pc file clearly wants to have the headers installed. In the past it was enough to include libx11 to also pull in the proto headers at build time. This is now broken. Some devs call libx11 broken though only its -devel part is.
After some discussion on IRC these solution are possible:
a) revert to make libx11 depend again on xorgproto headers. This is the pragmatic way and would not need any further work. It just installs header files to the user system that aren't needed in any way there. So we did in the past and I don't really like it as it's not correct to me.
I'm not even sure I understand the question. The current state of affairs is that the libx11 package provides two things: - the libx11 client libraries - the libx11 development headers This is per standard Arch policy to not split headers into subpackages. Part of the feature functionality of the libx11 package is broken without xorgproto installed. *only* libx11 cares about xorgproto. What makes this "wrong"? The functionality that the package is intended to provide is indeed functionality that depends on xorgproto. It's not even merely pragmatic -- it is technically correct. ... People who are sincerely bothered by the installation of 1.5MB of headers should consider optionally adding a pacman.conf NoExtract rule to not install them; on my machine, it would save me 400MB, although personally I rather like headers since I tend to use them.
b) stay with changed libx11 and add xorgproto to packages that check for any of its headers. This needs to be done to an amount of ~300 packages when hitting build errors over the next time.
This is unambiguously wrong, unless those 300 packages actually check for xproto.pc or kbproto.pc, which seems doubtful. The fact that it requires teaching hundreds of packages far too much about libx11 internals that they don't actually depend on is pretty annoying too, yes, but I'd argue this solution is technically incorrect either way.
c) go an unusual way here and split libx11 into libx11, libx11-devel depending on xorgproto and maybe even libx11-xcb. This is the way distros go that support splitting libraries. It's probably the technical correct solution but will also require packages to makedepend on libx11-devel and save us no work.
Is it the technically correct solution for just libx11, or for all packages? IMO this only makes sense if we do it consistently. Personally, the fact that Arch does *not* do this is one of the things I consider a great advantage over confusing distros like Debian.
Other distributions have chosen what they prefer. That a decision that needs to be done downstream.
I'd like to have either solution b) or c) in Arch to have a clear and more "transient" build time dependency. I guess it may help us also in the future when moving some day away from Xorg to its successor. But if majority wants solution a) back I'm fine reverting this change.
Please vote.
-Andy
-- Eli Schwartz Bug Wrangler and Trusted User
On 12/19/19 at 04:19pm, Andreas Radke via arch-dev-public wrote:
I've created a bug[1] to follow a long overdue Xorg cleanup to drop legacy and dead code from our packages. This will require fixing makedepends in a 2nd step. I will do the most work over the next days/weeks. This all didn't fit well into a single ToDo list.
If you find more really dead Xorg packages add it please to the bug tracker with a link to its removal. Use the mailing list for further discussion if needed when you think a certain feature isn't widely used and should be dropped as well.
-Andy
I just noticed that a lot of packages are not building in our reproducible builds test CI. This is due to the removal of 'xf86dgaproto' and possibly other Xorg related packages. Can we get this fixed? [1] [2] I am all for removing dead packages but breaking building packages is not ok. Seems the Python 2 removal also made people break other packages which are makedepends of another package such as python-pytest-shutil. [3] Please don't do this, you are giving other maintainers extra work by removing their makedepends! [1] https://www.archlinux.org/devel/reports/non-existing-dependencies/ [2] https://tests.reproducible-builds.org/archlinux/state_DEPWAIT.html [3] https://www.archlinux.org/packages/community/any/python-pytest-shutil/ -- Jelle van der Waa
participants (10)
-
Allan McRae
-
Andreas Radke
-
Antonio Rojas
-
David Runge
-
Eli Schwartz
-
Evangelos Foutras
-
Giancarlo Razzolini
-
Jan Alexander Steffens
-
Jelle van der Waa
-
Levente Polyak