[arch-dev-public] Upcoming PHP 8.1 update
Hi all,
a little heads up as it takes longer than expected. I'll start the PHP 8.1 update and the required rebuilds soon. I noticed some unexpected incompatibilities and I'd like to look into these first. I'll also evaluate if we could drop the php7 packages at the same time.
Greetings,
Pierre
Hi,
On 06/12/2021 16:11, Pierre Schmitz via arch-dev-public wrote:
I'll also evaluate if we could drop the php7 packages at the same time
Just checked, mediawiki does not officially support PHP 8 yet and our bugtracker is still PHP 7.
[1] https://phabricator.wikimedia.org/T248925
Greetings,
Jelle van der Waa
On 2021-12-06 16:11:45 (+0100), Pierre Schmitz via arch-dev-public wrote:
Hi all,
a little heads up as it takes longer than expected. I'll start the PHP 8.1 update and the required rebuilds soon. I noticed some unexpected incompatibilities and I'd like to look into these first. I'll also evaluate if we could drop the php7 packages at the same time.
Hi Pierre,
nextcloud does not yet support php > 8.0. The upcoming version 23.0.0 (currently in [community-testing]) neither unfortunately [1].
However, the package itself guards against upgrading to php 8.1, so it should be fine for current installations, but it will make installing nextcloud not possible for new systems.
Best, David
On 12/6/21 17:48, David Runge via arch-dev-public wrote:
On 2021-12-06 16:11:45 (+0100), Pierre Schmitz via arch-dev-public wrote:
Hi all,
a little heads up as it takes longer than expected. I'll start the PHP 8.1 update and the required rebuilds soon. I noticed some unexpected incompatibilities and I'd like to look into these first. I'll also evaluate if we could drop the php7 packages at the same time.
Hi Pierre,
nextcloud does not yet support php > 8.0. The upcoming version 23.0.0 (currently in [community-testing]) neither unfortunately [1].
However, the package itself guards against upgrading to php 8.1, so it should be fine for current installations, but it will make installing nextcloud not possible for new systems.
That guard is just a stop gap from breaking the application after an upgrade. That does not really help current installations either as it cuts off the system from potentially important security updates.
We really need to keep the dependency resolution intact which means if we are stuck for nextcloud the only viable options were either postpone php or provide a maintained set of php 8.0 that nextcloud can depend on.
cheers, Levente
On 2021-12-06 17:58:57 (+0100), Levente Polyak via arch-dev-public wrote:
On 12/6/21 17:48, David Runge via arch-dev-public wrote:
On 2021-12-06 16:11:45 (+0100), Pierre Schmitz via arch-dev-public wrote:
Hi all,
a little heads up as it takes longer than expected. I'll start the PHP 8.1 update and the required rebuilds soon. I noticed some unexpected incompatibilities and I'd like to look into these first. I'll also evaluate if we could drop the php7 packages at the same time.
Hi Pierre,
nextcloud does not yet support php > 8.0. The upcoming version 23.0.0 (currently in [community-testing]) neither unfortunately [1].
However, the package itself guards against upgrading to php 8.1, so it should be fine for current installations, but it will make installing nextcloud not possible for new systems.
That guard is just a stop gap from breaking the application after an upgrade. That does not really help current installations either as it cuts off the system from potentially important security updates.
I agree. I was referring to right out breaking functionality on a running system due to the php upgrade.
We really need to keep the dependency resolution intact which means if we are stuck for nextcloud the only viable options were either postpone php or provide a maintained set of php 8.0 that nextcloud can depend on.
We indeed need to make an effort in regards to these situations. It has happened before that nextcloud was not yet ready for a minor version upgrade of php and the upgrade broke functionality for users. That's why we have made the effort to signal the upper boundaries for the applications as clearly as possible (to not be in this situation anymore).
My suggestion would be to wait for a little bit more to figure out whether there will be patch level releases for php 8.0.x or whether we should introduce another intermediary package (I'm not a fan of the latter).
Best, David
Thanks for the feedback. I'll do some more research (and also check some of my own apps). So far it does not seem that bad and most things work when disabling deprecation warnings and ignore dependency on the exact minor version of PHP.
MediaWiki works fine with PHP 8.0 (I haven't check 8.1 yet). I am also preparing an update to 1.37. I don't think the remaining issues regarding Elasticsearch affect us.
Flyspray (our Bug-Tracker) seems to be compatible with PHP 8.0 (according to its composer.json). So there is a chance it will work with 8.1 as well.
In general we could provide PHP 7 till its end of life in about eleven months. But I don't think its worth providing several different minor versions at the same time. This is not how semver is supposed to work :-) (Someone should check Nextcloud but looking at their PR it mostly seems about tests, documentation and deprecations but no hard errors).
Greetings,
Pierre
On Mon, Dec 6, 2021 at 6:13 PM David Runge via arch-dev-public arch-dev-public@lists.archlinux.org wrote:
On 2021-12-06 17:58:57 (+0100), Levente Polyak via arch-dev-public wrote:
On 12/6/21 17:48, David Runge via arch-dev-public wrote:
On 2021-12-06 16:11:45 (+0100), Pierre Schmitz via arch-dev-public wrote:
Hi all,
a little heads up as it takes longer than expected. I'll start the PHP 8.1 update and the required rebuilds soon. I noticed some unexpected incompatibilities and I'd like to look into these first. I'll also evaluate if we could drop the php7 packages at the same time.
Hi Pierre,
nextcloud does not yet support php > 8.0. The upcoming version 23.0.0 (currently in [community-testing]) neither unfortunately [1].
However, the package itself guards against upgrading to php 8.1, so it should be fine for current installations, but it will make installing nextcloud not possible for new systems.
That guard is just a stop gap from breaking the application after an upgrade. That does not really help current installations either as it cuts off the system from potentially important security updates.
I agree. I was referring to right out breaking functionality on a running system due to the php upgrade.
We really need to keep the dependency resolution intact which means if we are stuck for nextcloud the only viable options were either postpone php or provide a maintained set of php 8.0 that nextcloud can depend on.
We indeed need to make an effort in regards to these situations. It has happened before that nextcloud was not yet ready for a minor version upgrade of php and the upgrade broke functionality for users. That's why we have made the effort to signal the upper boundaries for the applications as clearly as possible (to not be in this situation anymore).
My suggestion would be to wait for a little bit more to figure out whether there will be patch level releases for php 8.0.x or whether we should introduce another intermediary package (I'm not a fan of the latter).
Best, David
On 2021-12-06 18:32:49 (+0100), Pierre Schmitz via arch-dev-public wrote:
In general we could provide PHP 7 till its end of life in about eleven months. But I don't think its worth providing several different minor versions at the same time.
I agree.
This is not how semver is supposed to work :-) (Someone should check Nextcloud but looking at their PR it mostly seems about tests, documentation and deprecations but no hard errors).
I would definitely wait until the PR for nextcloud [1] has settled (it's still a draft). The changes apply to 23.0.0 (which has been in testing only for a few hours). Do note, that effects of php 8.1.0 on the nextcloud-apps is completely unaccounted for in this scenario.
I would like to have nextcloud 23.0.0 in [community] before we proceed with any testing with php 8.1.0 if that is fine with you.
Best, David
So, its been some time and things seem to look a lot better now. I see the nextcloud patch was merged some time ago as well. We might need to aplly it manually as I don't see a patch release for some reason. Most things work fine with PHP 8.1. So far the issues are mostly arbitrary versions constraints or actual bugs that would be broken on 8.0 as well. (e.g. conflicting type hints in sub classes)
My plan is to prepare updating the php package to 8.1 and create a rebuild list. At the same time I'll remove the php7 package. It's been a year now and I would consider anything that only works on php7 as unmaintained. Here is a list of php7-packages that would be affected: * mediawiki: works fine with PHP 8 * drupal: should work on PHP 8 according to upstream * zabbix-frontend-php: unclear; I read that it might have been fixed in version 5.4 * phpvirtualbox: probably broken; latest release was in 2018. This should be remvoed fro mthe repos imho * web-news: brobably broken; latest release was in 2009. Also should be removed. * phppgadmin: probably broken; development seem slow or stalled on PHP 8 support. We might need to drop this as well * phpldapadmin: might work. New version was just released and claims to support anything newer than PHP 7.2 * dokuwiki: unclear; might need patches
Greetings,
Pierre
On Mon, Dec 6, 2021 at 6:53 PM David Runge dave@sleepmap.de wrote:
On 2021-12-06 18:32:49 (+0100), Pierre Schmitz via arch-dev-public wrote:
In general we could provide PHP 7 till its end of life in about eleven months. But I don't think its worth providing several different minor versions at the same time.
I agree.
This is not how semver is supposed to work :-) (Someone should check Nextcloud but looking at their PR it mostly seems about tests, documentation and deprecations but no hard errors).
I would definitely wait until the PR for nextcloud [1] has settled (it's still a draft). The changes apply to 23.0.0 (which has been in testing only for a few hours). Do note, that effects of php 8.1.0 on the nextcloud-apps is completely unaccounted for in this scenario.
I would like to have nextcloud 23.0.0 in [community] before we proceed with any testing with php 8.1.0 if that is fine with you.
Best, David
I have finished the rebuild of modules und pushed all packages into [testing]: . https://archlinux.org/todo/draft-php-81-module-rebuild/
Unfortunately there are two packages that had build issues: * libguestfs: No idea why it has a PHP module anyway. But this package does not compile with [extra] packages as well, so it is likely unrelated to PHP and probably cuased by an ocaml update. I used the repos archive from last month to build this package for now. * php-snuffleupagus: Does not pass tests with PHP 8.1 and there seems to be no upstream patches we could apply. (pkgstats usage is only 0.03%) I have disabled the failing tests for now.
Next is to check if there remain any incompatibilities aside from modules; e.g. the nextcloud version constrain we talked about. I might create a todo list as well.
Removing php7 and everything that depends on it will be done in a separate step once php 8.1 has moved into [extra].
Greetings,
Pierre
On Sun, Jan 2, 2022 at 2:00 PM Pierre Schmitz pierre@archlinux.de wrote:
So, its been some time and things seem to look a lot better now. I see the nextcloud patch was merged some time ago as well. We might need to aplly it manually as I don't see a patch release for some reason. Most things work fine with PHP 8.1. So far the issues are mostly arbitrary versions constraints or actual bugs that would be broken on 8.0 as well. (e.g. conflicting type hints in sub classes)
My plan is to prepare updating the php package to 8.1 and create a rebuild list. At the same time I'll remove the php7 package. It's been a year now and I would consider anything that only works on php7 as unmaintained. Here is a list of php7-packages that would be affected:
- mediawiki: works fine with PHP 8
- drupal: should work on PHP 8 according to upstream
- zabbix-frontend-php: unclear; I read that it might have been fixed
in version 5.4
- phpvirtualbox: probably broken; latest release was in 2018. This
should be remvoed fro mthe repos imho
- web-news: brobably broken; latest release was in 2009. Also should be removed.
- phppgadmin: probably broken; development seem slow or stalled on PHP
8 support. We might need to drop this as well
- phpldapadmin: might work. New version was just released and claims
to support anything newer than PHP 7.2
- dokuwiki: unclear; might need patches
Greetings,
Pierre
On Mon, Dec 6, 2021 at 6:53 PM David Runge dave@sleepmap.de wrote:
On 2021-12-06 18:32:49 (+0100), Pierre Schmitz via arch-dev-public wrote:
In general we could provide PHP 7 till its end of life in about eleven months. But I don't think its worth providing several different minor versions at the same time.
I agree.
This is not how semver is supposed to work :-) (Someone should check Nextcloud but looking at their PR it mostly seems about tests, documentation and deprecations but no hard errors).
I would definitely wait until the PR for nextcloud [1] has settled (it's still a draft). The changes apply to 23.0.0 (which has been in testing only for a few hours). Do note, that effects of php 8.1.0 on the nextcloud-apps is completely unaccounted for in this scenario.
I would like to have nextcloud 23.0.0 in [community] before we proceed with any testing with php 8.1.0 if that is fine with you.
Best, David
-- Pierre Schmitz, https://pierre-schmitz.com
So far everything looks fine. I am even using PHP 8.1 in production for a week now, without any issues. So I'd like to move the packages to [extra].
The remaining blocker is nextcloud though. What is the best option to resolve this? 1) Remove the version constraint 2) same as above + try to apply the upstream patch: https://github.com/nextcloud/server/pull/29862 3) temporary depend on php7; I'd like to remove that package soon though 4) anything else?
Greetings,
Pierre
On Sat, Jan 8, 2022 at 11:09 AM Pierre Schmitz pierre@archlinux.de wrote:
I have finished the rebuild of modules und pushed all packages into [testing]: . https://archlinux.org/todo/draft-php-81-module-rebuild/
Unfortunately there are two packages that had build issues:
- libguestfs: No idea why it has a PHP module anyway. But this package
does not compile with [extra] packages as well, so it is likely unrelated to PHP and probably cuased by an ocaml update. I used the repos archive from last month to build this package for now.
- php-snuffleupagus: Does not pass tests with PHP 8.1 and there seems
to be no upstream patches we could apply. (pkgstats usage is only 0.03%) I have disabled the failing tests for now.
Next is to check if there remain any incompatibilities aside from modules; e.g. the nextcloud version constrain we talked about. I might create a todo list as well.
Removing php7 and everything that depends on it will be done in a separate step once php 8.1 has moved into [extra].
Greetings,
Pierre
On Sun, Jan 2, 2022 at 2:00 PM Pierre Schmitz pierre@archlinux.de wrote:
So, its been some time and things seem to look a lot better now. I see the nextcloud patch was merged some time ago as well. We might need to aplly it manually as I don't see a patch release for some reason. Most things work fine with PHP 8.1. So far the issues are mostly arbitrary versions constraints or actual bugs that would be broken on 8.0 as well. (e.g. conflicting type hints in sub classes)
My plan is to prepare updating the php package to 8.1 and create a rebuild list. At the same time I'll remove the php7 package. It's been a year now and I would consider anything that only works on php7 as unmaintained. Here is a list of php7-packages that would be affected:
- mediawiki: works fine with PHP 8
- drupal: should work on PHP 8 according to upstream
- zabbix-frontend-php: unclear; I read that it might have been fixed
in version 5.4
- phpvirtualbox: probably broken; latest release was in 2018. This
should be remvoed fro mthe repos imho
- web-news: brobably broken; latest release was in 2009. Also should be removed.
- phppgadmin: probably broken; development seem slow or stalled on PHP
8 support. We might need to drop this as well
- phpldapadmin: might work. New version was just released and claims
to support anything newer than PHP 7.2
- dokuwiki: unclear; might need patches
Greetings,
Pierre
On Mon, Dec 6, 2021 at 6:53 PM David Runge dave@sleepmap.de wrote:
On 2021-12-06 18:32:49 (+0100), Pierre Schmitz via arch-dev-public wrote:
In general we could provide PHP 7 till its end of life in about eleven months. But I don't think its worth providing several different minor versions at the same time.
I agree.
This is not how semver is supposed to work :-) (Someone should check Nextcloud but looking at their PR it mostly seems about tests, documentation and deprecations but no hard errors).
I would definitely wait until the PR for nextcloud [1] has settled (it's still a draft). The changes apply to 23.0.0 (which has been in testing only for a few hours). Do note, that effects of php 8.1.0 on the nextcloud-apps is completely unaccounted for in this scenario.
I would like to have nextcloud 23.0.0 in [community] before we proceed with any testing with php 8.1.0 if that is fine with you.
Best, David
-- Pierre Schmitz, https://pierre-schmitz.com
-- Pierre Schmitz, https://pierre-schmitz.com
On 2022-01-16 16:40:00 (+0100), Pierre Schmitz via arch-dev-public wrote:
So far everything looks fine. I am even using PHP 8.1 in production for a week now, without any issues. So I'd like to move the packages to [extra].
The remaining blocker is nextcloud though. What is the best option to resolve this?
- Remove the version constraint
- same as above + try to apply the upstream patch:
I'll look into 1 or 2 asap. Sorry, I have been pre-occupied with other packages/work.
Best, David
Thanks! Let me know if I may support you. Wrangling with PHP scripts is at least more fun than using gpg...
On Sun, Jan 16, 2022 at 5:03 PM David Runge dave@sleepmap.de wrote:
On 2022-01-16 16:40:00 (+0100), Pierre Schmitz via arch-dev-public wrote:
So far everything looks fine. I am even using PHP 8.1 in production for a week now, without any issues. So I'd like to move the packages to [extra].
The remaining blocker is nextcloud though. What is the best option to resolve this?
- Remove the version constraint
- same as above + try to apply the upstream patch:
I'll look into 1 or 2 asap. Sorry, I have been pre-occupied with other packages/work.
Best, David
On 2022-01-16 17:11:44 (+0100), Pierre Schmitz via arch-dev-public wrote:
Thanks! Let me know if I may support you.
It went quite okay (but I had to remove parts of the patch series as they apply to files that we do have in our tarball).
nextcloud 23.0.0-2 is now in [community-testing].
As upstream hasn't released a new version yet and all apps that I maintain have not yet had a release either, I guess it would be good to wait at least a few more days :S
Wrangling with PHP scripts is at least more fun than using gpg...
That might be true :D I can fully recommend using sq though! :)
Best, David
Thank you.
I would not expect a lot of updates specific to PHP 8.1. Most apps should work fine. I also do not expect a lot of feedback from testing users.
There should be a 8.1.2 release this week and I plan to move that one to extra.
Greetings,
Pierre
On Sun, Jan 16, 2022 at 6:02 PM David Runge dave@sleepmap.de wrote:
On 2022-01-16 17:11:44 (+0100), Pierre Schmitz via arch-dev-public wrote:
Thanks! Let me know if I may support you.
It went quite okay (but I had to remove parts of the patch series as they apply to files that we do have in our tarball).
nextcloud 23.0.0-2 is now in [community-testing].
As upstream hasn't released a new version yet and all apps that I maintain have not yet had a release either, I guess it would be good to wait at least a few more days :S
Wrangling with PHP scripts is at least more fun than using gpg...
That might be true :D I can fully recommend using sq though! :)
Best, David
I just released PHP 8.1 into [extra].
Please also check https://archlinux.org/todo/php-7-retiredment/ so PHP 7 can be dropped soon.
have a nice weekend,
Pierre
On Mon, Jan 17, 2022 at 3:38 PM Pierre Schmitz pierre@archlinux.de wrote:
Thank you.
I would not expect a lot of updates specific to PHP 8.1. Most apps should work fine. I also do not expect a lot of feedback from testing users.
There should be a 8.1.2 release this week and I plan to move that one to extra.
Greetings,
Pierre
On Sun, Jan 16, 2022 at 6:02 PM David Runge dave@sleepmap.de wrote:
On 2022-01-16 17:11:44 (+0100), Pierre Schmitz via arch-dev-public wrote:
Thanks! Let me know if I may support you.
It went quite okay (but I had to remove parts of the patch series as they apply to files that we do have in our tarball).
nextcloud 23.0.0-2 is now in [community-testing].
As upstream hasn't released a new version yet and all apps that I maintain have not yet had a release either, I guess it would be good to wait at least a few more days :S
Wrangling with PHP scripts is at least more fun than using gpg...
That might be true :D I can fully recommend using sq though! :)
Best, David
-- Pierre Schmitz, https://pierre-schmitz.com
On 2022-01-21 17:51:17 (+0100), Pierre Schmitz via arch-dev-public wrote:
I just released PHP 8.1 into [extra].
Please also check https://archlinux.org/todo/php-7-retiredment/ so PHP 7 can be dropped soon.
have a nice weekend,
Unfortunately, it likely won't be ;_;
https://bugs.archlinux.org/task/73452
Another reminder for users to *please* join the Testing Team [1] and help test our packages!
Best, David
Hi
On Sat, Jan 22, 2022 at 5:00 AM David Runge via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:
On 2022-01-21 17:51:17 (+0100), Pierre Schmitz via arch-dev-public wrote:
I just released PHP 8.1 into [extra].
Please also check https://archlinux.org/todo/php-7-retiredment/ so PHP 7 can be dropped soon.
have a nice weekend,
Unfortunately, it likely won't be ;_;
And there is another php8.1 bugreport https://bugs.archlinux.org/task/73451
Another reminder for users to *please* join the Testing Team [1] and help test our packages!
Best, David
Hi David,
sorry about the hassle. I did not expect much issues here. I would consider this one of the smoother PHP updates. Unless people ignored warnings by previous PHP versions. I guess that is what mostly happend here. PHP 8 gets more and more strict each version.
After reading the issues on Nextcloud's Github repository I guess we can conclude that they will probably lack behind at least one PHP minor version. This is quite incompatible with the Arch way.
Anyway, let's talk about some options to solve such issues:
1) Let's no longer package software that requires older versions of PHP. Personally I would run such complex software with very specific needs in a Docker container. E.g. Nextcloud even provides an official one. 2) Keep trying to patch upstream packages to keep them working. 3) We provide two sets of PHP packages: "php" would always be the latest stable version and be released no matter what. In addition to this there would be e.g. "php-legacy" packages providing the oldest supported version, currently 7.4. This would be updated to 8.0 in November when 7 is EOL and php-8.2 get's released. The difference to the currently available php7 package will be the lack of a version number in package and binary names. So both packages will be a moving target, but always two versions apart.
I would give option 3 a try. I'd like to get rid of versioned constraints then and reduce the amount of third party modules. While we would end up with more packages we need less testing and will be able to move faster.
*) https://www.php.net/supported-versions.php
On Sat, Jan 22, 2022 at 10:57 AM David Runge dave@sleepmap.de wrote:
On 2022-01-21 17:51:17 (+0100), Pierre Schmitz via arch-dev-public wrote:
I just released PHP 8.1 into [extra].
Please also check https://archlinux.org/todo/php-7-retiredment/ so PHP 7 can be dropped soon.
have a nice weekend,
Unfortunately, it likely won't be ;_;
https://bugs.archlinux.org/task/73452
Another reminder for users to *please* join the Testing Team [1] and help test our packages!
Best, David
Hi Pierre,
On 2022-01-22 20:45:45 (+0100), Pierre Schmitz via arch-dev-public wrote:
sorry about the hassle. I did not expect much issues here. I would consider this one of the smoother PHP updates. Unless people ignored warnings by previous PHP versions. I guess that is what mostly happend here. PHP 8 gets more and more strict each version.
So far there has been no further reaction on the ticket, so I hope there is not much more breakage. :)
Some transitions take a bit of time.
After reading the issues on Nextcloud's Github repository I guess we can conclude that they will probably lack behind at least one PHP minor version. This is quite incompatible with the Arch way.
Anyway, let's talk about some options to solve such issues:
- Let's no longer package software that requires older versions of
PHP. Personally I would run such complex software with very specific needs in a Docker container. E.g. Nextcloud even provides an official one.
This means giving up on system packaging and I do not agree with the mentality of hiding away all legacy in a container. That's neither beneficial for us nor for the upstream projects or for the diversity (in regards to how to use and run a piece of software) of the ecosystem in general.
This also means that running and installing this software in an Arch Linux based container would be not supported and/or flaky as well, which I consider a detriment to the distribution :-/ To spin this wheel a bit further: What is the point of having a php package, if it could also be provided by an upstream container?
- Keep trying to patch upstream packages to keep them working.
That's what we usually do and that is also what keeps momentum in certain upstream projects.
FWIW, that's what we do for the entire python ecosystem for better or worse and more specifically for e.g. python-django, which has a few dependents that can not easily be upgraded from one minor version to the next.
This sometimes means getting involved with upstreams, providing patches, using patches, etc. which is part of the packager role that we are in.
However, the problem we are currently facing is that of upstream frameworks and languages being unable to provide sufficient compatibility guarantees.
For this we need to make a greater effort to have testers and more clearly defined (automated) test scenarios. If we can not make this effort to some (even small) extent, I'm wondering why we are packaging languages at all anymore.
- We provide two sets of PHP packages: "php" would always be the
latest stable version and be released no matter what. In addition to this there would be e.g. "php-legacy" packages providing the oldest supported version, currently 7.4. This would be updated to 8.0 in November when 7 is EOL and php-8.2 get's released. The difference to the currently available php7 package will be the lack of a version number in package and binary names. So both packages will be a moving target, but always two versions apart.
Wouldn't this mean that those php versions are mutually exclusive? I would be more in favor of moving towards a unified php version again, else we might end up with a lua or java situation eventually.
I would give option 3 a try. I'd like to get rid of versioned constraints then and reduce the amount of third party modules. While we would end up with more packages we need less testing and will be able to move faster.
I don't think the option to "move faster" is a good one if there is nothing that is outright compatible anymore ;-) What would be the target of an up-to-date php, that has no packages using it? With a two version system (that is mutually exclusive?) we would have the same issues with incompatible projects as soon as we move from one version to the other, just at a different point in time.
We have version constraints to guard against these major/minor version upgrade scenarios and to be able to have an overview as to what is not yet compatible and then act based on that information in a testing environment. I do not believe abolishing that just so that we can upgrade and break everything without even knowing about it makes a lot of sense, but maybe I did not understand your intentions correctly.
Best, David
On Sat, Jan 22, 2022 at 9:45 PM David Runge dave@sleepmap.de wrote:
On 2022-01-22 20:45:45 (+0100), Pierre Schmitz via arch-dev-public wrote: ...
- Let's no longer package software that requires older versions of
PHP. Personally I would run such complex software with very specific needs in a Docker container. E.g. Nextcloud even provides an official one.
This means giving up on system packaging and I do not agree with the mentality of hiding away all legacy in a container. That's neither beneficial for us nor for the upstream projects or for the diversity (in regards to how to use and run a piece of software) of the ecosystem in general.
This also means that running and installing this software in an Arch Linux based container would be not supported and/or flaky as well, which I consider a detriment to the distribution :-/ To spin this wheel a bit further: What is the point of having a php package, if it could also be provided by an upstream container?
I saw this more as an option for rare exceptions and not the default. This is great if you cannot run your software at all otherwise. This is also not a real option as users can do this today already. :-)
- Keep trying to patch upstream packages to keep them working.
That's what we usually do and that is also what keeps momentum in certain upstream projects.
FWIW, that's what we do for the entire python ecosystem for better or worse and more specifically for e.g. python-django, which has a few dependents that can not easily be upgraded from one minor version to the next.
This sometimes means getting involved with upstreams, providing patches, using patches, etc. which is part of the packager role that we are in.
However, the problem we are currently facing is that of upstream frameworks and languages being unable to provide sufficient compatibility guarantees.
For this we need to make a greater effort to have testers and more clearly defined (automated) test scenarios. If we can not make this effort to some (even small) extent, I'm wondering why we are packaging languages at all anymore.
I agree this should be the way to go. Not sure if I get your right here, but (extensive) testing is best done by upstream projects though. And a lot of them even test with release candidates so their releases even work on day one. I usually provide release candidates for Arch's PHP packages as well. I could make that more public for maintainers that want to test as early as possible. We might also ask packagers to run basic unit tests during packaging.
- We provide two sets of PHP packages: "php" would always be the
latest stable version and be released no matter what. In addition to this there would be e.g. "php-legacy" packages providing the oldest supported version, currently 7.4. This would be updated to 8.0 in November when 7 is EOL and php-8.2 get's released. The difference to the currently available php7 package will be the lack of a version number in package and binary names. So both packages will be a moving target, but always two versions apart.
Wouldn't this mean that those php versions are mutually exclusive? I would be more in favor of moving towards a unified php version again, else we might end up with a lua or java situation eventually.
They would be installable and usable at the same time. In the same we as you may currently use PHP 7 and 8, which was originally meant to be a temporary solution. There is a catch though: The user has to alter the configuration when he wants to switch versions.
I would give option 3 a try. I'd like to get rid of versioned constraints then and reduce the amount of third party modules. While we would end up with more packages we need less testing and will be able to move faster.
I don't think the option to "move faster" is a good one if there is nothing that is outright compatible anymore ;-) What would be the target of an up-to-date php, that has no packages using it?
This is likely exaggerated. Adaption of new versions is quite fast these days, especially minor ones. Most of the frameworks, packages and libraries had compatible release before the first patch release. But of course this heavily depends on what you work with. And here is the whole point of the issue: On one hand people want or need the latest packages and on the other hand some projects might need more time.
With a two version system (that is mutually exclusive?) we would have the same issues with incompatible projects as soon as we move from one version to the other, just at a different point in time.
This might be true and as we have some packages with dead upstream projects this is even likely. The only option would be to remove such packages from our repositories then. I wouldn't want to provide PHP packages that do not receive any upstream security fixes anymore. This might sound harsher as it actually is as this would mean packages did not manage to work with PHP versions released two years ago.
We have version constraints to guard against these major/minor version upgrade scenarios and to be able to have an overview as to what is not yet compatible and then act based on that information in a testing environment. I do not believe abolishing that just so that we can upgrade and break everything without even knowing about it makes a lot of sense, but maybe I did not understand your intentions correctly.
I would hope this would break less as we would depend such packages to PHP versions that are actually supported by the upstream projects. As an example: As we now know that Nextcloud usually needs some more time we would suggest to install php-legacy. Packages like WordPress or phpMyAdmin could depend on the latest php packages as these are usually compatible.
Greetings,
Pierre
Hi again,
after waiting another couple of weeks, the situation with nextcloud unfortunately has still not improved. We see issues with utf-8 compatibility [1] and meanwhile the version 24.0.0 which is supposed to provide native support for php 8.1 is being delayed until end of April [2].
This all being what it is, I believe the sanest (but also a bit complicated to maneuvre) way forward is to downgrade its dependencies to php7, as otherwise we will be stuck even longer with broken/disfunctional setups.
I agree this should be the way to go. Not sure if I get your right here, but (extensive) testing is best done by upstream projects though. And a lot of them even test with release candidates so their releases even work on day one. I usually provide release candidates for Arch's PHP packages as well. I could make that more public for maintainers that want to test as early as possible. We might also ask packagers to run basic unit tests during packaging.
In our packaging use-case it already fails in the most basic ways when looking at nextcloud and the nextcloud apps. Some apps at this point are still not or have up to recently not been compatible with php 8.1 (e.g. nextcloud-app-calendar [1], nextcloud-app-notes [2] and some apps in the store, e.g. providing TOTP), rendering nextcloud disfunctional. We need more check() implementations in the app PKGBUILDs that also test for the php requirement (those are encoded in the apps) and fail when trying to build against a php version that is not supported. As an example: For the app compatibility towards nextcloud (server) we are automatically creating lower and upper version boundaries by now and this usually works out fine and provides a much safer upgrade scenario than before. The same needs to happen for the apps, as they will fail to load outside of their support window.
They would be installable and usable at the same time. In the same we as you may currently use PHP 7 and 8, which was originally meant to be a temporary solution. There is a catch though: The user has to alter the configuration when he wants to switch versions.
Could you elaborate a bit further on what you mean by that? How would namespacing of that "older" php version look like specifically? How and why would this differ from the way we currently provide e.g. php7 (and potentially a php80 or similar)?
This is likely exaggerated. Adaption of new versions is quite fast these days, especially minor ones. Most of the frameworks, packages and libraries had compatible release before the first patch release. But of course this heavily depends on what you work with. And here is the whole point of the issue: On one hand people want or need the latest packages and on the other hand some projects might need more time.
The version constraints are implemented directly towards the language version the application is run with and would mean a good safe-guard against breakage in packaging and expectation. As it stands we would need to add a provides php=$version to the "older" php package (this also needs to be done for the current php7), which would allow more or less seamless upgrades/downgrades based upon the requirements of a given project. By default projects would always use the latest php, if not declaring version requirements. An upgrade to a given project, introducing an updated php version requirement, would automatically pull in the required php* package.
I'm still a bit unsure about how to deal with this in the context of php modules though. For those we would still need to specifically touch a given PKGBUILD to change a dependency (e.g. php-gd vs. php7-gd) unless we come up with a way to abstract this with a common virtual provides.
This might be true and as we have some packages with dead upstream projects this is even likely. The only option would be to remove such packages from our repositories then. I wouldn't want to provide PHP packages that do not receive any upstream security fixes anymore. This might sound harsher as it actually is as this would mean packages did not manage to work with PHP versions released two years ago.
Projects that fall out of the window of what we can support should indeed be removed.
I would hope this would break less as we would depend such packages to PHP versions that are actually supported by the upstream projects. As an example: As we now know that Nextcloud usually needs some more time we would suggest to install php-legacy. Packages like WordPress or phpMyAdmin could depend on the latest php packages as these are usually compatible.
I would try to handle this entirely with version constraints in the given leaf packages, similar to how we do this with java environments.
Best, David
[1] https://github.com/nextcloud/server/issues/31212 [2] https://github.com/nextcloud/server/wiki/Maintenance-and-Release-Schedule [3] https://bugs.archlinux.org/task/74016 [4] https://bugs.archlinux.org/task/73930
On 2022-03-08 13:56:04 (+0100), David Runge via arch-dev-public wrote:
after waiting another couple of weeks, the situation with nextcloud unfortunately has still not improved. We see issues with utf-8 compatibility [1] and meanwhile the version 24.0.0 which is supposed to provide native support for php 8.1 is being delayed until end of April [2].
This all being what it is, I believe the sanest (but also a bit complicated to maneuvre) way forward is to downgrade its dependencies to php7, as otherwise we will be stuck even longer with broken/disfunctional setups.
Meanwhile more time has passed and there has not been any progress on this. For nextcloud 23.0.4 the compatibility patches do not apply cleanly anymore.
Therefore I will implement a specific version check for it to check its own php compatibility and fail otherwise, while removing the patches.
The version constraints are implemented directly towards the language version the application is run with and would mean a good safe-guard against breakage in packaging and expectation. As it stands we would need to add a provides php=$version to the "older" php package (this also needs to be done for the current php7), which would allow more or less seamless upgrades/downgrades based upon the requirements of a given project. By default projects would always use the latest php, if not declaring version requirements. An upgrade to a given project, introducing an updated php version requirement, would automatically pull in the required php* package.
I'm still a bit unsure about how to deal with this in the context of php modules though. For those we would still need to specifically touch a given PKGBUILD to change a dependency (e.g. php-gd vs. php7-gd) unless we come up with a way to abstract this with a common virtual provides.
I will introduce this for php7 in [testing] now and build nextcloud 23.0.4 against it.
This works for the interpreter, but it's more involved when looking at the remaining interpreter-specific modules.
It would make sense to implement an abstraction for each that pulls in the correct package. E.g. for php7-apcu add a `provides=(php-apcu-interpreter=7)` and for php-apcu add a `provides=(php-apcu-interpreter=8)`. That way consumers can decide which interpreter they require by depending on e.g. `php-apcu-interpreter=8`. This works similar to how we do things for java.
Do you have any thoughts on this?
Best, David
I'd still suggest to provide two different php versions as mentioned some time ago: the current "php" and "php-legacy" which will always be the oldest supported version. These may provide the versions as you suggested: php-legacy provides php=7.4 (and will be updated to 8.0 soon)
The benefit of not encoding the versions within the package name is easier maintenance and user wont have to manually update their config and systemd files.
The "-interpreter" suffix is required to make it work with pacman or could php-legacy-apcu provide php-apcu=7.4?
On a side node: PHP 7 is on its way out of support and wont work with e.g. OpenSSL 3 :-) So we have to update to 8.0 in a few month at latest.
Greetings,
Pierre
On Sun, Apr 24, 2022 at 6:57 PM David Runge dave@sleepmap.de wrote:
On 2022-03-08 13:56:04 (+0100), David Runge via arch-dev-public wrote:
after waiting another couple of weeks, the situation with nextcloud unfortunately has still not improved. We see issues with utf-8 compatibility [1] and meanwhile the version 24.0.0 which is supposed to provide native support for php 8.1 is being delayed until end of April [2].
This all being what it is, I believe the sanest (but also a bit complicated to maneuvre) way forward is to downgrade its dependencies to php7, as otherwise we will be stuck even longer with broken/disfunctional setups.
Meanwhile more time has passed and there has not been any progress on this. For nextcloud 23.0.4 the compatibility patches do not apply cleanly anymore.
Therefore I will implement a specific version check for it to check its own php compatibility and fail otherwise, while removing the patches.
The version constraints are implemented directly towards the language version the application is run with and would mean a good safe-guard against breakage in packaging and expectation. As it stands we would need to add a provides php=$version to the "older" php package (this also needs to be done for the current php7), which would allow more or less seamless upgrades/downgrades based upon the requirements of a given project. By default projects would always use the latest php, if not declaring version requirements. An upgrade to a given project, introducing an updated php version requirement, would automatically pull in the required php* package.
I'm still a bit unsure about how to deal with this in the context of php modules though. For those we would still need to specifically touch a given PKGBUILD to change a dependency (e.g. php-gd vs. php7-gd) unless we come up with a way to abstract this with a common virtual provides.
I will introduce this for php7 in [testing] now and build nextcloud 23.0.4 against it.
This works for the interpreter, but it's more involved when looking at the remaining interpreter-specific modules.
It would make sense to implement an abstraction for each that pulls in the correct package. E.g. for php7-apcu add a `provides=(php-apcu-interpreter=7)` and for php-apcu add a `provides=(php-apcu-interpreter=8)`. That way consumers can decide which interpreter they require by depending on e.g. `php-apcu-interpreter=8`. This works similar to how we do things for java.
Do you have any thoughts on this?
Best, David
On 4/24/22 19:08, Pierre Schmitz via arch-dev-public wrote:
I'd still suggest to provide two different php versions as mentioned some time ago: the current "php" and "php-legacy" which will always be the oldest supported version. These may provide the versions as you suggested: php-legacy provides php=7.4 (and will be updated to 8.0 soon)
But this means that in case of nextcloud once you update php-legacy to provide php=8.0 (when the next version comes out) that you then either can't update php-legacy until nextcloud works with 8.0 or you break the php dependency to nextcloud which essentially breaks nextcloud.
On a side note I have nextcloud running on my server and for example currently TOTP is not working as the plugin doesn't install against php 8
Best regards
Segaja
On 2022-04-24 19:08:37 (+0200), Pierre Schmitz via arch-dev-public wrote:
I'd still suggest to provide two different php versions as mentioned some time ago: the current "php" and "php-legacy" which will always be the oldest supported version. These may provide the versions as you suggested: php-legacy provides php=7.4 (and will be updated to 8.0 soon)
What would be the name of the executable provided by the old php package? If it stayed the same that could be nice (less things to change).
I'd probably change the package name from php-legacy to something else (e.g. php-old or php-lts).
The benefit of not encoding the versions within the package name is easier maintenance and user wont have to manually update their config and systemd files.
Do you mean this also in regards to executable naming? Because that would indeed be more stable then (although less specific of course). It could prove useful to provide compat symlinks.
The "-interpreter" suffix is required to make it work with pacman or could php-legacy-apcu provide php-apcu=7.4?
The "-interpreter" suffix for the package provides would only be relevant for packages depending on e.g. a specific interpreter version. That way e.g. an optional dependency on php-apcu-interpreter<8.1 could be used by nextcloud to depend on a php-apcu provider which supports a php interpreter < 8.1 (in this case php7-apcu would match).
The only downside to this scenario is, that we need to rebuild all php module packages when updating minor versions of php (e.g. from 8.0 -> 8.1) so that the respective packages are rebuilt and pickup the correct version they now provide.
On a side node: PHP 7 is on its way out of support and wont work with e.g. OpenSSL 3 :-) So we have to update to 8.0 in a few month at latest.
That's okay and I hope nextcloud is updated by then to support it.
Best, David
On 2022-04-24 21:13:43 (+0200), David Runge via arch-dev-public wrote:
On 2022-04-24 19:08:37 (+0200), Pierre Schmitz via arch-dev-public wrote:
I'd still suggest to provide two different php versions as mentioned some time ago: the current "php" and "php-legacy" which will always be the oldest supported version. These may provide the versions as you suggested: php-legacy provides php=7.4 (and will be updated to 8.0 soon)
What would be the name of the executable provided by the old php package? If it stayed the same that could be nice (less things to change).
I'd probably change the package name from php-legacy to something else (e.g. php-old or php-lts).
The benefit of not encoding the versions within the package name is easier maintenance and user wont have to manually update their config and systemd files.
Do you mean this also in regards to executable naming? Because that would indeed be more stable then (although less specific of course). It could prove useful to provide compat symlinks.
The "-interpreter" suffix is required to make it work with pacman or could php-legacy-apcu provide php-apcu=7.4?
The "-interpreter" suffix for the package provides would only be relevant for packages depending on e.g. a specific interpreter version. That way e.g. an optional dependency on php-apcu-interpreter<8.1 could be used by nextcloud to depend on a php-apcu provider which supports a php interpreter < 8.1 (in this case php7-apcu would match).
The only downside to this scenario is, that we need to rebuild all php module packages when updating minor versions of php (e.g. from 8.0 -> 8.1) so that the respective packages are rebuilt and pickup the correct version they now provide.
One further downside of the "-interpreter" approach is, that it is terrible in optdepends, as we can't specify a range (e.g. foo-interpreter>=7.4 and foo-interpreter<8.1) in one optdepends entry :S
Other than that, it might provide better stability going forward, but we would still need to ensure to rebuild the entire lot when updating minor versions of the interpreter (which I think is still an okay outcome).
And a note on the versioned provides (e.g. `php-apcu=7.4` for the current php7-apcu). I believe this should be done, but it would only prove useful if packagers then relied on two dependency contraints (I'm choosing a different module, because php-apcu has always the same version as the interpreter): Depend on e.g. php-igbinary>3.2 and php-igbinary-interpreter=7.4 (this way php7-igbinary, when following the current naming scheme, would be selected).
Once again, rebuilds **have to** be done on minor version interpreter updates, else this does not work.
What do you think?
Best, David
On 2022-04-25 17:25:47 (+0200), David Runge via arch-dev-public wrote:
And a note on the versioned provides (e.g. `php-apcu=7.4` for the current php7-apcu). I believe this should be done, but it would only prove useful if packagers then relied on two dependency contraints (I'm choosing a different module, because php-apcu has always the same version as the interpreter): Depend on e.g. php-igbinary>3.2 and php-igbinary-interpreter=7.4 (this way php7-igbinary, when following the current naming scheme, would be selected).
Once again, rebuilds **have to** be done on minor version interpreter updates, else this does not work.
Addendum to the addendum...
So, it turns out that pacman does not resolve versioned dependencies the way I expected it to [1]. As php is a real package, it is resolved directly and a version constraint does not lead to a user selection. This can be observed when php7 provides php=7.4.29 and nextcloud depends on php>=7.4.29 and php<8.1. The former is resolved as php (without offered selection) while the latter is resolved as php7.
According to Andrew Gregory it would work to add a virtual provide (e.g. php-interpreter) which is added to both php (php-interpreter=8.1.5) and php7 (php-interpreter=7.4.29) and that packages may then use to distinguish versions with.
I'll try this out in [testing] and with nextcloud further in [community-staging] tomorrow to see whether this is workable.
Best, David
On Sat, Jan 22, 2022 at 20:45:45 +0100, Pierre Schmitz via arch-dev-public wrote:
Hi David,
sorry about the hassle. I did not expect much issues here. I would consider this one of the smoother PHP updates. Unless people ignored warnings by previous PHP versions. I guess that is what mostly happend here. PHP 8 gets more and more strict each version.
After reading the issues on Nextcloud's Github repository I guess we can conclude that they will probably lack behind at least one PHP minor version. This is quite incompatible with the Arch way.
Anyway, let's talk about some options to solve such issues:
- Let's no longer package software that requires older versions of
PHP. Personally I would run such complex software with very specific needs in a Docker container. E.g. Nextcloud even provides an official one.
FYI: As stated earlier by Jelle[1], our bugtracker isn't currently compatible with PHP8. In case PHP7 is dropped from the repos and no one steps up fixing the flyspray code[1]. The DevOps team will likely just run the the bugtracker in a container of some sort.
[1] https://lists.archlinux.org/pipermail/arch-dev-public/2021-December/030571.h... [2] https://gitlab.archlinux.org/archlinux/flyspray/
- Kristian
- Keep trying to patch upstream packages to keep them working.
- We provide two sets of PHP packages: "php" would always be the
latest stable version and be released no matter what. In addition to this there would be e.g. "php-legacy" packages providing the oldest supported version, currently 7.4. This would be updated to 8.0 in November when 7 is EOL and php-8.2 get's released. The difference to the currently available php7 package will be the lack of a version number in package and binary names. So both packages will be a moving target, but always two versions apart.
I would give option 3 a try. I'd like to get rid of versioned constraints then and reduce the amount of third party modules. While we would end up with more packages we need less testing and will be able to move faster.
*) https://www.php.net/supported-versions.php
On Sat, Jan 22, 2022 at 10:57 AM David Runge dave@sleepmap.de wrote:
On 2022-01-21 17:51:17 (+0100), Pierre Schmitz via arch-dev-public wrote:
I just released PHP 8.1 into [extra].
Please also check https://archlinux.org/todo/php-7-retiredment/ so PHP 7 can be dropped soon.
have a nice weekend,
Unfortunately, it likely won't be ;_;
https://bugs.archlinux.org/task/73452
Another reminder for users to *please* join the Testing Team [1] and help test our packages!
Best, David
-- Pierre Schmitz, https://pierre-schmitz.com
Hi Kristian,
I would not recommend to deploy PHP 7 after its end of life in November this year. In theory someone might be able to backport security fixes though.
In this case I would urge to update Flyspray to its latest version which has support for PHP 8.0: https://github.com/Flyspray/flyspray/releases We seem to use version 0.9.9.7 which we updated 9 years ago. From there on its probably easier to make to work with recent PHP versions.
Greetings,
Pierre
On Sun, Jan 23, 2022 at 3:15 PM Kristian Klausen kristian@klausen.dk wrote:
On Sat, Jan 22, 2022 at 20:45:45 +0100, Pierre Schmitz via arch-dev-public wrote:
Hi David,
sorry about the hassle. I did not expect much issues here. I would consider this one of the smoother PHP updates. Unless people ignored warnings by previous PHP versions. I guess that is what mostly happend here. PHP 8 gets more and more strict each version.
After reading the issues on Nextcloud's Github repository I guess we can conclude that they will probably lack behind at least one PHP minor version. This is quite incompatible with the Arch way.
Anyway, let's talk about some options to solve such issues:
- Let's no longer package software that requires older versions of
PHP. Personally I would run such complex software with very specific needs in a Docker container. E.g. Nextcloud even provides an official one.
FYI: As stated earlier by Jelle[1], our bugtracker isn't currently compatible with PHP8. In case PHP7 is dropped from the repos and no one steps up fixing the flyspray code[1]. The DevOps team will likely just run the the bugtracker in a container of some sort.
[1] https://lists.archlinux.org/pipermail/arch-dev-public/2021-December/030571.h... [2] https://gitlab.archlinux.org/archlinux/flyspray/
- Kristian
- Keep trying to patch upstream packages to keep them working.
- We provide two sets of PHP packages: "php" would always be the
latest stable version and be released no matter what. In addition to this there would be e.g. "php-legacy" packages providing the oldest supported version, currently 7.4. This would be updated to 8.0 in November when 7 is EOL and php-8.2 get's released. The difference to the currently available php7 package will be the lack of a version number in package and binary names. So both packages will be a moving target, but always two versions apart.
I would give option 3 a try. I'd like to get rid of versioned constraints then and reduce the amount of third party modules. While we would end up with more packages we need less testing and will be able to move faster.
*) https://www.php.net/supported-versions.php
On Sat, Jan 22, 2022 at 10:57 AM David Runge dave@sleepmap.de wrote:
On 2022-01-21 17:51:17 (+0100), Pierre Schmitz via arch-dev-public wrote:
I just released PHP 8.1 into [extra].
Please also check https://archlinux.org/todo/php-7-retiredment/ so PHP 7 can be dropped soon.
have a nice weekend,
Unfortunately, it likely won't be ;_;
https://bugs.archlinux.org/task/73452
Another reminder for users to *please* join the Testing Team [1] and help test our packages!
Best, David
-- Pierre Schmitz, https://pierre-schmitz.com
participants (7)
-
Anatol Pomozov
-
Andreas 'Segaja' Schleifer
-
David Runge
-
Jelle van der Waa
-
Kristian Klausen
-
Levente Polyak
-
Pierre Schmitz