Re: [arch-dev-public] openssl 3.0
Hi Jelle, (also forwarding to dev-public) definitely yes, OpenSSL 3.0 is on my wish list! :-) I did not want to jump on it at day one though. Even the last minor updates were quite painful and we still have packages requiring version 1.0 and are still not compatible with 1.1. While they claim that most packages should work with a recompile, it would be nice to actually know which packages are not compatible. This should help whether we need another compatibility package are would be able to just replace openssl 1.1 with version 3. I know about foutrelis' awesome rebuilder script, but I wonder if we have something similar that I just could run for half a day to get an idea which package would break and which wont? Like a dry run that wont commit anything. If no such thing exists yet, I might have a look myself. Greetings, Pierre On Wed, Nov 3, 2021 at 9:14 PM Jelle van der Waa <jelle@vdwaa.nl> wrote:
Hi Pierre,
Shall we start an openssl 3.0 rebuild soon? Fedora/Debian/Alpine seens to have already started.
https://fedoraproject.org/wiki/Changes/OpenSSL3.0
Greetings,
Jelle
-- Pierre Schmitz, https://pierre-schmitz.com
just a small update: This is going to be a little more complicated and I suggest we tackle this at the beginning of next year. I got some very helpful feedback from our community (Thanks a lot loqs). * We might be able to drop version 1.0 (which is no longer maintained by upstream anyway). packages that only work with 1.0 should be dropped imho. * We are going to need to provide 1.1 for a couple of packages (hopefully not for long) * We are going to have to solve the bootstrap issue with pacman. I guess by either linking it statically, make it depend on the 1.1 package at first Greetings, Pierre On Sat, Nov 6, 2021 at 10:32 AM Pierre Schmitz <pierre@archlinux.de> wrote:
Hi Jelle, (also forwarding to dev-public)
definitely yes, OpenSSL 3.0 is on my wish list! :-)
I did not want to jump on it at day one though. Even the last minor updates were quite painful and we still have packages requiring version 1.0 and are still not compatible with 1.1.
While they claim that most packages should work with a recompile, it would be nice to actually know which packages are not compatible. This should help whether we need another compatibility package are would be able to just replace openssl 1.1 with version 3.
I know about foutrelis' awesome rebuilder script, but I wonder if we have something similar that I just could run for half a day to get an idea which package would break and which wont? Like a dry run that wont commit anything. If no such thing exists yet, I might have a look myself.
Greetings,
Pierre
On Wed, Nov 3, 2021 at 9:14 PM Jelle van der Waa <jelle@vdwaa.nl> wrote:
Hi Pierre,
Shall we start an openssl 3.0 rebuild soon? Fedora/Debian/Alpine seens to have already started.
https://fedoraproject.org/wiki/Changes/OpenSSL3.0
Greetings,
Jelle
-- Pierre Schmitz, https://pierre-schmitz.com
-- Pierre Schmitz, https://pierre-schmitz.com
a follow up: * Retiring OpenSSL 1.0 will take place here: https://archlinux.org/todo/openssl-10-retirement/ This wont affect the 1.1 -> 3.0 transition though. * I have placed an openssl-1.1 package into [staging] that should make it easier to migrate as it provides the 1.1 version of libcrypto.so and libssl.so * The idea was to have openssl-3.0 depend on that at first to make the transition more seamless. I still need to solve the bootstrap issue though. As this is going to be a massive rebuild we should plan a time frame when to do this and avoid any other rebuilds. ATM there are more than 700 packages in our staging repos. - Pierre On Mon, Dec 6, 2021 at 6:41 PM Pierre Schmitz <pierre@archlinux.de> wrote:
just a small update: This is going to be a little more complicated and I suggest we tackle this at the beginning of next year. I got some very helpful feedback from our community (Thanks a lot loqs). * We might be able to drop version 1.0 (which is no longer maintained by upstream anyway). packages that only work with 1.0 should be dropped imho. * We are going to need to provide 1.1 for a couple of packages (hopefully not for long) * We are going to have to solve the bootstrap issue with pacman. I guess by either linking it statically, make it depend on the 1.1 package at first
Greetings,
Pierre
On Sat, Nov 6, 2021 at 10:32 AM Pierre Schmitz <pierre@archlinux.de> wrote:
Hi Jelle, (also forwarding to dev-public)
definitely yes, OpenSSL 3.0 is on my wish list! :-)
I did not want to jump on it at day one though. Even the last minor updates were quite painful and we still have packages requiring version 1.0 and are still not compatible with 1.1.
While they claim that most packages should work with a recompile, it would be nice to actually know which packages are not compatible. This should help whether we need another compatibility package are would be able to just replace openssl 1.1 with version 3.
I know about foutrelis' awesome rebuilder script, but I wonder if we have something similar that I just could run for half a day to get an idea which package would break and which wont? Like a dry run that wont commit anything. If no such thing exists yet, I might have a look myself.
Greetings,
Pierre
On Wed, Nov 3, 2021 at 9:14 PM Jelle van der Waa <jelle@vdwaa.nl> wrote:
Hi Pierre,
Shall we start an openssl 3.0 rebuild soon? Fedora/Debian/Alpine seens to have already started.
https://fedoraproject.org/wiki/Changes/OpenSSL3.0
Greetings,
Jelle
-- Pierre Schmitz, https://pierre-schmitz.com
-- Pierre Schmitz, https://pierre-schmitz.com
-- Pierre Schmitz, https://pierre-schmitz.com
Hi all, I have prepared a openssl-3.0 and 1.1 packages with the bootstrapped dependencies. In addition to this there is a hopefully complete todo list: https://archlinux.org/todo/openssl-30/ containing about 500 packages. Next steps: 1) Let's agree on a time window where no other rebuild can take place within our staging repos. How about at least the first two weeks in February? 2) I guess we have to at least build the core and toolchain packages manually. (*) Hopefully we may automate everything else. If you like to take a look: [openssl] Server = https://repo.pierre-schmitz.com/$repo/os/$arch Important: Only use this to check building packages within a chroot. Installing this on a system will break it. *) libarchive already fails \o/; but hopefully this unit test can be ignored: https://github.com/libarchive/libarchive/issues/1596 On Sat, Jan 8, 2022 at 10:24 PM Pierre Schmitz <pierre@archlinux.de> wrote:
a follow up:
* Retiring OpenSSL 1.0 will take place here: https://archlinux.org/todo/openssl-10-retirement/ This wont affect the 1.1 -> 3.0 transition though. * I have placed an openssl-1.1 package into [staging] that should make it easier to migrate as it provides the 1.1 version of libcrypto.so and libssl.so * The idea was to have openssl-3.0 depend on that at first to make the transition more seamless. I still need to solve the bootstrap issue though.
As this is going to be a massive rebuild we should plan a time frame when to do this and avoid any other rebuilds. ATM there are more than 700 packages in our staging repos.
- Pierre
On Mon, Dec 6, 2021 at 6:41 PM Pierre Schmitz <pierre@archlinux.de> wrote:
just a small update: This is going to be a little more complicated and I suggest we tackle this at the beginning of next year. I got some very helpful feedback from our community (Thanks a lot loqs). * We might be able to drop version 1.0 (which is no longer maintained by upstream anyway). packages that only work with 1.0 should be dropped imho. * We are going to need to provide 1.1 for a couple of packages (hopefully not for long) * We are going to have to solve the bootstrap issue with pacman. I guess by either linking it statically, make it depend on the 1.1 package at first
Greetings,
Pierre
On Sat, Nov 6, 2021 at 10:32 AM Pierre Schmitz <pierre@archlinux.de> wrote:
Hi Jelle, (also forwarding to dev-public)
definitely yes, OpenSSL 3.0 is on my wish list! :-)
I did not want to jump on it at day one though. Even the last minor updates were quite painful and we still have packages requiring version 1.0 and are still not compatible with 1.1.
While they claim that most packages should work with a recompile, it would be nice to actually know which packages are not compatible. This should help whether we need another compatibility package are would be able to just replace openssl 1.1 with version 3.
I know about foutrelis' awesome rebuilder script, but I wonder if we have something similar that I just could run for half a day to get an idea which package would break and which wont? Like a dry run that wont commit anything. If no such thing exists yet, I might have a look myself.
Greetings,
Pierre
On Wed, Nov 3, 2021 at 9:14 PM Jelle van der Waa <jelle@vdwaa.nl> wrote:
Hi Pierre,
Shall we start an openssl 3.0 rebuild soon? Fedora/Debian/Alpine seens to have already started.
https://fedoraproject.org/wiki/Changes/OpenSSL3.0
Greetings,
Jelle
-- Pierre Schmitz, https://pierre-schmitz.com
-- Pierre Schmitz, https://pierre-schmitz.com
-- Pierre Schmitz, https://pierre-schmitz.com
-- Pierre Schmitz, https://pierre-schmitz.com
Pierre Schmitz via arch-dev-public <arch-dev-public@lists.archlinux.org> on Sun, 2022/01/23 12:50:
Next steps: 1) Let's agree on a time window where no other rebuild can take place within our staging repos. How about at least the first two weeks in February?
I guess the ffmpeg 5.0 will be blocking for some time... -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
On Thu, 2022-01-27 at 16:47 +0100, Christian Hesse via arch-dev-public wrote:
Pierre Schmitz via arch-dev-public <arch-dev-public@lists.archlinux.org> on Sun, 2022/01/23 12:50:
Next steps: 1) Let's agree on a time window where no other rebuild can take place within our staging repos. How about at least the first two weeks in February?
I guess the ffmpeg 5.0 will be blocking for some time...
Not necessarily. There are too many packages that don't build, I will maintain a temporary ffmpeg4.4 package to get this todo out quickly. Should have time for this over the weekend. Cheers, -- Maxime
So I just build the 437 packages (pkgbase) and let my computer compile for just 25 hours. The initial results can be seen here https://md.archlinux.org/s/t8HOyhNOi Currently there are 27 packages in [core]/[extra] and 92 in [community] that do not build. I did not check the logs for every package yet, but I guess there are these categories: * package does not build regardless of OpenSSL (e.g. unavailable sources, checksum mismatch, issues due to LTO, ...) * packages also links to a not yet update package that still uses openssl-1.1 * packages are actually incompatible with OpenSSL 3.0 I'll need some help with: * Document why a package fails (complete build logs are attached to the document linked above) * Create a todo list for packages that are broken for other reasons and fix them * Review the legacy openssl-1.1 package and check if this approach is valid. (last time we patched versiond symbols in 1.0 which I did not apply here) See https://github.com/archlinux/svntogit-packages/tree/packages/openssl-1.1/tru... and https://github.com/archlinux/svntogit-packages/tree/packages/openssl-1.0/tru... * Fix the incompatible packages and as a last resort link to 1.1 PS: if there is a tool that is able to rebuild and install packages in the correct order (not by explicit dependency but by so lib links), let me know. Greetings, Pierre On Fri, Jan 28, 2022 at 9:41 AM Maxime Gauduin via arch-dev-public <arch-dev-public@lists.archlinux.org> wrote:
On Thu, 2022-01-27 at 16:47 +0100, Christian Hesse via arch-dev-public wrote:
Pierre Schmitz via arch-dev-public <arch-dev-public@lists.archlinux.org> on Sun, 2022/01/23 12:50:
Next steps: 1) Let's agree on a time window where no other rebuild can take place within our staging repos. How about at least the first two weeks in February?
I guess the ffmpeg 5.0 will be blocking for some time...
Not necessarily. There are too many packages that don't build, I will maintain a temporary ffmpeg4.4 package to get this todo out quickly. Should have time for this over the weekend.
Cheers, -- Maxime
-- Pierre Schmitz, https://pierre-schmitz.com
What's the status? Can we start the actual move and rebuilds? There should be enough work done by other distributions to fix major issues. We'are already late at that party. -Andy
On 9/20/22 11:23, Andreas Radke wrote:
What's the status? Can we start the actual move and rebuilds? There should be enough work done by other distributions to fix major issues. We'are already late at that party.
Yes. According to https://github.com/loqs/PACKAGES-OSSL3 all packages are either okay or fixable at this point. As there's no huge rebuilds now taking place in staging repos, I think we should push the new packages to [staging] and start the work now. @Pierre I think we have a buildbot infrastructure as well as foutrelis' rebuilder which is capable of handling most of the dependency ordering issues. It would also be great if you would join IRC for better coordination. -- Regards, Felix Yan
Pierre Schmitz via arch-dev-public <arch-dev-public@lists.archlinux.org> on Sun, 2022/01/23 12:50:
Next steps: 1) Let's agree on a time window where no other rebuild can take place within our staging repos. How about at least the first two weeks in February?
The todo list has been around for too long already. Any news on this? -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
Hi Christian, there were some delays due to other rebuilds and lack of time/other issues. So far I did not get any feedback, so I'd like to repeat my request for help. * If someone with more C knowledge could review the openssl-1.1 package that would be great * To all maintainers: Please have a look at https://md.archlinux.org/s/t8HOyhNOi and check if your package is listed as "failed to build". It would be great to add a note about the Openssl 3.0 support status (see notes I added for some core packages for example) Greetings, Pierre On Tue, Mar 15, 2022 at 11:21 AM Christian Hesse <list@eworm.de> wrote:
Pierre Schmitz via arch-dev-public <arch-dev-public@lists.archlinux.org> on Sun, 2022/01/23 12:50:
Next steps: 1) Let's agree on a time window where no other rebuild can take place within our staging repos. How about at least the first two weeks in February?
The todo list has been around for too long already. Any news on this? -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
-- Pierre Schmitz, https://pierre-schmitz.com
Am Mon, 21 Mar 2022 18:52:14 +0100 schrieb Pierre Schmitz via arch-dev-public <arch-dev-public@lists.archlinux.org>:
Hi Christian,
there were some delays due to other rebuilds and lack of time/other issues.
So far I did not get any feedback, so I'd like to repeat my request for help. * If someone with more C knowledge could review the openssl-1.1 package that would be great * To all maintainers: Please have a look at https://md.archlinux.org/s/t8HOyhNOi and check if your package is listed as "failed to build". It would be great to add a note about the Openssl 3.0 support status (see notes I added for some core packages for example)
Greetings,
Pierre
Are you aware of this list? https://github.com/loqs/PACKAGES-OSSL3 -Andy
On Sat, Jan 08, 2022 at 10:24:34PM +0100, Pierre Schmitz via arch-dev-public wrote:
* Retiring OpenSSL 1.0 will take place here: https://archlinux.org/todo/openssl-10-retirement/ This wont affect the 1.1 -> 3.0 transition though.
I have now completed this todo and removed openssl-1.0 from the [core] repository :) I'll wait a few days before removing it from svn in-case something would require us to put it back into the repos. -- Morten Linderud PGP: 9C02FF419FECBE16
Hi Pierre, We were discussing on IRC about starting the OpenSSL 3.0 rebuild. For bootstrapping, we could make the openssl package depend on openssl-1.1 while building the following: - coreutils - curl - kmod - krb5 - libarchive - libevent - libssh2 - pacman - sudo - systemd After these are linked to OpenSSL 3.0 libraries, we'll drop the openssl-1.1 dep from openssl and re-rebuild them to remove the openssl-1.1 reference from .PKGINFO. If the above approach seems good, please commit the updated PKGBUILD to svn. We'll then start the rebuilds on [1] and see how they go. [1] https://rebuilds.foutrelis.com/?all
Hey, On 25/10/2022 11:15, Evangelos Foutras wrote:
Hi Pierre,
We were discussing on IRC about starting the OpenSSL 3.0 rebuild. For bootstrapping, we could make the openssl package depend on openssl-1.1 while building the following:
- coreutils - curl - kmod - krb5 - libarchive - libevent - libssh2 - pacman - sudo - systemd
After these are linked to OpenSSL 3.0 libraries, we'll drop the openssl-1.1 dep from openssl and re-rebuild them to remove the openssl-1.1 reference from .PKGINFO.
If the above approach seems good, please commit the updated PKGBUILD to svn. We'll then start the rebuilds on [1] and see how they go.
Sounds good, we might need openssl-1.1 in the end for something.. as we still have openssl-1.0 :(
On Tue, 25 Oct 2022 at 12:15, Evangelos Foutras <evangelos@foutrelis.com> wrote:
If the above approach seems good, please commit the updated PKGBUILD to svn. We'll then start the rebuilds on [1] and see how they go.
Slight change of plans, Jan is going to push GNOME 43 first. We should be able to do OpenSSL 3 right after that.
On 10/25/22 17:49, Evangelos Foutras wrote:
On Tue, 25 Oct 2022 at 12:15, Evangelos Foutras <evangelos@foutrelis.com> wrote:
If the above approach seems good, please commit the updated PKGBUILD to svn. We'll then start the rebuilds on [1] and see how they go.
Slight change of plans, Jan is going to push GNOME 43 first. We should be able to do OpenSSL 3 right after that.
That sounds good, we should either way most likely chill until 1st of November while not already having OpenSSL 3 in the repos yet: https://mta.openssl.org/pipermail/openssl-announce/2022-October/000238.html Cheers, Levente
On Tue, 25 Oct 2022 at 20:03, Levente Polyak <anthraxx@archlinux.org> wrote:
https://mta.openssl.org/pipermail/openssl-announce/2022-October/000238.html
Started the rebuilds with 3.0.5, let's not forget to bump to 3.0.7 before moving to testing!
Thanks a lot for pushing this forward! I updated to 3.0.7 in staging. I had libprovides in my branch. Do you guys think this might be handy to define versioned dependencies when we have potentially three different openssl verions to maintain? provides=('libcrypto.so' 'libssl.so') At the same time 1.1.1s was released which mainly fixes a regression from 1.1.1r (we are on 1.1.q). Do you guys think it is worth to release that 1.1.1 update in the meantime? Greetings, Pierre On Tue, Nov 1, 2022 at 12:19 PM Evangelos Foutras <evangelos@foutrelis.com> wrote:
On Tue, 25 Oct 2022 at 20:03, Levente Polyak <anthraxx@archlinux.org> wrote:
https://mta.openssl.org/pipermail/openssl-announce/2022-October/000238.html
Started the rebuilds with 3.0.5, let's not forget to bump to 3.0.7 before moving to testing!
-- Pierre Schmitz, https://pierre-schmitz.com
On 2022-11-01 18:27:23 (+0100), Pierre Schmitz wrote:
I updated to 3.0.7 in staging. I had libprovides in my branch. Do you guys think this might be handy to define versioned dependencies when we have potentially three different openssl verions to maintain?
provides=('libcrypto.so' 'libssl.so')
Always a fan! Please add! :)
At the same time 1.1.1s was released which mainly fixes a regression from 1.1.1r (we are on 1.1.q). Do you guys think it is worth to release that 1.1.1 update in the meantime?
If it is not critical, maybe better to let the openssl 3 rebuild roll through. Best, David -- https://sleepmap.de
participants (10)
-
Andreas Radke
-
Christian Hesse
-
David Runge
-
Evangelos Foutras
-
Felix Yan
-
Jelle van der Waa
-
Levente Polyak
-
Maxime Gauduin
-
Morten Linderud
-
Pierre Schmitz