On Wed, 23 Feb 2022, Maarten de Vries via arch-general wrote:

On Wed, 23 Feb 2022 at 11:40, Knut Ahlers via arch-general < arch-general@lists.archlinux.org> wrote:

...

Hey there,

Through the latest update in glibc an issue / question emerged for me and maybe someone on this list has some input for me how to solve this:

We do have a repository with custom built packages (for example a custom version of ffmpeg we need for internal purposes)… These packages obviously do have dependencies on libraries and break when there is a change in a library. In this case: "version `GLIBC_2.34' not found".

Is there anyone who solved this already? How is it solved in the Archlinux core / community repo?

What is your trigger to rebuild packages and prevent running into some command not finding its dependencies? So how do you know when to rebuild [some binary package depending on glibc] without manually keeping track of the glibc version you've built the package against and then triggering the re-build when any of the underlying library changes its version?

How does the Archlinux community/core/extra build system keep track of this and knows when to rebuild ffmpeg because glibc changed?

Thanks in advance for any input!

Cheers, Knut

-- Knut Ahlers Software & Infrastructure Developer

Web & Blog: https://ahlers.me/ GPG-Key: 0xD215AD7D

Hey,

I'm not related to the Arch Linux project, but I manage a few custom repositories with additional packages for internal use. I've solved the problem you describe by mirroring the Arch Linux repositories, and building the custom packages against that mirror.

Then, about once a week, I run a script which updates the mirror in a staging location. The script checks the .BUILDINFO for all custom packages. That gives the version number of dependencies that were installed when the custom package was built. Those versions are checked against the versions in the synced Arch Linux repositories. If any of the runtime dependencies is updated, the package and all its reverse dependencies are flagged to be rebuild. Then, all flagged packages are rebuild in dependency order in a clean container (which is configured to use the staging repository).

When the build is done, the staging repository becomes the live mirror. This is done by overwriting a symlink to make it atomic. I also use btrfs snapshots to create a staging repository to avoid copying it every time for a rebuild.

Note that this approach rebuilds more often than necessary. You could further inspect library dependencies of ELF files if you want to reduce the amount of rebuilds. However, I prefer to err on the safe side in case ABI breakage happened without matching soname bump, at the cost of doing more rebuilds than necessary.

Maybe you can take some inspiration from this :)

Kind regards, Maarten de Vries

Hi Knut, I have a similar problem, which I solved a little different: I put as many versions as possible on my dependencies (i.e. depend on "foo=1.2.3" instead of just "foo") and use "libfoo.so=..." dependencies instead of "foo=..." wherever possible. This has the same disadvantage, that you may recompile more often than necessary, but it safes you from already installing broken packages (i.e. at latest, you notice during "pacman -Syu", that you should recompile - and not when you try to run your software). To automate the recompile, I use a similar approach as Maarten: I scrape the sync database of pacman, extract the provided versions and update them at the PKGBUILDs in my repo as necessary and recompile the packages. This runs much more often than weekly on my machine, though (I think, I set the timer to once an hour or so). regards, Erich