[PATCH] Replace libdepends/libprovides
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like: LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32') At the end of package building, makepkg will look in the library directories and add a provide. E.g. for pacman: provide = lib:libalpm.so.13 Note the prefix matches the prefix given to the relevant directory in LIB_DIRS. Similarly, makepkg can add dependencies on libraries. E.g. pacman may have: depends = lib:libgpgme.so.11 Note, to help with bootstrapping this system, or if packages just do not want to add libprovides, the depends entries are only added if a package actually provides them. This is different to the APK system for libraries which uses "so" as the prefix and is not configurable. But Alpine used musl, which has no concept of multilib, so we need to be a bit more flexible. The dependency/provides addotions can all be disabled in pacman.conf with the '!autodeps' option. Note that APK has similar things for binaries and pkg-config files. e.g. provides = cmd:pacman provides = pc:libalpm These can readily be added as dropins to libmakepkg. Allan McRae (4): makepkg: remove libdepends and libprovides libmakepkg: add framework for autodeps libmakepkg: automatically add library sonames to provides libmakepkg: automatically add library dependencies doc/PKGBUILD.5.asciidoc | 9 -- doc/makepkg.conf.5.asciidoc | 11 ++ etc/makepkg.conf.in | 8 +- scripts/libmakepkg/autodep.sh.in | 38 ++++++ .../libmakepkg/autodep/library_depends.sh.in | 75 +++++++++++ .../libmakepkg/autodep/library_provides.sh.in | 56 +++++++++ scripts/libmakepkg/autodep/meson.build | 18 +++ scripts/libmakepkg/meson.build | 1 + scripts/makepkg.sh.in | 119 +----------------- 9 files changed, 206 insertions(+), 129 deletions(-) create mode 100644 scripts/libmakepkg/autodep.sh.in create mode 100644 scripts/libmakepkg/autodep/library_depends.sh.in create mode 100644 scripts/libmakepkg/autodep/library_provides.sh.in create mode 100644 scripts/libmakepkg/autodep/meson.build -- 2.34.1
This will be replaced by a better system
Signed-off-by: Allan McRae
Signed-off-by: Allan McRae
When the option "autodeps" is enabled, makepkg will add provides
entries for libraries found in the directories specified in LIB_DIRS
in makepkg.conf. The entries LIB_DIRS array have the format
"prefix:directory". For example, the entry "lib:usr/lib" will search
$pkgdir/usr/lib for library sonames and add "lib:libfoo.so.1" to the
provides array.
Signed-off-by: Allan McRae
Add linked libraries to a packages dependency list. This is the partner
to automatically generated library provides, and thus depends take the
same format. To help with bootstrapping, library dependencies are only
added if the relevant provide exists.
Signed-off-by: Allan McRae
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
At the end of package building, makepkg will look in the library directories and add a provide. E.g. for pacman:
provide = lib:libalpm.so.13
If I understand correctly, we create a mapping pretty-name:actual-path in the makepkg.conf and use the pretty-name in PKGBUILD. Do we add the pretty-name or actual-path in .PKGINFO? Does pacman print these autodeps ... even if only for informational purposes? What is expected to happen if someone typos the existing mappings - say "lib32:usr/lib23"? Or re-defines the standard ones "lib:my/special/path"? Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
Note the prefix matches the prefix given to the relevant directory in LIB_DIRS. Similarly, makepkg can add dependencies on libraries. E.g. pacman may have:
depends = lib:libgpgme.so.11
Note, to help with bootstrapping this system, or if packages just do not want to add libprovides, the depends entries are only added if a package actually provides them.
This is different to the APK system for libraries which uses "so" as the prefix and is not configurable. But Alpine used musl, which has no concept of multilib, so we need to be a bit more flexible.
I think the above questions might be the reason why Alpine chose to keep them hard-coded. In particular, while this approach provides flexibility, it leaves things open to various mistakes and misuse. Off the top of my head, we can enhance makepkg to catch most of those issues. For example - we can use the actual-path in .PKGINFO, explicitly check for mistakes/changes in the default mappings - lib, lib32... Although at this point one might as well have the defaults hardcoded in makepkg?
The dependency/provides addotions can all be disabled in pacman.conf with the '!autodeps' option.
s/addotions/additions/;s/pacman.conf/makepkg.conf/ typos? At a glance we can disable autodeps in the PKGBUILD itself, right? HTH Emil
On 14/12/21 20:40, Emil Velikov wrote:
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae
wrote: This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
Not really... This is the path for adding library dependencies & provides. If other autodeps get added, they may need their own configuration option.
At the end of package building, makepkg will look in the library directories and add a provide. E.g. for pacman:
provide = lib:libalpm.so.13
If I understand correctly, we create a mapping pretty-name:actual-path in the makepkg.conf and use the pretty-name in PKGBUILD.
Do we add the pretty-name or actual-path in .PKGINFO?
That is the line from the .PKGINFO provide = <tag>:<soname> Does pacman
print these autodeps ... even if only for informational purposes?
They are treated the same as any provide or depend. For example, using -Qi on my pacman-git package currently shows: Provides : pacman=6.0.1 lib:libalpm.so.13 Depends On : archlinux-keyring bash curl gpgme libarchive pacman-mirrorlist lib:libcurl.so.4
What is expected to happen if someone typos the existing mappings - say "lib32:usr/lib23"? Or re-defines the standard ones "lib:my/special/path"?
It their makepkg.conf has "lib32:usr/lib23" it is very likely that no dependencies/provides will be added.
Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
As above, nothing will happen. The usr/lib23 directory will (I guess) never occur in a package, so never be searched for files with an soname. I see this as being the same as any other configuration typo. e.g. if MAN_DIRS pointed to non-existent places, no man pages would be compressed.
Note the prefix matches the prefix given to the relevant directory in LIB_DIRS. Similarly, makepkg can add dependencies on libraries. E.g. pacman may have:
depends = lib:libgpgme.so.11
Note, to help with bootstrapping this system, or if packages just do not want to add libprovides, the depends entries are only added if a package actually provides them.
This is different to the APK system for libraries which uses "so" as the prefix and is not configurable. But Alpine used musl, which has no concept of multilib, so we need to be a bit more flexible.
I think the above questions might be the reason why Alpine chose to keep them hard-coded. In particular, while this approach provides flexibility, it leaves things open to various mistakes and misuse.
MAN_DIRS, DOC_DIRS, PURGE_TARGETS, ... all could have typos in their configuration and not work.
Off the top of my head, we can enhance makepkg to catch most of those issues. For example - we can use the actual-path in .PKGINFO, explicitly check for mistakes/changes in the default mappings - lib, lib32... Although at this point one might as well have the defaults hardcoded in makepkg?
This needs to be configurable. e.g. some distros will have: LIB_DIRS=('so:lib' 'so:usr/lib') when they have not done a /usr merge. Note, I selected the "so" prefix as being clearer for that example. I'm thinking "so:usr/lib" and "so32:usr/lib32" might be better for the configuration example.
The dependency/provides addotions can all be disabled in pacman.conf with the '!autodeps' option.
s/addotions/additions/;s/pacman.conf/makepkg.conf/ typos?
Yes - I meant the OPTIONS array in makepkg.conf.
At a glance we can disable autodeps in the PKGBUILD itself, right?
Yes - it is can be specified in options=(). Thanks for the comments. Allan
On Tue, 14 Dec 2021 at 11:28, Allan McRae
On 14/12/21 20:40, Emil Velikov wrote:
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae
wrote: This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
Not really... This is the path for adding library dependencies & provides. If other autodeps get added, they may need their own configuration option.
This sounds great. [snip]
Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
As above, nothing will happen. The usr/lib23 directory will (I guess) never occur in a package, so never be searched for files with an soname.
We might want to have a simple check in makepkg, to high-light those. If PKGBUILD has "provides" to a non-existant file, we could error out IMHO. This will catch both typos on the packaging side as well as buggy upstream - say they dropped/renamed the library, or a particular configure combination no longer builds one of the dozen+ DSOs. This is not a blocking suggestion, but I can see it as quite beneficial. [snip]
Although at this point one might as well have the defaults hardcoded in makepkg?
This needs to be configurable. e.g. some distros will have:
LIB_DIRS=('so:lib' 'so:usr/lib')
when they have not done a /usr merge.
Agreed. When I said hard-coded I was thinking of a default, which can be overwritten during build time. Say something like: meson -D lib_dirs="lib:lib64/, lib32:lib/" ... Thanks for elevating my concerns o/ -Emil
On 14/12/21 22:28, Emil Velikov wrote:
On Tue, 14 Dec 2021 at 11:28, Allan McRae
wrote: On 14/12/21 20:40, Emil Velikov wrote:
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae
wrote: This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
Not really... This is the path for adding library dependencies & provides. If other autodeps get added, they may need their own configuration option.
This sounds great.
[snip]
Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
As above, nothing will happen. The usr/lib23 directory will (I guess) never occur in a package, so never be searched for files with an soname.
We might want to have a simple check in makepkg, to high-light those. If PKGBUILD has "provides" to a non-existant file, we could error out IMHO.
This will catch both typos on the packaging side as well as buggy upstream - say they dropped/renamed the library, or a particular configure combination no longer builds one of the dozen+ DSOs.
I don't understand this suggestion. There would be no non-existent provide, as makepkg would not add one unless it finds one in the configured path. This is a complete change from the current system where the name of the library needed to be in the PKGBUILD. Now makepkg will automatically detect and add these values. And we can't distinguish between a package that has no libraries to be provided and a makepkg.conf with a typo pointing to the wrong directory.
This is not a blocking suggestion, but I can see it as quite beneficial.
[snip]
Although at this point one might as well have the defaults hardcoded in makepkg?
This needs to be configurable. e.g. some distros will have:
LIB_DIRS=('so:lib' 'so:usr/lib')
when they have not done a /usr merge.
Agreed. When I said hard-coded I was thinking of a default, which can be overwritten during build time. Say something like: meson -D lib_dirs="lib:lib64/, lib32:lib/" ...
I'm going to suggest the default is not to have this feature running at all. So blank is a good default! Or course each distro will provide a suitable makepkg.conf with their opinionated configuration. A
On Tue, 14 Dec 2021 at 12:38, Allan McRae
On 14/12/21 22:28, Emil Velikov wrote:
On Tue, 14 Dec 2021 at 11:28, Allan McRae
wrote: On 14/12/21 20:40, Emil Velikov wrote:
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae
wrote: This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
Not really... This is the path for adding library dependencies & provides. If other autodeps get added, they may need their own configuration option.
This sounds great.
[snip]
Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
As above, nothing will happen. The usr/lib23 directory will (I guess) never occur in a package, so never be searched for files with an soname.
We might want to have a simple check in makepkg, to high-light those. If PKGBUILD has "provides" to a non-existant file, we could error out IMHO.
This will catch both typos on the packaging side as well as buggy upstream - say they dropped/renamed the library, or a particular configure combination no longer builds one of the dozen+ DSOs.
I don't understand this suggestion. There would be no non-existent provide, as makepkg would not add one unless it finds one in the configured path. This is a complete change from the current system where the name of the library needed to be in the PKGBUILD. Now makepkg will automatically detect and add these values.
And we can't distinguish between a package that has no libraries to be provided and a makepkg.conf with a typo pointing to the wrong directory.
My bad, I didn't fully read the series and made the wrong assumption. Please ignore the above. -Emil
On Sun, Dec 12, 2021 at 11:54 AM Allan McRae
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
What about packages that install libraries into non-standard dirs and then configure ld.so to look? On my system I currently have: /usr/lib/libfakeroot /usr/lib/opencollada /usr/lib/openmpi /usr/lib/perf Can this somehow be covered? I guess ignoring it wouldn't leave us any worse off.
On 14/12/21 21:51, Jan Alexander Steffens (heftig) wrote:
On Sun, Dec 12, 2021 at 11:54 AM Allan McRae
mailto:allan@archlinux.org> wrote: This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
What about packages that install libraries into non-standard dirs and then configure ld.so to look?
On my system I currently have:
/usr/lib/libfakeroot /usr/lib/opencollada /usr/lib/openmpi /usr/lib/perf
Can this somehow be covered? I guess ignoring it wouldn't leave us any worse off.
Sure. E.g. the fakeroot PKGBUILD can add: LIB_DIRS+=('lib:usr/lib/libfakeroot') and lib:libfakeroot-0.so will be added as a provide.
Excerpts from Allan McRae's message of December 14, 2021 13:10:
On 14/12/21 21:51, Jan Alexander Steffens (heftig) wrote:
On Sun, Dec 12, 2021 at 11:54 AM Allan McRae
mailto:allan@archlinux.org> wrote: This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
What about packages that install libraries into non-standard dirs and then configure ld.so to look?
On my system I currently have:
/usr/lib/libfakeroot /usr/lib/opencollada /usr/lib/openmpi /usr/lib/perf
Can this somehow be covered? I guess ignoring it wouldn't leave us any worse off.
Sure. E.g. the fakeroot PKGBUILD can add:
LIB_DIRS+=('lib:usr/lib/libfakeroot')
and lib:libfakeroot-0.so will be added as a provide.
With the currently proposed patch it should already be added even without touching LIB_DIRS, since the `find` command used doesn't limit the depth. And adding it to LIB_DIRS would cause it to be added to provides= twice, unless the list is filtered later on in makepkg. They just won't be added to depends= automatically unless added to LIB_DIRS. -- Sincerely, Johannes Löthberg :: SA0DEM
On 14/12/21 22:39, Johannes Löthberg wrote:
Excerpts from Allan McRae's message of December 14, 2021 13:10:
On 14/12/21 21:51, Jan Alexander Steffens (heftig) wrote:
On Sun, Dec 12, 2021 at 11:54 AM Allan McRae
mailto:allan@archlinux.org> wrote: This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
What about packages that install libraries into non-standard dirs and then configure ld.so to look?
On my system I currently have:
/usr/lib/libfakeroot /usr/lib/opencollada /usr/lib/openmpi /usr/lib/perf
Can this somehow be covered? I guess ignoring it wouldn't leave us any worse off.
Sure. E.g. the fakeroot PKGBUILD can add:
LIB_DIRS+=('lib:usr/lib/libfakeroot')
and lib:libfakeroot-0.so will be added as a provide.
With the currently proposed patch it should already be added even without touching LIB_DIRS, since the `find` command used doesn't limit the depth. And adding it to LIB_DIRS would cause it to be added to provides= twice, unless the list is filtered later on in makepkg.
Good point! It will not be added twice, as the list of sonames is run through a "sort -u" before finally adding them to depends.
They just won't be added to depends= automatically unless added to LIB_DIRS.
Correct. Allan
participants (4)
-
Allan McRae
-
Emil Velikov
-
Jan Alexander Steffens (heftig)
-
Johannes Löthberg