[PATCH] Replace libdepends/libprovides
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like: LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32') At the end of package building, makepkg will look in the library directories and add a provide. E.g. for pacman: provide = lib:libalpm.so.13 Note the prefix matches the prefix given to the relevant directory in LIB_DIRS. Similarly, makepkg can add dependencies on libraries. E.g. pacman may have: depends = lib:libgpgme.so.11 Note, to help with bootstrapping this system, or if packages just do not want to add libprovides, the depends entries are only added if a package actually provides them. This is different to the APK system for libraries which uses "so" as the prefix and is not configurable. But Alpine used musl, which has no concept of multilib, so we need to be a bit more flexible. The dependency/provides addotions can all be disabled in pacman.conf with the '!autodeps' option. Note that APK has similar things for binaries and pkg-config files. e.g. provides = cmd:pacman provides = pc:libalpm These can readily be added as dropins to libmakepkg. Allan McRae (4): makepkg: remove libdepends and libprovides libmakepkg: add framework for autodeps libmakepkg: automatically add library sonames to provides libmakepkg: automatically add library dependencies doc/PKGBUILD.5.asciidoc | 9 -- doc/makepkg.conf.5.asciidoc | 11 ++ etc/makepkg.conf.in | 8 +- scripts/libmakepkg/autodep.sh.in | 38 ++++++ .../libmakepkg/autodep/library_depends.sh.in | 75 +++++++++++ .../libmakepkg/autodep/library_provides.sh.in | 56 +++++++++ scripts/libmakepkg/autodep/meson.build | 18 +++ scripts/libmakepkg/meson.build | 1 + scripts/makepkg.sh.in | 119 +----------------- 9 files changed, 206 insertions(+), 129 deletions(-) create mode 100644 scripts/libmakepkg/autodep.sh.in create mode 100644 scripts/libmakepkg/autodep/library_depends.sh.in create mode 100644 scripts/libmakepkg/autodep/library_provides.sh.in create mode 100644 scripts/libmakepkg/autodep/meson.build -- 2.34.1
This will be replaced by a better system Signed-off-by: Allan McRae <allan@archlinux.org> --- doc/PKGBUILD.5.asciidoc | 9 --- scripts/makepkg.sh.in | 118 ---------------------------------------- 2 files changed, 127 deletions(-) diff --git a/doc/PKGBUILD.5.asciidoc b/doc/PKGBUILD.5.asciidoc index dee15f5e..4ca8eb3b 100644 --- a/doc/PKGBUILD.5.asciidoc +++ b/doc/PKGBUILD.5.asciidoc @@ -187,11 +187,6 @@ contain whitespace characters. than or equal to), `<=` (less than or equal to), `=` (equal to), `>` (greater than), or `<` (less than). + -If the dependency name appears to be a library (ends with .so), makepkg will -try to find a binary that depends on the library in the built package and -append the version needed by the binary. Appending the version yourself -disables automatic detection. -+ Additional architecture-specific depends can be added by appending an underscore and the architecture name e.g., 'depends_x86_64=()'. @@ -245,10 +240,6 @@ example, dcron can provide 'cron=2.0' to satisfy the 'cron>=2.0' dependency of other packages. Provisions involving the `>` and `<` operators are invalid as only specific versions of a package may be provided. + -If the provision name appears to be a library (ends with .so), makepkg will -try to find the library in the built package and append the correct -version. Appending the version yourself disables automatic detection. -+ Additional architecture-specific provides can be added by appending an underscore and the architecture name e.g., 'provides_x86_64=()'. diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 3c7977db..639ea84a 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -463,121 +463,6 @@ run_package() { run_function_safe "package${1:+_$1}" } -find_libdepends() { - local d sodepends - - sodepends=0 - for d in "${depends[@]}"; do - if [[ $d = *.so ]]; then - sodepends=1 - break - fi - done - - if (( sodepends == 0 )); then - (( ${#depends[@]} )) && printf '%s\n' "${depends[@]}" - return 0 - fi - - local libdeps filename soarch sofile soname soversion - declare -A libdeps - - while IFS= read -rd '' filename; do - # get architecture of the file; if soarch is empty it's not an ELF binary - soarch=$(LC_ALL=C readelf -h "$filename" 2>/dev/null | sed -n 's/.*Class.*ELF\(32\|64\)/\1/p') - [[ -n "$soarch" ]] || continue - - # process all libraries needed by the binary - for sofile in $(LC_ALL=C readelf -d "$filename" 2>/dev/null | sed -nr 's/.*Shared library: \[(.*)\].*/\1/p') - do - # extract the library name: libfoo.so - soname="${sofile%.so?(+(.+([0-9])))}".so - # extract the major version: 1 - soversion="${sofile##*\.so\.}" - - if [[ ${libdeps[$soname]} ]]; then - if [[ ${libdeps[$soname]} != *${soversion}-${soarch}* ]]; then - libdeps[$soname]+=" ${soversion}-${soarch}" - fi - else - libdeps[$soname]="${soversion}-${soarch}" - fi - done - done < <(find "$pkgdir" -type f -perm -u+x -print0) - - local libdepends v - for d in "${depends[@]}"; do - case "$d" in - *.so) - if [[ ${libdeps[$d]} ]]; then - for v in ${libdeps[$d]}; do - libdepends+=("$d=$v") - done - else - warning "$(gettext "Library listed in %s is not required by any files: %s")" "'depends'" "$d" - libdepends+=("$d") - fi - ;; - *) - libdepends+=("$d") - ;; - esac - done - - (( ${#libdepends[@]} )) && printf '%s\n' "${libdepends[@]}" -} - - -find_libprovides() { - local p versioned_provides libprovides missing - for p in "${provides[@]}"; do - missing=0 - versioned_provides=() - case "$p" in - *.so) - mapfile -t filename < <(find "$pkgdir" -type f -name $p\* | LC_ALL=C sort) - if [[ $filename ]]; then - # packages may provide multiple versions of the same library - for fn in "${filename[@]}"; do - # check if we really have a shared object - if LC_ALL=C readelf -h "$fn" 2>/dev/null | grep -q '.*Type:.*DYN (Shared object file).*'; then - # get the string binaries link to (e.g. libfoo.so.1.2 -> libfoo.so.1) - local sofile=$(LC_ALL=C readelf -d "$fn" 2>/dev/null | sed -n 's/.*Library soname: \[\(.*\)\].*/\1/p') - if [[ -z "$sofile" ]]; then - warning "$(gettext "Library listed in %s is not versioned: %s")" "'provides'" "$p" - continue - fi - - # get the library architecture (32 or 64 bit) - local soarch=$(LC_ALL=C readelf -h "$fn" | sed -n 's/.*Class.*ELF\(32\|64\)/\1/p') - - # extract the library major version - local soversion="${sofile##*\.so\.}" - - versioned_provides+=("${p}=${soversion}-${soarch}") - else - warning "$(gettext "Library listed in %s is not a shared object: %s")" "'provides'" "$p" - fi - done - else - missing=1 - fi - ;; - esac - - if (( missing )); then - warning "$(gettext "Cannot find library listed in %s: %s")" "'provides'" "$p" - fi - if (( ${#versioned_provides[@]} > 0 )); then - libprovides+=("${versioned_provides[@]}") - else - libprovides+=("$p") - fi - done - - (( ${#libprovides[@]} )) && printf '%s\n' "${libprovides[@]}" -} - write_kv_pair() { local key="$1" shift @@ -617,9 +502,6 @@ write_pkginfo() { write_kv_pair "size" "$size" write_kv_pair "arch" "$pkgarch" - mapfile -t provides < <(find_libprovides) - mapfile -t depends < <(find_libdepends) - write_kv_pair "license" "${license[@]}" write_kv_pair "replaces" "${replaces[@]}" write_kv_pair "group" "${groups[@]}" -- 2.34.1
Signed-off-by: Allan McRae <allan@archlinux.org> --- etc/makepkg.conf.in | 6 +++-- scripts/libmakepkg/autodep.sh.in | 38 ++++++++++++++++++++++++++++++++ scripts/makepkg.sh.in | 1 + 3 files changed, 43 insertions(+), 2 deletions(-) create mode 100644 scripts/libmakepkg/autodep.sh.in diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index 5ad3d490..f69a7add 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -76,7 +76,8 @@ BUILDENV=(!distcc color !ccache check !sign) # These are default values for the options=() settings ######################################################################### # -# Makepkg defaults: OPTIONS=(!strip docs libtool staticlibs emptydirs !zipman !purge !debug !lto) +# Makepkg defaults: +# OPTIONS=(!strip docs libtool staticlibs emptydirs !zipman !purge !debug !lto !autodeps) # A negated option will do the opposite of the comments below. # #-- strip: Strip symbols from binaries/libraries @@ -88,8 +89,9 @@ BUILDENV=(!distcc color !ccache check !sign) #-- purge: Remove files specified by PURGE_TARGETS #-- debug: Add debugging flags as specified in DEBUG_* variables #-- lto: Add compile flags for building with link time optimization +#-- autodeps: Automatically add depends/provides # -OPTIONS=(strip docs libtool staticlibs emptydirs zipman purge !debug !lto) +OPTIONS=(strip docs libtool staticlibs emptydirs zipman purge !debug !lto !autodeps) #-- File integrity checks to use. Valid: ck, md5, sha1, sha224, sha256, sha384, sha512, b2 INTEGRITY_CHECK=(ck) diff --git a/scripts/libmakepkg/autodep.sh.in b/scripts/libmakepkg/autodep.sh.in new file mode 100644 index 00000000..80f66e70 --- /dev/null +++ b/scripts/libmakepkg/autodep.sh.in @@ -0,0 +1,38 @@ +#!/bin/bash +# +# autodep.sh - functions for automatically adding depends/provides +# +# Copyright (c) 2021 Pacman Development Team <pacman-dev@lists.archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[[ -n "$LIBMAKEPKG_AUTODEP_SH" ]] && return +LIBMAKEPKG_AUTODEP_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +declare -a autodep_functions + +for lib in "$LIBRARY/autodep/"*.sh; do + source "$lib" +done + +readonly -a autodep_functions + +generate_autodeps() { + for func in ${autodep_functions[@]}; do + $func + done +} diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 639ea84a..818eb471 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -480,6 +480,7 @@ write_pkginfo() { local size=$(dirsize) merge_arch_attrs + generate_autodeps printf "# Generated by makepkg %s\n" "$makepkg_version" printf "# using %s\n" "$(fakeroot -v)" -- 2.34.1
When the option "autodeps" is enabled, makepkg will add provides entries for libraries found in the directories specified in LIB_DIRS in makepkg.conf. The entries LIB_DIRS array have the format "prefix:directory". For example, the entry "lib:usr/lib" will search $pkgdir/usr/lib for library sonames and add "lib:libfoo.so.1" to the provides array. Signed-off-by: Allan McRae <allan@archlinux.org> --- doc/makepkg.conf.5.asciidoc | 11 ++++ etc/makepkg.conf.in | 2 + .../libmakepkg/autodep/library_provides.sh.in | 56 +++++++++++++++++++ scripts/libmakepkg/autodep/meson.build | 18 ++++++ scripts/libmakepkg/meson.build | 1 + 5 files changed, 88 insertions(+) create mode 100644 scripts/libmakepkg/autodep/library_provides.sh.in create mode 100644 scripts/libmakepkg/autodep/meson.build diff --git a/doc/makepkg.conf.5.asciidoc b/doc/makepkg.conf.5.asciidoc index 76c27f6a..39c5c808 100644 --- a/doc/makepkg.conf.5.asciidoc +++ b/doc/makepkg.conf.5.asciidoc @@ -193,6 +193,11 @@ Options Enable building packages using link time optimization. Adds '-flto' to both CFLAGS and CXXFLAGS. + *autodep*;; + Enable the automatic addition of libraries to the depends and + provides arrays. Search library directories are controlled by + the LIB_DIRS variable defined below. + **INTEGRITY_CHECK=(**check1 ...**)**:: File integrity checks to use. Multiple checks may be specified; this affects both generation and checking. The current valid options are: @@ -223,6 +228,12 @@ Options that are located in opt/, you may need to add the directory to this array. *NOTE:* Do not add the leading slash to the directory name. +**LIB_DIRS=(**lib:usr/lib ...**)**:: + If `autodeps` is specified in the `OPTIONS` array, this variable will + instruct makepkg where to look to find librarys to add to the `provides` + array. The format is "prefix:path", where provides will be added for + libraries found in "path" with the specified prefix added. + **PURGE_TARGETS=(**usr/{,share}/info/dir .podlist *.pod...**)**:: If `purge` is specified in the `OPTIONS` array, this variable will instruct makepkg which files to remove from the package. This is diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index f69a7add..0c911cce 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -109,6 +109,8 @@ DOC_DIRS=(usr/{,local/}{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc}) PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod) #-- Directory to store source code in for debug packages DBGSRCDIR="/usr/src/debug" +#-- Prefix and directories for library autodeps +LIB_DIRS=('lib:/usr/lib' 'lib32:/usr/lib32') ######################################################################### # PACKAGE OUTPUT diff --git a/scripts/libmakepkg/autodep/library_provides.sh.in b/scripts/libmakepkg/autodep/library_provides.sh.in new file mode 100644 index 00000000..5cd497fa --- /dev/null +++ b/scripts/libmakepkg/autodep/library_provides.sh.in @@ -0,0 +1,56 @@ +#!/bin/bash +# +# library_provides.sh - Automatically add a packages libraries to provides +# +# Copyright (c) 2021 Pacman Development Team <pacman-dev@lists.archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[[ -n "$LIBMAKEPKG_AUTODEP_LIBRARY_PROVIDES_SH" ]] && return +LIBMAKEPKG_AUTODEP_LIBRARY_PROVIDES_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +autodep_functions+=('library_provides') + +library_provides() { + if check_option "autodeps" "y"; then + for lib in ${LIB_DIRS[@]}; do + dir=${lib/*:} + prefix=${lib/:*} + + if [[ ! -d "$pkgdir/$dir" ]]; then + continue; + fi + + mapfile -t filenames < <(find "$pkgdir/$dir" -type f | LC_ALL=C sort) + + for fn in "${filenames[@]}"; do + # check we have a shared library + if LC_ALL=C readelf -h "$fn" 2>/dev/null | grep -q '.*Type:.*DYN (Shared object file).*'; then + # extract library soname + local sofile=$(LC_ALL=C readelf -d "$fn" 2>/dev/null | sed -n 's/.*Library soname: \[\(.*\)\].*/\1/p') + + if [[ -z "$sofile" ]]; then + # the library is not versioned + continue + fi + + provides+=("$prefix:$sofile") + fi + done + done + fi +} diff --git a/scripts/libmakepkg/autodep/meson.build b/scripts/libmakepkg/autodep/meson.build new file mode 100644 index 00000000..e0af26a6 --- /dev/null +++ b/scripts/libmakepkg/autodep/meson.build @@ -0,0 +1,18 @@ +libmakepkg_module = 'tidy' + +sources = [ + 'library_provides.sh.in', + +] + +foreach src : sources + output_dir = join_paths(get_option('datadir'), 'makepkg', libmakepkg_module) + + custom_target( + libmakepkg_module + '_' + src.underscorify(), + command : [ SCRIPT_EDITOR, '@INPUT@', '@OUTPUT@' ], + input : src, + output : '@BASENAME@', + install : true, + install_dir : output_dir) +endforeach diff --git a/scripts/libmakepkg/meson.build b/scripts/libmakepkg/meson.build index 50eb905b..6e83a4a8 100644 --- a/scripts/libmakepkg/meson.build +++ b/scripts/libmakepkg/meson.build @@ -1,4 +1,5 @@ libmakepkg_modules = [ + { 'name' : 'autodep', 'has_subdir' : true }, { 'name' : 'buildenv', 'has_subdir' : true }, { 'name' : 'executable', 'has_subdir' : true }, { 'name' : 'integrity', 'has_subdir' : true }, -- 2.34.1
Add linked libraries to a packages dependency list. This is the partner to automatically generated library provides, and thus depends take the same format. To help with bootstrapping, library dependencies are only added if the relevant provide exists. Signed-off-by: Allan McRae <allan@archlinux.org> --- .../libmakepkg/autodep/library_depends.sh.in | 75 +++++++++++++++++++ scripts/libmakepkg/autodep/meson.build | 2 +- 2 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 scripts/libmakepkg/autodep/library_depends.sh.in diff --git a/scripts/libmakepkg/autodep/library_depends.sh.in b/scripts/libmakepkg/autodep/library_depends.sh.in new file mode 100644 index 00000000..4c387241 --- /dev/null +++ b/scripts/libmakepkg/autodep/library_depends.sh.in @@ -0,0 +1,75 @@ +#!/bin/bash +# +# library_depends.sh - Automatically add library requirements to depends +# +# Copyright (c) 2021 Pacman Development Team <pacman-dev@lists.archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[[ -n "$LIBMAKEPKG_AUTODEP_LIBRARY_DEPENDS_SH" ]] && return +LIBMAKEPKG_AUTODEP_LIBRARY_DEPENDS_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +autodep_functions+=('library_depends') + +library_depends() { + if check_option "autodeps" "y"; then + local dep filename libdeps libdir libpath prefix sofile + declare -a libdeps + + while IFS= read -rd '' filename; do + for sofile in $(LC_ALL=C readelf -d $filename 2>/dev/null | sed -nr 's/.*Shared library: \[(.*)\].*/\1/p'); do + # get the full path of the library + libpath=$(ldd $filename | sed -nr "s/.$sofile => (.*) \(.*\)/\1/p") + + # if ldd can not find the library, it is likely part of the package and not in filesystem + if [[ -z $libpath ]]; then + continue + fi + + # skip if the library is part of the package + if [[ -e "$pkgdir/$libpath" ]]; then + continue + fi + + # find the prefix for the dependency + libpath=${libpath#/} + libpath=${libpath%/*} + + unset prefix + for libdir in ${LIB_DIRS[@]}; do + if [[ ${libdir/*:} == ${libpath} ]]; then + prefix=${libdir/:*} + fi + done + + if [[ -z ${prefix} ]]; then + continue + fi + + # only add library dependency if it exists - this helps bootstraping dependencies + if [[ $(run_pacman -T "$prefix:$sofile") ]]; then + continue + fi + + libdeps+=("$prefix:$sofile") + done + + done < <(find "$pkgdir" -type f -perm -u+x -print0) + + depends+=($(printf '%s\n' "${libdeps[@]}" | LC_ALL=C sort -u)) + fi +} diff --git a/scripts/libmakepkg/autodep/meson.build b/scripts/libmakepkg/autodep/meson.build index e0af26a6..08c4c818 100644 --- a/scripts/libmakepkg/autodep/meson.build +++ b/scripts/libmakepkg/autodep/meson.build @@ -1,8 +1,8 @@ libmakepkg_module = 'tidy' sources = [ + 'library_depends.sh.in', 'library_provides.sh.in', - ] foreach src : sources -- 2.34.1
Hey Allan, I really like the idea, although I might have spotted some gotchas. On Sun, 12 Dec 2021 at 10:54, Allan McRae <allan@archlinux.org> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
At the end of package building, makepkg will look in the library directories and add a provide. E.g. for pacman:
provide = lib:libalpm.so.13
If I understand correctly, we create a mapping pretty-name:actual-path in the makepkg.conf and use the pretty-name in PKGBUILD. Do we add the pretty-name or actual-path in .PKGINFO? Does pacman print these autodeps ... even if only for informational purposes? What is expected to happen if someone typos the existing mappings - say "lib32:usr/lib23"? Or re-defines the standard ones "lib:my/special/path"? Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
Note the prefix matches the prefix given to the relevant directory in LIB_DIRS. Similarly, makepkg can add dependencies on libraries. E.g. pacman may have:
depends = lib:libgpgme.so.11
Note, to help with bootstrapping this system, or if packages just do not want to add libprovides, the depends entries are only added if a package actually provides them.
This is different to the APK system for libraries which uses "so" as the prefix and is not configurable. But Alpine used musl, which has no concept of multilib, so we need to be a bit more flexible.
I think the above questions might be the reason why Alpine chose to keep them hard-coded. In particular, while this approach provides flexibility, it leaves things open to various mistakes and misuse. Off the top of my head, we can enhance makepkg to catch most of those issues. For example - we can use the actual-path in .PKGINFO, explicitly check for mistakes/changes in the default mappings - lib, lib32... Although at this point one might as well have the defaults hardcoded in makepkg?
The dependency/provides addotions can all be disabled in pacman.conf with the '!autodeps' option.
s/addotions/additions/;s/pacman.conf/makepkg.conf/ typos? At a glance we can disable autodeps in the PKGBUILD itself, right? HTH Emil
On 14/12/21 20:40, Emil Velikov wrote:
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae <allan@archlinux.org> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
Not really... This is the path for adding library dependencies & provides. If other autodeps get added, they may need their own configuration option.
At the end of package building, makepkg will look in the library directories and add a provide. E.g. for pacman:
provide = lib:libalpm.so.13
If I understand correctly, we create a mapping pretty-name:actual-path in the makepkg.conf and use the pretty-name in PKGBUILD.
Do we add the pretty-name or actual-path in .PKGINFO?
That is the line from the .PKGINFO provide = <tag>:<soname> Does pacman
print these autodeps ... even if only for informational purposes?
They are treated the same as any provide or depend. For example, using -Qi on my pacman-git package currently shows: Provides : pacman=6.0.1 lib:libalpm.so.13 Depends On : archlinux-keyring bash curl gpgme libarchive pacman-mirrorlist lib:libcurl.so.4
What is expected to happen if someone typos the existing mappings - say "lib32:usr/lib23"? Or re-defines the standard ones "lib:my/special/path"?
It their makepkg.conf has "lib32:usr/lib23" it is very likely that no dependencies/provides will be added.
Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
As above, nothing will happen. The usr/lib23 directory will (I guess) never occur in a package, so never be searched for files with an soname. I see this as being the same as any other configuration typo. e.g. if MAN_DIRS pointed to non-existent places, no man pages would be compressed.
Note the prefix matches the prefix given to the relevant directory in LIB_DIRS. Similarly, makepkg can add dependencies on libraries. E.g. pacman may have:
depends = lib:libgpgme.so.11
Note, to help with bootstrapping this system, or if packages just do not want to add libprovides, the depends entries are only added if a package actually provides them.
This is different to the APK system for libraries which uses "so" as the prefix and is not configurable. But Alpine used musl, which has no concept of multilib, so we need to be a bit more flexible.
I think the above questions might be the reason why Alpine chose to keep them hard-coded. In particular, while this approach provides flexibility, it leaves things open to various mistakes and misuse.
MAN_DIRS, DOC_DIRS, PURGE_TARGETS, ... all could have typos in their configuration and not work.
Off the top of my head, we can enhance makepkg to catch most of those issues. For example - we can use the actual-path in .PKGINFO, explicitly check for mistakes/changes in the default mappings - lib, lib32... Although at this point one might as well have the defaults hardcoded in makepkg?
This needs to be configurable. e.g. some distros will have: LIB_DIRS=('so:lib' 'so:usr/lib') when they have not done a /usr merge. Note, I selected the "so" prefix as being clearer for that example. I'm thinking "so:usr/lib" and "so32:usr/lib32" might be better for the configuration example.
The dependency/provides addotions can all be disabled in pacman.conf with the '!autodeps' option.
s/addotions/additions/;s/pacman.conf/makepkg.conf/ typos?
Yes - I meant the OPTIONS array in makepkg.conf.
At a glance we can disable autodeps in the PKGBUILD itself, right?
Yes - it is can be specified in options=(). Thanks for the comments. Allan
On Sun, Dec 12, 2021 at 11:54 AM Allan McRae <allan@archlinux.org> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
What about packages that install libraries into non-standard dirs and then configure ld.so to look? On my system I currently have: /usr/lib/libfakeroot /usr/lib/opencollada /usr/lib/openmpi /usr/lib/perf Can this somehow be covered? I guess ignoring it wouldn't leave us any worse off.
On 14/12/21 21:51, Jan Alexander Steffens (heftig) wrote:
On Sun, Dec 12, 2021 at 11:54 AM Allan McRae <allan@archlinux.org <mailto:allan@archlinux.org>> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
What about packages that install libraries into non-standard dirs and then configure ld.so to look?
On my system I currently have:
/usr/lib/libfakeroot /usr/lib/opencollada /usr/lib/openmpi /usr/lib/perf
Can this somehow be covered? I guess ignoring it wouldn't leave us any worse off.
Sure. E.g. the fakeroot PKGBUILD can add: LIB_DIRS+=('lib:usr/lib/libfakeroot') and lib:libfakeroot-0.so will be added as a provide.
On Tue, 14 Dec 2021 at 11:28, Allan McRae <allan@archlinux.org> wrote:
On 14/12/21 20:40, Emil Velikov wrote:
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae <allan@archlinux.org> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
Not really... This is the path for adding library dependencies & provides. If other autodeps get added, they may need their own configuration option.
This sounds great. [snip]
Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
As above, nothing will happen. The usr/lib23 directory will (I guess) never occur in a package, so never be searched for files with an soname.
We might want to have a simple check in makepkg, to high-light those. If PKGBUILD has "provides" to a non-existant file, we could error out IMHO. This will catch both typos on the packaging side as well as buggy upstream - say they dropped/renamed the library, or a particular configure combination no longer builds one of the dozen+ DSOs. This is not a blocking suggestion, but I can see it as quite beneficial. [snip]
Although at this point one might as well have the defaults hardcoded in makepkg?
This needs to be configurable. e.g. some distros will have:
LIB_DIRS=('so:lib' 'so:usr/lib')
when they have not done a /usr merge.
Agreed. When I said hard-coded I was thinking of a default, which can be overwritten during build time. Say something like: meson -D lib_dirs="lib:lib64/, lib32:lib/" ... Thanks for elevating my concerns o/ -Emil
On 14/12/21 22:28, Emil Velikov wrote:
On Tue, 14 Dec 2021 at 11:28, Allan McRae <allan@archlinux.org> wrote:
On 14/12/21 20:40, Emil Velikov wrote:
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae <allan@archlinux.org> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
Not really... This is the path for adding library dependencies & provides. If other autodeps get added, they may need their own configuration option.
This sounds great.
[snip]
Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
As above, nothing will happen. The usr/lib23 directory will (I guess) never occur in a package, so never be searched for files with an soname.
We might want to have a simple check in makepkg, to high-light those. If PKGBUILD has "provides" to a non-existant file, we could error out IMHO.
This will catch both typos on the packaging side as well as buggy upstream - say they dropped/renamed the library, or a particular configure combination no longer builds one of the dozen+ DSOs.
I don't understand this suggestion. There would be no non-existent provide, as makepkg would not add one unless it finds one in the configured path. This is a complete change from the current system where the name of the library needed to be in the PKGBUILD. Now makepkg will automatically detect and add these values. And we can't distinguish between a package that has no libraries to be provided and a makepkg.conf with a typo pointing to the wrong directory.
This is not a blocking suggestion, but I can see it as quite beneficial.
[snip]
Although at this point one might as well have the defaults hardcoded in makepkg?
This needs to be configurable. e.g. some distros will have:
LIB_DIRS=('so:lib' 'so:usr/lib')
when they have not done a /usr merge.
Agreed. When I said hard-coded I was thinking of a default, which can be overwritten during build time. Say something like: meson -D lib_dirs="lib:lib64/, lib32:lib/" ...
I'm going to suggest the default is not to have this feature running at all. So blank is a good default! Or course each distro will provide a suitable makepkg.conf with their opinionated configuration. A
Excerpts from Allan McRae's message of December 14, 2021 13:10:
On 14/12/21 21:51, Jan Alexander Steffens (heftig) wrote:
On Sun, Dec 12, 2021 at 11:54 AM Allan McRae <allan@archlinux.org <mailto:allan@archlinux.org>> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
What about packages that install libraries into non-standard dirs and then configure ld.so to look?
On my system I currently have:
/usr/lib/libfakeroot /usr/lib/opencollada /usr/lib/openmpi /usr/lib/perf
Can this somehow be covered? I guess ignoring it wouldn't leave us any worse off.
Sure. E.g. the fakeroot PKGBUILD can add:
LIB_DIRS+=('lib:usr/lib/libfakeroot')
and lib:libfakeroot-0.so will be added as a provide.
With the currently proposed patch it should already be added even without touching LIB_DIRS, since the `find` command used doesn't limit the depth. And adding it to LIB_DIRS would cause it to be added to provides= twice, unless the list is filtered later on in makepkg. They just won't be added to depends= automatically unless added to LIB_DIRS. -- Sincerely, Johannes Löthberg :: SA0DEM
On Tue, 14 Dec 2021 at 12:38, Allan McRae <allan@archlinux.org> wrote:
On 14/12/21 22:28, Emil Velikov wrote:
On Tue, 14 Dec 2021 at 11:28, Allan McRae <allan@archlinux.org> wrote:
On 14/12/21 20:40, Emil Velikov wrote:
Hey Allan,
I really like the idea, although I might have spotted some gotchas.
On Sun, 12 Dec 2021 at 10:54, Allan McRae <allan@archlinux.org> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
Considering your examples (below) also handle "cmd" and "pc" the LIB_DIRS name is misleading. Alas no better suggestion comes to mind ATM.
Not really... This is the path for adding library dependencies & provides. If other autodeps get added, they may need their own configuration option.
This sounds great.
[snip]
Are we going to continue or error out - is the error message going to be meaningful or rather cryptic?
As above, nothing will happen. The usr/lib23 directory will (I guess) never occur in a package, so never be searched for files with an soname.
We might want to have a simple check in makepkg, to high-light those. If PKGBUILD has "provides" to a non-existant file, we could error out IMHO.
This will catch both typos on the packaging side as well as buggy upstream - say they dropped/renamed the library, or a particular configure combination no longer builds one of the dozen+ DSOs.
I don't understand this suggestion. There would be no non-existent provide, as makepkg would not add one unless it finds one in the configured path. This is a complete change from the current system where the name of the library needed to be in the PKGBUILD. Now makepkg will automatically detect and add these values.
And we can't distinguish between a package that has no libraries to be provided and a makepkg.conf with a typo pointing to the wrong directory.
My bad, I didn't fully read the series and made the wrong assumption. Please ignore the above. -Emil
On 14/12/21 22:39, Johannes Löthberg wrote:
Excerpts from Allan McRae's message of December 14, 2021 13:10:
On 14/12/21 21:51, Jan Alexander Steffens (heftig) wrote:
On Sun, Dec 12, 2021 at 11:54 AM Allan McRae <allan@archlinux.org <mailto:allan@archlinux.org>> wrote:
This patch series replaces the old libdepends/libprovides system into something akin to that used by APK. In short, makepkg.conf will have a variable like:
LIB_DIRS=('lib:usr/lib' 'lib32:usr/lib32')
What about packages that install libraries into non-standard dirs and then configure ld.so to look?
On my system I currently have:
/usr/lib/libfakeroot /usr/lib/opencollada /usr/lib/openmpi /usr/lib/perf
Can this somehow be covered? I guess ignoring it wouldn't leave us any worse off.
Sure. E.g. the fakeroot PKGBUILD can add:
LIB_DIRS+=('lib:usr/lib/libfakeroot')
and lib:libfakeroot-0.so will be added as a provide.
With the currently proposed patch it should already be added even without touching LIB_DIRS, since the `find` command used doesn't limit the depth. And adding it to LIB_DIRS would cause it to be added to provides= twice, unless the list is filtered later on in makepkg.
Good point! It will not be added twice, as the list of sonames is run through a "sort -u" before finally adding them to depends.
They just won't be added to depends= automatically unless added to LIB_DIRS.
Correct. Allan
participants (4)
-
Allan McRae
-
Emil Velikov
-
Jan Alexander Steffens (heftig)
-
Johannes Löthberg