October 2019 - Pacman-dev - lists.archlinux.org

[pacman-dev] Alternatives system
by Allan McRae 06 Dec '20

06 Dec '20

Hi all, I plan to finish implementing an alternatives system for pacman post 5.2 release: https://wiki.archlinux.org/index.php/User:Allan/Alternatives Any comments or suggestions? Thanks, Allan

6 7

[pacman-dev] [PATCH] libmakepkg: when stripping split debug symbols, warn on duplicate paths
by Eli Schwartz 03 Feb '20

03 Feb '20

Trying to strip multiple files installed to the same filepath (in different components of a split package) can produce surprising results if those files are built using different options. Per FS#63356 this is not a supported PKGBUILD configuration. Instead, warn about such cases when producing debug packages. Fixes FS#63366 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- scripts/libmakepkg/tidy/strip.sh.in | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/scripts/libmakepkg/tidy/strip.sh.in b/scripts/libmakepkg/tidy/strip.sh.in index 5937e13c..eaa764b7 100644 --- a/scripts/libmakepkg/tidy/strip.sh.in +++ b/scripts/libmakepkg/tidy/strip.sh.in @@ -46,7 +46,13 @@ strip_file() { if check_option "debug" "y"; then local bid=$(build_id "$binary") - # has this file already been stripped + # has this filepath been stripped before under a different Build Id + if [[ -f "$dbgdir/$binary.debug" ]]; then + if [[ -n $bid && $bid != $(build_id "$dbgdir/$binary.debug") ]]; then + warning "Split package with multiple copies of the file '%s' have different Build Ids" "$binary" + fi + fi + # has this file's hardlink already been stripped if [[ -n "$bid" ]]; then if [[ -f "$dbgdir/.build-id/${bid:0:2}/${bid:2}.debug" ]]; then return -- 2.22.0

2 4

[pacman-dev] [PATCH] libmakepkg: fix empty arguments in parseopts
by Ethan Sommer 05 Nov '19

05 Nov '19

Previously parseopts checked if there was an argument by checking that the string was non-empty, resulting in empty arguments being incorrectly considered non-existent. This change makes parseopts check if arguments exist at all, rather than checking that they are non-empty Signed-off-by: Ethan Sommer <e5ten.arch(a)gmail.com> --- scripts/libmakepkg/util/parseopts.sh.in | 8 ++++---- test/scripts/parseopts_test.sh | 5 ++++- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/scripts/libmakepkg/util/parseopts.sh.in b/scripts/libmakepkg/util/parseopts.sh.in index c056cb1e..8fb862d8 100644 --- a/scripts/libmakepkg/util/parseopts.sh.in +++ b/scripts/libmakepkg/util/parseopts.sh.in @@ -102,7 +102,7 @@ parseopts() { OPTRET+=("${1:i+1}") break # if we're at the end, grab the the next positional, if it exists - elif (( i == ${#1} - 1 )) && [[ $2 ]]; then + elif (( i == ${#1} - 1 && $# > 1 )); then OPTRET+=("$2") shift break @@ -121,7 +121,7 @@ parseopts() { case $? in 0) # parse failure - if [[ $optarg ]]; then + if [[ $1 = *=* ]]; then printf "${0##*/}: $(gettext "option '%s' does not allow an argument")\n" "--$opt" >&2 OPTRET=(--) return 1 @@ -132,10 +132,10 @@ parseopts() { ;; 1) # --longopt=optarg - if [[ $optarg ]]; then + if [[ $1 = *=* ]]; then OPTRET+=("--$opt" "$optarg") # --longopt optarg - elif [[ $2 ]]; then + elif (( $# > 1 )); then OPTRET+=("--$opt" "$2" ) shift # parse failure diff --git a/test/scripts/parseopts_test.sh b/test/scripts/parseopts_test.sh index 9674c6a6..1d76f1ad 100755 --- a/test/scripts/parseopts_test.sh +++ b/test/scripts/parseopts_test.sh @@ -31,7 +31,7 @@ tap_parse() { unset OPTRET } -tap_plan 50 +tap_plan 52 # usage: tap_parse <expected result> <token count> test-params... # a failed tap_parse will match only the end of options marker '--' @@ -111,4 +111,7 @@ tap_parse '--force --' 2 --force # exact match on possible stem (opt has optarg) tap_parse '--clean foo --' 3 --clean=foo +# all possible ways to specify empty optargs +tap_parse '--key --pkg -p --' 7 --key '' --pkg='' -p '' + tap_finish -- 2.23.0

2 5

[pacman-dev] Alternatives system brainstorm
by Allan McRae 04 Nov '19

04 Nov '19

Hi all, Some further brainstorming on alternatives system. Last time, I suggested something like: alternatives=(python->python3) But as was pointed out, there may be many other things you want to bring along with the python binary. Some directly related: /usr/share/man/man1/python3.1.gz -> python.1.gz Some less directly related: /usr/bin/idle /usr/bin/pydoc /usr/bin/python-config Another example would be lua, where you want to set the main interpreter and the header directory. In Debian & Fedora land, these additional files are called slaves. Note that both those distros use the "same" tool to manage these things. In addition, each alternative is given a priority. So if (e.g.) two python2 and python3 are installed at the same time, it would select the one with the higher priority. Fedora also has "ghost" files (not sure about Debian). I.e. the python3 package contains a ghost /usr/bin/python, so searching file databases gives a match. Now how does that inform our potential implementation? Mostly, I don't like that they set up all the alternatives in the post install with one long command passed to their tool. I'd prefer we had a file with a list of needed symlinks so this could happen automatically. So here is a starting point for discussion: In the PKGBUILD: alternatives=('python::python3') that would look for a file in our build root called "python3.alternatives" and include it in the package root. Note we could have more than one such entry. This file could look like: /usr/bin/python3 -> python /usr/bin/idle3 -> idle /usr/include/example3/ -> example/ Could also include an optional priority statement? Or should that be captured in the PKGBUILD? The file would be placed at /var/lib/pacman/alternatives/python/python3. Any comments on that as a starting point. Cheers, Allan

9 38

[pacman-dev] [PATCH v3] libmakepkg: add optional argument support to parseopts
by Ethan Sommer 04 Nov '19

04 Nov '19

Adds a "?" suffix that can be used to indicate that an option's argument is optional. This allows options to have a default behaviour when the user doesn't specify one, e.g.: --color=[when] being able to behave like --color=auto when only --color is passed Options with optional arguments given on the command line will be returned in the form "--opt=optarg" and "-o=optarg". Despite that not being the syntax for passing an argument with a shortopt (trying to pass -o=foo would make -o's argument "=foo"), this is done to allow the caller to split the option and its optarg easily Signed-off-by: Ethan Sommer <e5ten.arch(a)gmail.com> --- scripts/libmakepkg/util/parseopts.sh.in | 116 +++++++++++++++--------- test/scripts/parseopts_test.sh | 12 ++- 2 files changed, 83 insertions(+), 45 deletions(-) diff --git a/scripts/libmakepkg/util/parseopts.sh.in b/scripts/libmakepkg/util/parseopts.sh.in index c056cb1e..42540438 100644 --- a/scripts/libmakepkg/util/parseopts.sh.in +++ b/scripts/libmakepkg/util/parseopts.sh.in @@ -18,16 +18,23 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. # # A getopt_long-like parser which portably supports longopts and -# shortopts with some GNU extensions. It does not allow for options -# with optional arguments. For both short and long opts, options -# requiring an argument should be suffixed with a colon. After the -# first argument containing the short opts, any number of valid long -# opts may be be passed. The end of the options delimiter must then be -# added, followed by the user arguments to the calling program. +# shortopts with some GNU extensions. For both short and long opts, +# options requiring an argument should be suffixed with a colon, and +# options with optional arguments should be suffixed with a question +# mark. After the first argument containing the short opts, any number +# of valid long opts may be be passed. The end of the options delimiter +# must then be added, followed by the user arguments to the calling +# program. +# +# Options with optional arguments will be returned as "--longopt=optarg" +# for longopts, or "-o=optarg" for shortopts. This isn't actually a valid +# way to pass an optional argument with a shortopt on the command line, +# but is done by parseopts to enable the caller script to split the option +# and its optarg easily. # # Recommended Usage: -# OPT_SHORT='fb:z' -# OPT_LONG=('foo' 'bar:' 'baz') +# OPT_SHORT='fb:zq?' +# OPT_LONG=('foo' 'bar:' 'baz' 'qux?') # if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then # exit 1 # fi @@ -49,29 +56,30 @@ parseopts() { longoptmatch() { local o longmatch=() for o in "${longopts[@]}"; do - if [[ ${o%:} = "$1" ]]; then + if [[ ${o%[:?]} = "$1" ]]; then longmatch=("$o") break fi - [[ ${o%:} = "$1"* ]] && longmatch+=("$o") + [[ ${o%[:?]} = "$1"* ]] && longmatch+=("$o") done case ${#longmatch[*]} in 1) - # success, override with opt and return arg req (0 == none, 1 == required) - opt=${longmatch%:} - if [[ $longmatch = *: ]]; then - return 1 - else - return 0 - fi ;; + # success, override with opt and return arg req (0 == none, 1 == required, 2 == optional) + opt=${longmatch%[:?]} + case $longmatch in + *:) return 1 ;; + *\?) return 2 ;; + *) return 0 ;; + esac + ;; 0) # fail, no match found return 255 ;; *) # fail, ambiguous match printf "${0##*/}: $(gettext "option '%s' is ambiguous; possibilities:")" "--$1" - printf " '%s'" "${longmatch[@]%:}" + printf " '%s'" "${longmatch[@]%[:?]}" printf '\n' return 254 ;; esac >&2 @@ -87,32 +95,47 @@ parseopts() { for (( i = 1; i < ${#1}; i++ )); do opt=${1:i:1} - # option doesn't exist - if [[ $shortopts != *$opt* ]]; then - printf "${0##*/}: $(gettext "invalid option") -- '%s'\n" "$opt" >&2 - OPTRET=(--) - return 1 - fi - - OPTRET+=("-$opt") - # option requires optarg - if [[ $shortopts = *$opt:* ]]; then - # if we're not at the end of the option chunk, the rest is the optarg - if (( i < ${#1} - 1 )); then - OPTRET+=("${1:i+1}") - break - # if we're at the end, grab the the next positional, if it exists - elif (( i == ${#1} - 1 )) && [[ $2 ]]; then - OPTRET+=("$2") - shift - break - # parse failure - else - printf "${0##*/}: $(gettext "option requires an argument") -- '%s'\n" "$opt" >&2 + case $shortopts in + # option requires optarg + *$opt:*) + # if we're not at the end of the option chunk, the rest is the optarg + if (( i < ${#1} - 1 )); then + OPTRET+=("-$opt" "${1:i+1}") + break + # if we're at the end, grab the the next positional, if it exists + elif (( i == ${#1} - 1 )) && [[ $2 ]]; then + OPTRET+=("-$opt" "$2") + shift + break + # parse failure + else + printf "${0##*/}: $(gettext "option requires an argument") -- '%s'\n" "$opt" >&2 + OPTRET=(--) + return 1 + fi + ;; + # option's optarg is optional + *$opt\?*) + # if we're not at the end of the option chunk, the rest is the optarg + if (( i < ${#1} - 1 )); then + OPTRET+=("-$opt=${1:i+1}") + break + # option has no optarg + else + OPTRET+=("-$opt") + fi + ;; + # option has no optarg + *$opt*) + OPTRET+=("-$opt") + ;; + # option doesn't exist + *) + printf "${0##*/}: $(gettext "invalid option") -- '%s'\n" "$opt" >&2 OPTRET=(--) return 1 - fi - fi + ;; + esac done ;; --?*=*|--?*) # long option @@ -145,6 +168,15 @@ parseopts() { return 1 fi ;; + 2) + # --longopt=optarg + if [[ $1 = *=* ]]; then + OPTRET+=("--$opt=$optarg") + # --longopt + else + OPTRET+=("--$opt") + fi + ;; 254) # ambiguous option -- error was reported for us by longoptmatch() OPTRET=(--) diff --git a/test/scripts/parseopts_test.sh b/test/scripts/parseopts_test.sh index 9674c6a6..7d14bf29 100755 --- a/test/scripts/parseopts_test.sh +++ b/test/scripts/parseopts_test.sh @@ -16,12 +16,12 @@ if ! type -t parseopts &>/dev/null; then fi # borrow opts from makepkg -OPT_SHORT="AcdefFghiLmop:rRsV" +OPT_SHORT="AcdefFghiLmop:rRsVb;" OPT_LONG=('allsource' 'asroot' 'ignorearch' 'check' 'clean:' 'cleanall' 'nodeps' 'noextract' 'force' 'forcever:' 'geninteg' 'help' 'holdver' 'install' 'key:' 'log' 'nocolor' 'nobuild' 'nocheck' 'noprepare' 'nosign' 'pkg:' 'rmdeps' 'repackage' 'skipinteg' 'sign' 'source' 'syncdeps' 'version' 'config:' - 'noconfirm' 'noprogressbar') + 'noconfirm' 'noprogressbar' 'opt;') tap_parse() { local result=$1 tokencount=$2; shift 2 @@ -31,7 +31,7 @@ tap_parse() { unset OPTRET } -tap_plan 50 +tap_plan 54 # usage: tap_parse <expected result> <token count> test-params... # a failed tap_parse will match only the end of options marker '--' @@ -111,4 +111,10 @@ tap_parse '--force --' 2 --force # exact match on possible stem (opt has optarg) tap_parse '--clean foo --' 3 --clean=foo +# long opt with empty, non-empty, and no optional arg +tap_parse '--opt= --opt=foo --opt --' 4 --opt= --opt=foo --opt + +# short opt with and without optional arg, and non-option arg +tap_parse '-b=foo -A -b -- foo' 5 -bfoo -Ab foo + tap_finish -- 2.23.0

3 4

[pacman-dev] [PATCH] Update NEWS for pacman-5.2.1
by Allan McRae 31 Oct '19

31 Oct '19

Signed-off-by: Allan McRae <allan(a)archlinux.org> --- I will release 5.2.1 off the master branch when I next need to procrastinate writing a fellowship application (could be in a day, could be in an hour...). Then the release/5.2.x branch will be cut. NEWS | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/NEWS b/NEWS index 9ceff259..6510de6d 100644 --- a/NEWS +++ b/NEWS @@ -1,5 +1,16 @@ VERSION DESCRIPTION ----------------------------------------------------------------------------- +5.2.1 - fix segfault on importing PGP keys for -U operations + - fix distribution of meson files + - fix inode command for darwin/bsd + - distribute all documentation files + - update bash/zsh completion for -F changes, remove --force + - makepkg: + - fix calculation of package sizes in presence of hardlinks + - do not warn about PACKAGER format if not set + - only run --clean when a package is built + - repo-add: + - fix compression of databases with zstd 5.2.0 - completely remove delta support (CVE-2019-18183) - add support to pacman and pacman-key for downloading PGP signing keys using the WKD protocol (FS#63171) -- 2.23.0

1 0

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.2.0-14-g4d76cf6f
by Allan McRae 30 Oct '19

30 Oct '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 4d76cf6f074854c6c7fbd6b3a279c1a0c123d8c8 (commit) from 866a5cd431e35803f85ad7a8045f0050c0b240e5 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 4d76cf6f074854c6c7fbd6b3a279c1a0c123d8c8 Author: Allan McRae <allan(a)archlinux.org> Date: Wed Oct 30 14:24:47 2019 +1000 Comma fail ----------------------------------------------------------------------- Summary of changes: scripts/libmakepkg/util/meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.2.0-13-g866a5cd4
by Allan McRae 30 Oct '19

30 Oct '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 866a5cd431e35803f85ad7a8045f0050c0b240e5 (commit) via 5c2059db28cc8d0dfa96f288517f2e10d1b6137a (commit) via 0272fca993718460bf7ecb7fdc3ca7dad1c7e6cd (commit) via a745d97c174ef1d85649b2a8ad7ca22e3dea9e58 (commit) via 99639dc27c4c037b3fe7ddf2b77d036a7332b5e3 (commit) from ae5cf26b5b5b92c9f68b5b29787e7d636f334c12 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 866a5cd431e35803f85ad7a8045f0050c0b240e5 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Oct 24 02:54:31 2019 -0400 autotools: be more templated when getting list of doc dist files Distribute asciidoc sources for all manpages instead of remembering to add files to both variables. Fixes regression in 377d47142f7aaa01ca782e6587f2d4caf663865b which broke building the website from a dist tarball: make: *** No rule to make target 'pacman-conf.8.html', needed by 'html'. Stop. (Technically this regression is already fixed by commit 942b909829d529409216939b36af11e8480726f5, but this is just going to keep happening, I suspect, so we should fix the root cause.) Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 5c2059db28cc8d0dfa96f288517f2e10d1b6137a Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon Oct 28 11:37:39 2019 -0400 makepkg: only run --clean actions if we built a package Fixes issue where users were allowed to run cleanup while running --geninteg or --printsrcinfo or --packagelist, thus mixing invalid responses into stdout. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 0272fca993718460bf7ecb7fdc3ca7dad1c7e6cd Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Oct 27 03:41:46 2019 -0400 makepkg: do not count hard links multiple times when calculating pkg size Exclude files with hardlinks when cat'ing all the files, and do a second run to look at each file with hardlinks, keep track of the ones we've already operated on, and only cat each inode once. Then use "wc -c" to get the size of all (deduplicated) files the same way we were already doing. Original-patch-by: Ronan Pigott <rpigott(a)berkeley.edu> Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit a745d97c174ef1d85649b2a8ad7ca22e3dea9e58 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Oct 27 03:05:10 2019 -0400 makepkg: protect against unexpected whitespace in filenames zipman: read -r protects against those evil manpages whose filenames contain backslash escapes, (muahahaha?) IFS= read protects against filenames with: - leading whitespace (but no one is actually stupid enough to configure their MAN_DIRS=() in makepkg.conf with such silly directories, *right*?) - trailing whitespace (but likewise, no one should be stupid enough to write an uncompressed manpage for section '1 ' or something) Also fix several other cases where we read filenames without protecting against surrounding whitespace, or without using null-delimited filenames when we could trivially do so. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 99639dc27c4c037b3fe7ddf2b77d036a7332b5e3 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Oct 27 02:11:28 2019 -0400 meson: fix inodecmd for darwin/bsd The BSD stat command uses %N, not %n, and was incorrectly ported to meson. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: doc/Makefile.am | 15 +------ meson.build | 4 +- scripts/Makefile.am | 1 + scripts/libmakepkg/tidy/staticlibs.sh.in | 2 +- scripts/libmakepkg/tidy/strip.sh.in | 6 +-- scripts/libmakepkg/tidy/zipman.sh.in | 8 ++-- .../{lint_config/ext.sh.in => util/dirsize.sh.in} | 48 ++++++++++------------ scripts/libmakepkg/util/meson.build | 1 + scripts/makepkg.sh.in | 10 ++--- 9 files changed, 41 insertions(+), 54 deletions(-) copy scripts/libmakepkg/{lint_config/ext.sh.in => util/dirsize.sh.in} (54%) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.2.0-8-gae5cf26b
by Allan McRae 29 Oct '19

29 Oct '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via ae5cf26b5b5b92c9f68b5b29787e7d636f334c12 (commit) via dc55701132abf51de2f4289488f416a4c07afecf (commit) from f37a3752b38473236720e01230392377a42249e6 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit ae5cf26b5b5b92c9f68b5b29787e7d636f334c12 Author: Allan McRae <allan(a)archlinux.org> Date: Thu Oct 24 23:38:50 2019 +1000 Fix segfault importing PGP key for pacman -U operations Use after free. Signed-off-by: Allan McRae <allan(a)archlinux.org> commit dc55701132abf51de2f4289488f416a4c07afecf Author: Evangelos Foutras <evangelos(a)foutrelis.com> Date: Thu Oct 24 10:16:04 2019 +0300 Add pacman-conf(8) to the documentation index Signed-off-by: Allan McRae <allan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: doc/index.asciidoc | 1 + lib/libalpm/be_package.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [PATCH 1/1] makepkg: send output from clean_up to stderr
by Christian Hesse 29 Oct '19

29 Oct '19

From: Christian Hesse <mail(a)eworm.de> This is required when redirecting checksums to PKGBUILD: makepkg --clean --geninteg >> PKGBUILD Signed-off-by: Christian Hesse <mail(a)eworm.de> --- scripts/makepkg.sh.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 183e1a3e..3017f31b 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1101,7 +1101,7 @@ while [[ $1 ]]; do done # setup signal traps -trap 'clean_up' 0 +trap 'clean_up >&2' 0 for signal in TERM HUP QUIT; do trap "trap_exit $signal \"$(gettext "%s signal caught. Exiting...")\" \"$signal\"" "$signal" done

2 2