October 2020 - Pacman-dev - lists.archlinux.org

[pacman-dev] [PATCH] makepkg: --source should download repos with PGP signatures
by Eli Schwartz 08 Apr '21

08 Apr '21

We optimize this out for sourceballs since VCS sources don't get their checksums verified. But this logic is broken ever since we implemented PGP signature checking for git sources -- if the git source is signed, we still check it, but we don't make sure to download it first. makepkg then fails to generate a sourceball unless you previously ran --verifysource or attempted to build. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- scripts/libmakepkg/source.sh.in | 5 ++++- scripts/libmakepkg/source/git.sh.in | 9 ++++++--- scripts/makepkg.sh.in | 2 +- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/scripts/libmakepkg/source.sh.in b/scripts/libmakepkg/source.sh.in index a0c6b662..b95e6be8 100644 --- a/scripts/libmakepkg/source.sh.in +++ b/scripts/libmakepkg/source.sh.in @@ -35,7 +35,7 @@ done download_sources() { local netfile all_sources - local get_source_fn=get_all_sources_for_arch get_vcs=1 + local get_source_fn=get_all_sources_for_arch get_vcs=1 get_pgp=0 msg "$(gettext "Retrieving sources...")" @@ -47,6 +47,9 @@ download_sources() { novcs) get_vcs=0 ;; + getpgp) + (( SKIPPGPCHECK )) || get_pgp=1 + ;; *) break ;; diff --git a/scripts/libmakepkg/source/git.sh.in b/scripts/libmakepkg/source/git.sh.in index 7d191b8d..d090f14e 100644 --- a/scripts/libmakepkg/source/git.sh.in +++ b/scripts/libmakepkg/source/git.sh.in @@ -29,13 +29,16 @@ source "$LIBRARY/util/pkgbuild.sh" download_git() { + local netfile=$1 + local query=$(get_uri_query "$netfile") + # abort early if parent says not to fetch if declare -p get_vcs > /dev/null 2>&1; then - (( get_vcs )) || return + if (( ! get_pgp )) || [[ $query != signed ]]; then + (( get_vcs )) || return + fi fi - local netfile=$1 - local dir=$(get_filepath "$netfile") [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 7e8d6805..c9940f0a 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1416,7 +1416,7 @@ if (( SOURCEONLY )); then download_sources allarch elif ( (( ! SKIPCHECKSUMS )) || \ ( (( ! SKIPPGPCHECK )) && source_has_signatures ) ); then - download_sources allarch novcs + download_sources allarch novcs getpgp fi check_source_integrity all cd_safe "$startdir" -- 2.28.0

2 1

[pacman-dev] [PATCH] makepkg: Don't double-layer distcc on ccache
by Matti Niemenmaa 08 Feb '21

08 Feb '21

From: Matti Niemenmaa <matti.niemenmaa+git(a)iki.fi> buildenv is set once for build() and a second time for package(). When using both distcc and ccache, this lead to CCACHE_PREFIX="distcc distcc" in package(), which breaks PKGBUILDs that execute the compiler in package() because distcc complains: distcc[383041] (main) CRITICAL! distcc seems to have invoked itself recursively! Avoid causing this error by only adding "distcc" to CCACHE_PREFIX if it's not yet there. Signed-off-by: Matti Niemenmaa <matti.niemenmaa+git(a)iki.fi> --- scripts/libmakepkg/buildenv/compiler.sh.in | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) ncurses is an example of a package that runs into this problem. Setting buildenv twice was done on purpose in commit 02a0bf550a22e199f48537b7eee87361b112e8a0, but at a glance I'm not sure whether it provides any benefit. All the meaningful changes are to environment variables that are exported from the parent process anyway, so all the changes are either repeated or even duplicated (like in PATH, or this CCACHE_PREFIX issue). Maybe it would be better to simply remove the prepare_buildenv call from the INFAKEROOT case in makepkg.sh.in? diff --git a/scripts/libmakepkg/buildenv/compiler.sh.in b/scripts/libmakepkg/buildenv/compiler.sh.in index 69f58a29..c93c77b4 100644 --- a/scripts/libmakepkg/buildenv/compiler.sh.in +++ b/scripts/libmakepkg/buildenv/compiler.sh.in @@ -44,7 +44,9 @@ buildenv_ccache() { buildenv_distcc() { if check_buildoption "distcc" "y"; then if (( using_ccache )); then - export CCACHE_PREFIX="${CCACHE_PREFIX:+$CCACHE_PREFIX }distcc" + if ! [[ "$CCACHE_PREFIX" =~ (^| )distcc($| ) ]]; then + export CCACHE_PREFIX="${CCACHE_PREFIX:+$CCACHE_PREFIX }distcc" + fi export CCACHE_BASEDIR="$srcdir" elif [[ -d /usr/lib/distcc/bin ]]; then export PATH="/usr/lib/distcc/bin:$PATH" -- 2.24.1

3 6

[pacman-dev] Alternatives system
by Allan McRae 07 Dec '20

07 Dec '20

Hi all, I plan to finish implementing an alternatives system for pacman post 5.2 release: https://wiki.archlinux.org/index.php/User:Allan/Alternatives Any comments or suggestions? Thanks, Allan

6 7

[pacman-dev] [PATCH] lock file may be left behind when a signal arrives while in cleanup
by lilydjwg 03 Dec '20

03 Dec '20

When a SIGINT or SIGHUP arrives before alpm_unlock (in alpm_release) but after remove_soft_interrupt_handler, the lock file will be left behind. Signed-off-by: lilydjwg <lilydjwg(a)gmail.com> --- src/pacman/pacman.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c index fefd3fa4..2d2e41d2 100644 --- a/src/pacman/pacman.c +++ b/src/pacman/pacman.c @@ -287,7 +287,6 @@ static void setuseragent(void) */ static void cleanup(int ret) { - remove_soft_interrupt_handler(); if(config) { /* free alpm library resources */ if(config->handle && alpm_release(config->handle) == -1) { @@ -297,6 +296,7 @@ static void cleanup(int ret) config_free(config); config = NULL; } + remove_soft_interrupt_handler(); /* free memory */ FREELIST(pm_targets); -- 2.28.0

2 4

[pacman-dev] [PATCH v2] libmakepkg: lint all arrays for empty values
by morganamilo 26 Nov '20

26 Nov '20

Empty values break pacman's db format, as an empty line indicates an end of section. --- I've seen this out in the wild: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=vim-coc-highlight-gi… diff --git a/scripts/libmakepkg/lint_pkgbuild/variable.sh.in b/scripts/libmakepkg/lint_pkgbuild/variable.sh.in index 1bc49722..22f5fbbb 100644 --- a/scripts/libmakepkg/lint_pkgbuild/variable.sh.in +++ b/scripts/libmakepkg/lint_pkgbuild/variable.sh.in @@ -31,7 +31,7 @@ lint_pkgbuild_functions+=('lint_variable') lint_variable() { - local i a pkg out bad ret=0 + local i a pkg out bad array var ret=0 # global variables for i in ${pkgbuild_schema_arrays[@]}; do @@ -93,5 +93,62 @@ lint_variable() { done done + # ensure lists don't contain empty values + for i in ${pkgbuild_schema_arrays[@]}; do + if declare -p $i > /dev/null 2>&1; then + array_build "array" "$i" + for var in "${array[@]}"; do + if [[ -z "$var" ]]; then + error "$(gettext "%s is not allowed to be empty")" "$i" + ret=1 + fi + done + fi + done + + for a in ${arch[@]}; do + [[ $a == "any" ]] && continue + + for i in ${pkgbuild_schema_arch_arrays[@]}; do + if declare -p "${i}_${a}" > /dev/null 2>&1; then + array_build "array" "${i}_${a}" + for var in "${array[@]}"; do + if [[ -z "$var" ]]; then + error "$(gettext "%s is not allowed to be empty")" "${i}_${a}" + ret=1 + fi + done + fi + done + done + + for pkg in ${pkgname[@]}; do + for i in ${pkgbuild_schema_arrays[@]}; do + if extract_function_variable "package_$pkg" $i 1 out; then + for val in "${out[@]}" ;do + if [[ -z "$val" ]]; then + error "$(gettext "%s is not allowed to be empty")" "$i" + ret=1 + fi + done + fi + done + + for a in ${arch[@]}; do + [[ $a == "any" ]] && continue + + for i in ${pkgbuild_schema_arch_arrays[@]}; do + if extract_function_variable "package_$pkg" "${i}_${a}" 1 out; then + for val in "${out[@]}" ;do + if [[ -z "$val" ]]; then + error "$(gettext "%s is not allowed to be empty")" "${i}_${a}" + ret=1 + fi + done + fi + done + done + done + return $ret } -- 2.29.1

2 1

[pacman-dev] [PATCH v2] Go to the end of screen if 'printonly' mode enabled
by Anatol Pomozov 26 Nov '20

26 Nov '20

At the end of download operation our code makes sure the cursor is moved to the end of the drawing area. But 'printonly' mode has its own if() branch that skips this cursor alignment. Add cursor_goto_end() to the 'printonly' codepath to make sure it does not clobber previous output. Fixes FS#68355 Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> --- src/pacman/callback.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/pacman/callback.c b/src/pacman/callback.c index 4240a779..53518101 100644 --- a/src/pacman/callback.c +++ b/src/pacman/callback.c @@ -226,6 +226,7 @@ static int number_length(size_t n) void cb_event(alpm_event_t *event) { if(config->print) { + cursor_goto_end(); return; } switch(event->type) { -- 2.29.0

2 1

[pacman-dev] [PATCH] Support parallel download with xfercommand
by lesto 31 Oct '20

31 Oct '20

Signed-off-by: lesto <lestofante88(a)gmail.com> --- lib/libalpm/alpm.h | 10 ++++++++++ lib/libalpm/dload.c | 9 +++++++++ lib/libalpm/handle.c | 13 +++++++++++++ lib/libalpm/handle.h | 1 + src/pacman/conf.c | 5 ++++- src/pacman/conf.h | 1 + src/pacman/pacman-conf.c | 3 +++ 7 files changed, 41 insertions(+), 1 deletion(-) diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index 614a530c..c2af60e8 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -755,6 +755,11 @@ typedef void (*alpm_cb_totaldl)(off_t total); typedef int (*alpm_cb_fetch)(const char *url, const char *localpath, int force); +/** A callback for waiting for download of files + * @return 0 on success, -1 on error. + */ +typedef int (*alpm_cb_fetch_lock)(void); + /** Fetch a list of remote packages. * @param handle the context handle * @param urls list of package URLs to download @@ -787,6 +792,11 @@ alpm_cb_fetch alpm_option_get_fetchcb(alpm_handle_t *handle); /** Sets the downloading callback. */ int alpm_option_set_fetchcb(alpm_handle_t *handle, alpm_cb_fetch cb); +/** Returns the downloading lock callback. */ +alpm_cb_fetch_lock alpm_option_get_fetch_lockcb(alpm_handle_t *handle); +/** Sets the downloading lock callback. */ +int alpm_option_set_fetch_lockcb(alpm_handle_t *handle, alpm_cb_fetch_lock cb); + /** Returns the callback used to report total download size. */ alpm_cb_totaldl alpm_option_get_totaldlcb(alpm_handle_t *handle); /** Sets the callback used to report total download size. */ diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 673e769f..174d559d 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -824,6 +824,15 @@ int _alpm_download(alpm_handle_t *handle, RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1); } } + + if (handle->fetch_lockcb != NULL) { + // if fetch_lockcb is set, fetchcb is non-blocking; here we wait for all download to complete + int ret = handle->fetch_lockcb(); + if (ret == -1) { + RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1); + } + } + return 0; } } diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c index 1310601a..683e678d 100644 --- a/lib/libalpm/handle.c +++ b/lib/libalpm/handle.c @@ -174,6 +174,12 @@ alpm_cb_fetch SYMEXPORT alpm_option_get_fetchcb(alpm_handle_t *handle) return handle->fetchcb; } +alpm_cb_fetch_lock SYMEXPORT alpm_option_get_fetch_lockcb(alpm_handle_t *handle) +{ + CHECK_HANDLE(handle, return NULL); + return handle->fetch_lockcb; +} + alpm_cb_totaldl SYMEXPORT alpm_option_get_totaldlcb(alpm_handle_t *handle) { CHECK_HANDLE(handle, return NULL); @@ -321,6 +327,13 @@ int SYMEXPORT alpm_option_set_fetchcb(alpm_handle_t *handle, alpm_cb_fetch cb) return 0; } +int SYMEXPORT alpm_option_set_fetch_lockcb(alpm_handle_t *handle, alpm_cb_fetch_lock cb) +{ + CHECK_HANDLE(handle, return -1); + handle->fetch_lockcb = cb; + return 0; +} + int SYMEXPORT alpm_option_set_totaldlcb(alpm_handle_t *handle, alpm_cb_totaldl cb) { CHECK_HANDLE(handle, return -1); diff --git a/lib/libalpm/handle.h b/lib/libalpm/handle.h index 9fef0fbf..dc00751b 100644 --- a/lib/libalpm/handle.h +++ b/lib/libalpm/handle.h @@ -73,6 +73,7 @@ struct __alpm_handle_t { alpm_cb_download dlcb; /* Download callback function */ alpm_cb_totaldl totaldlcb; /* Total download callback function */ alpm_cb_fetch fetchcb; /* Download file callback function */ + alpm_cb_fetch_lock fetch_lockcb; /* Download lock file callback function */ alpm_cb_event eventcb; alpm_cb_question questioncb; alpm_cb_progress progresscb; diff --git a/src/pacman/conf.c b/src/pacman/conf.c index 3a3ef605..53de73b8 100644 --- a/src/pacman/conf.c +++ b/src/pacman/conf.c @@ -157,6 +157,7 @@ int config_free(config_t *oldconfig) FREELIST(oldconfig->hookdirs); FREELIST(oldconfig->cachedirs); free(oldconfig->xfercommand); + free(oldconfig->xferlockcommand); free(oldconfig->print_format); free(oldconfig->arch); wordsplit_free(oldconfig->xfercommand_argv); @@ -319,7 +320,9 @@ static int download_with_xfercommand(const char *url, const char *localpath, for(i = 0; i <= config->xfercommand_argc; i++) { const char *val = config->xfercommand_argv[i]; if(val && strcmp(val, "%o") == 0) { - usepart = 1; + if (config->xferlockcommand == NULL) { + usepart = 1; + } val = tempfile; } else if(val && strcmp(val, "%u") == 0) { val = url; diff --git a/src/pacman/conf.h b/src/pacman/conf.h index b8a451ad..1a9d637d 100644 --- a/src/pacman/conf.h +++ b/src/pacman/conf.h @@ -130,6 +130,7 @@ typedef struct __config_t { char *xfercommand; char **xfercommand_argv; size_t xfercommand_argc; + char *xferlockcommand; /* our connection to libalpm */ alpm_handle_t *handle; diff --git a/src/pacman/pacman-conf.c b/src/pacman/pacman-conf.c index 463badf1..7c4f4cc9 100644 --- a/src/pacman/pacman-conf.c +++ b/src/pacman/pacman-conf.c @@ -259,6 +259,7 @@ static void dump_config(void) show_str("Architecture", config->arch); show_str("XferCommand", config->xfercommand); + show_str("XferLockCommand", config->xferlockcommand); show_bool("UseSyslog", config->usesyslog); show_bool("Color", config->color); @@ -366,6 +367,8 @@ static int list_directives(void) show_str("Architecture", config->arch); } else if(strcasecmp(i->data, "XferCommand") == 0) { show_str("XferCommand", config->xfercommand); + } else if(strcasecmp(i->data, "XferLockCommand") == 0) { + show_str("XferLockCommand", config->xferlockcommand); } else if(strcasecmp(i->data, "UseSyslog") == 0) { show_bool("UseSyslog", config->usesyslog); -- 2.28.0

4 7

[pacman-dev] [PATCH] libmakepkg: lint all arrays for empty values
by morganamilo 28 Oct '20

28 Oct '20

--- I've seen this out in the wild: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=vim-coc-highlight-gi… diff --git a/scripts/libmakepkg/lint_pkgbuild/variable.sh.in b/scripts/libmakepkg/lint_pkgbuild/variable.sh.in index 1bc49722..22f5fbbb 100644 --- a/scripts/libmakepkg/lint_pkgbuild/variable.sh.in +++ b/scripts/libmakepkg/lint_pkgbuild/variable.sh.in @@ -31,7 +31,7 @@ lint_pkgbuild_functions+=('lint_variable') lint_variable() { - local i a pkg out bad ret=0 + local i a pkg out bad array var ret=0 # global variables for i in ${pkgbuild_schema_arrays[@]}; do @@ -93,5 +93,62 @@ lint_variable() { done done + # ensure lists don't contain empty values + for i in ${pkgbuild_schema_arrays[@]}; do + if declare -p $i > /dev/null 2>&1; then + array_build "array" "$i" + for var in "${array[@]}"; do + if [[ -z "$var" ]]; then + error "$(gettext "%s is not allowed to be empty")" "$i" + ret=1 + fi + done + fi + done + + for a in ${arch[@]}; do + [[ $a == "any" ]] && continue + + for i in ${pkgbuild_schema_arch_arrays[@]}; do + if declare -p "${i}_${a}" > /dev/null 2>&1; then + array_build "array" "${i}_${a}" + for var in "${array[@]}"; do + if [[ -z "$var" ]]; then + error "$(gettext "%s is not allowed to be empty")" "${i}_${a}" + ret=1 + fi + done + fi + done + done + + for pkg in ${pkgname[@]}; do + for i in ${pkgbuild_schema_arrays[@]}; do + if extract_function_variable "package_$pkg" $i 1 out; then + for val in "${out[@]}" ;do + if [[ -z "$val" ]]; then + error "$(gettext "%s is not allowed to be empty")" "$i" + ret=1 + fi + done + fi + done + + for a in ${arch[@]}; do + [[ $a == "any" ]] && continue + + for i in ${pkgbuild_schema_arch_arrays[@]}; do + if extract_function_variable "package_$pkg" "${i}_${a}" 1 out; then + for val in "${out[@]}" ;do + if [[ -z "$val" ]]; then + error "$(gettext "%s is not allowed to be empty")" "${i}_${a}" + ret=1 + fi + done + fi + done + done + done + return $ret } -- 2.29.1

5 10

[pacman-dev] [PATCH] FS#68355 go to the end of screen if 'printonly' mode enabled
by Anatol Pomozov 22 Oct '20

22 Oct '20

At the end of download operation our code makes sure the cursor is moved to the end of the drawing area. But 'printonly' mode has its own if() branch that skips this cursor alignment. Add cursor_goto_end() to the 'printonly' codepath to make sure it does not clobber previous output. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> --- src/pacman/callback.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/pacman/callback.c b/src/pacman/callback.c index 4240a779..53518101 100644 --- a/src/pacman/callback.c +++ b/src/pacman/callback.c @@ -226,6 +226,7 @@ static int number_length(size_t n) void cb_event(alpm_event_t *event) { if(config->print) { + cursor_goto_end(); return; } switch(event->type) { -- 2.29.0

1 0

[pacman-dev] [PATCH v2] libmakepkg: compress: fix tar extension
by Michael Straube 21 Oct '20

21 Oct '20

With commit 74aacf44958e1343b910b3fbdcf753393857f070 creating uncompressed .tar packages fails. -> Compressing package... /usr/share/makepkg/util/compress.sh: line 70: COMPRESS.TAR[@]: invalid variable name bsdtar: Write error Empty the '$ext' variable for the '.tar' extension in get_compress_command() to fix this. We would fallback to cat for 'tar' anyways. Signed-off-by: Michael Straube <michael.straubej(a)gmail.com> --- v1 -> v2 Added a comment. scripts/libmakepkg/util/compress.sh.in | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/libmakepkg/util/compress.sh.in b/scripts/libmakepkg/util/compress.sh.in index d35a01fa..b88d0c2f 100644 --- a/scripts/libmakepkg/util/compress.sh.in +++ b/scripts/libmakepkg/util/compress.sh.in @@ -65,6 +65,10 @@ get_compression_command() { esac ext=${ext#*.tar.} + # empty the variable for tar so we fallback to cat, otherwise we would + # have '.tar' and get an error in the next if block + ext=${ext#*.tar} + if [[ -n $ext ]]; then extarray="COMPRESS${ext^^}[@]" resolvecmd=("${!extarray}") -- 2.28.0

1 0