- Pacman-dev - lists.archlinux.org

[pacman-dev] [PATCH] bash-completion: support file redirection completions
by Eli Schwartz 19 Mar '19

19 Mar '19

The current completions don't properly handle redirection operators, and attempt to complete command completions rather than completing filenames to redirect to. bash-completion provides both _get_comp_words_by_ref and a higher-level wrapper _init_completion, but the latter provides handling of redirection operators, so switch to using that. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- scripts/completion/bash_completion.in | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/scripts/completion/bash_completion.in b/scripts/completion/bash_completion.in index c3249cd0..a1ab9fec 100644 --- a/scripts/completion/bash_completion.in +++ b/scripts/completion/bash_completion.in @@ -41,9 +41,9 @@ _pacman_keyids() { } _pacman_key() { - local o cur opts prev wantfiles - COMPREPLY=() - _get_comp_words_by_ref cur prev + local o opts wantfiles + local cur prev words cword + _init_completion || return opts=('add config delete edit-key export finger gpgdir help import import-trustdb init keyserver list-keys list-sigs lsign-key nocolor populate recv-keys refresh-keys updatedb @@ -72,9 +72,9 @@ _pacman_key() { _makepkg() { compopt +o default - local cur opts prev - COMPREPLY=() - _get_comp_words_by_ref cur prev + local opts + local cur prev words cword + _init_completion || return if [[ $prev = @(-p|--config) ]]; then compopt -o default elif [[ ! $prev =~ ^-(-(config|help|key|version)$|\w*[Vh]) ]]; then @@ -105,9 +105,9 @@ _pacman_repo_list() { _pacman() { compopt -o default - local common core cur database files prev query remove sync upgrade o - COMPREPLY=() - _get_comp_words_by_ref cur prev + local common core database files query remove sync upgrade o + local cur prev words cword + _init_completion || return database=('asdeps asexplicit') files=('list machinereadable owns search refresh regex' 'l o s x y') query=('changelog check deps explicit file foreign groups info list native owns -- 2.21.0

2 2

[pacman-dev] [PATCH] makepkg: use "shared" git clones when checking out sources
by Eli Schwartz 19 Mar '19

19 Mar '19

In order to cache sources offline, makpekg creates *two* copies of every git repo. This is a useful tradeoff for network time, but comes at the cost of increased disk space. Normally, git can smooth this over automagically. Whenever possible, git objects are hardlinked to save space, but this does not work when SRCDEST and BUILDDIR are on separate filesystems. When the repo in question is both very large (linux.git for example is 2.2 GB) and crosses filesystem boundaries, this results in a lot of extra disk space being used; the most likely scenario is where BUILDDIR is a tmpfs for bonus ouch. git(1) has a builtin feature which serves this case handily: the --shared flag will create the info/alternates file instructing git to not copy or hardlink or create objects/packs at all, but merely look for them in an external location (that being the source of the clone). The downside of using shared clones, is that if you modify and drop commits from the original repo, or simply delete the whole repo altogether, you break the copy. But we don't care about that here, because 1) the BUILDDIR copy is meant to be a temporary copy strictly derived via PKGBUILD syntax from the SRCDEST, and must be able to be recreated at any time, 2) if the SRCDEST disappears, makepkg will redownload it, thus restoring the objects needed by the BUILDDIR clone, 3) if the user does non-default things like hacking on the BUILDDIR copy then deleting and re-cloning the SRCDEST may result in momentary breakage, but ultimately should be fine -- the unique objects they created will be stored in the BUILDDIR copy. While it's theoretically possible that upstream will force-push to overwrite the base tree from which makepkg is building (which they should not do), *and* the user deleted their SRCDEST which they should not do, *and* they saved work in makepkg's working directory which they should not do either... ... this is an unlikely chain of events for which we should not care. Using --shared is therefore helpful in immediately useful ways and IMHO has no actual downsides; we should use it. An alternative implementation would be to use worktrees. I've rejected this since it is essentially the same as shared clones, except adding additional restrictions on the branch namespace, and could potentially break existing use cases such as manually handling the SRCDEST in order to share repositories with normal working copies. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- scripts/libmakepkg/source/git.sh.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/libmakepkg/source/git.sh.in b/scripts/libmakepkg/source/git.sh.in index 497a668c..96d79623 100644 --- a/scripts/libmakepkg/source/git.sh.in +++ b/scripts/libmakepkg/source/git.sh.in @@ -91,7 +91,7 @@ extract_git() { exit 1 fi cd_safe "$srcdir" - elif ! git clone "$dir" "${dir##*/}"; then + elif ! git clone -s "$dir" "${dir##*/}"; then error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" plain "$(gettext "Aborting...")" exit 1 -- 2.21.0

2 2

[pacman-dev] [PATCH 1/3] doc: describe when and why the prepare function should be used
by Eli Schwartz 19 Mar '19

19 Mar '19

We don't want people to run architecture/OS/makepkg.conf specific processes during prepare() and in fact it's been observed that makepkg by design doesn't even run prepare_buildenv() for it, so the prohibition against this is now baked into makepkg. Reflect this differentiation in the documentation on just what, exactly, a prepare() function is. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- This patch coincides with quequotion's clarification of the options=() array, so thanks for inspiring me to write this! I believe that both patches have merit independent of each other. doc/PKGBUILD.5.asciidoc | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/doc/PKGBUILD.5.asciidoc b/doc/PKGBUILD.5.asciidoc index e8ce691f..285627ab 100644 --- a/doc/PKGBUILD.5.asciidoc +++ b/doc/PKGBUILD.5.asciidoc @@ -342,9 +342,12 @@ files into the packaging directory, with optional `prepare()`, `build()`, and *prepare() Function*:: An optional `prepare()` function can be specified in which operations to prepare the sources for building, such as patching, are performed. This - function is run after the source extraction and before the `build()` - function. The `prepare()` function is skipped when source extraction - is skipped. + function is run exactly once, after the source extraction and before the + `build()` function. The `prepare()` function is skipped when source + extraction is skipped. No system-specific or build-specific commands should + be run during `prepare()` under any circumstances, as they are meant to run + exclusively during `build()`, and features like `buildflags` or `makeflags` + are expressly not available. *build() Function*:: The optional `build()` function is use to compile and/or adjust the source -- 2.21.0

2 6

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.1.1-158-g9f5412d8
by Allan McRae 19 Mar '19

19 Mar '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 9f5412d809a50a7ad18013222582d5b943216ae1 (commit) via 7b15918b6526b52823b869050e430b24b30c3ca1 (commit) via ee5f24e50ca5792d37cfbdcbb134fb58777d3eb8 (commit) via 35a0d5e744d1c3f93d4f4f79b2ca2e3e411fdd5a (commit) via 0a72874734ceafdf0a9f9e7a96c8b3f88507a54b (commit) via 226d2c124884c7318b7fb86c2e6bdf792eddd1a9 (commit) via bbfb75fd292e3f2818cdc0eae68c292bb3fbc50b (commit) via f26cb61cb6a16c8ce85f33e6090763aced0118c3 (commit) via c61cd050f9b6475ea128a1da02e88b284ac509b5 (commit) from c0e9be7973be6c81b22fde91516fb8991e7bb07b (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 9f5412d809a50a7ad18013222582d5b943216ae1 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Tue Mar 12 13:01:13 2019 -0400 doc: generalize description of the bash nature of PKGBUILD functions Currently this is scoped to the build() function, which is simply wrong as it equally applies to any function. Simply moving the paragraphs up to the main manpage section makes this clear. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 7b15918b6526b52823b869050e430b24b30c3ca1 Author: Que Quotion <quequotion(a)gmail.com> Date: Wed Mar 13 01:01:43 2019 +0900 Clarify that build_options only apply during build() Signed-off-by: Que Quotion <quequotion(a)gmail.com> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit ee5f24e50ca5792d37cfbdcbb134fb58777d3eb8 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Tue Mar 12 13:28:27 2019 -0400 libmakepkg: fix reporting of invalid archive extensions in compress.sh In commit 1825bd6716c2a51c92642e8b96beac0101e83805 this was split out from makepkg, but the warning was not properly migrated; $ext did not ever exist. As a result, no matter what you did, the only possible warning was: ==> WARNING: '' is not a valid archive extension. Fix to filter based on the presence of .tar in the argument, and building the $ext variable for all checking and messaging purposes within the function. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 35a0d5e744d1c3f93d4f4f79b2ca2e3e411fdd5a Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Mar 7 23:18:23 2019 -0500 makepkg: use "shared" git clones when checking out sources In order to cache sources offline, makepkg creates *two* copies of every git repo. This is a useful tradeoff for network time, but comes at the cost of increased disk space. Normally, git can smooth this over automagically. Whenever possible, git objects are hardlinked to save space, but this does not work when SRCDEST and BUILDDIR are on separate filesystems. When the repo in question is both very large (linux.git for example is 2.2 GB) and crosses filesystem boundaries, this results in a lot of extra disk space being used; the most likely scenario is where BUILDDIR is a tmpfs for bonus ouch. git(1) has a builtin feature which serves this case handily: the --shared flag will create the info/alternates file instructing git to not copy or hardlink or create objects/packs at all, but merely look for them in an external location (that being the source of the clone). The downside of using shared clones, is that if you modify and drop commits from the original repo, or simply delete the whole repo altogether, you break the copy. But we don't care about that here, because 1) the BUILDDIR copy is meant to be a temporary copy strictly derived via PKGBUILD syntax from the SRCDEST, and must be able to be recreated at any time, 2) if the SRCDEST disappears, makepkg will redownload it, thus restoring the objects needed by the BUILDDIR clone, 3) if the user does non-default things like hacking on the BUILDDIR copy then deleting and re-cloning the SRCDEST may result in momentary breakage, but ultimately should be fine -- the unique objects they created will be stored in the BUILDDIR copy. While it's theoretically possible that upstream will force-push to overwrite the base tree from which makepkg is building (which they should not do), *and* the user deleted their SRCDEST which they should not do, *and* they saved work in makepkg's working directory which they should not do either... ... this is an unlikely chain of events for which we should not care. Using --shared is therefore helpful in immediately useful ways and IMHO has no actual downsides; we should use it. An alternative implementation would be to use worktrees. I've rejected this since it is essentially the same as shared clones, except adding additional restrictions on the branch namespace, and could potentially break existing use cases such as manually handling the SRCDEST in order to share repositories with normal working copies. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 0a72874734ceafdf0a9f9e7a96c8b3f88507a54b Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Mar 7 23:10:38 2019 -0500 build: check for gpgme with pkg-config before gpgme-config gpgme in git master now supports pkg-config and with the next release we can and should prefer its use. However, retain the legacy code that enables building with older versions of gpgme, as a fallback. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 226d2c124884c7318b7fb86c2e6bdf792eddd1a9 Author: Santiago Torres <santiago(a)archlinux.org> Date: Mon Mar 11 21:18:16 2019 -0400 drop DU* config variables Since DUFLAGS and DUPATH are not needed anymore remove them from the source Signed-off-by: Santiago Torres <santiago(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit bbfb75fd292e3f2818cdc0eae68c292bb3fbc50b Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Mar 7 22:07:55 2019 -0500 build: remove references to variable replacements from pacman-optimize MODECMD and OWNERCMD are not used by pacman itself, so we don't need to check for and replace them now that pacman-optimize is removed. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit f26cb61cb6a16c8ce85f33e6090763aced0118c3 Author: Santiago Torres <santiago(a)archlinux.org> Date: Mon Mar 11 22:24:35 2019 -0400 Make makepkg compute sizes properly Makepkg used to use du --apparent-size to compute the size of the package. Unfortunately, this would result in different sizes depending on the filesystem used (e.g., btrfs vs ext4), which would affect reproducible builds. Use a wc-based approach to compute sizes Signed-off-by: Santiago Torres <santiago(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit c61cd050f9b6475ea128a1da02e88b284ac509b5 Author: Florian Wehner <florian(a)whnr.de> Date: Thu Mar 7 20:14:55 2019 -0500 Reformatting log timestamp to include time-zone The time logged is currently given as localtime without any timezone information. This is confusing in various scenarios. Examples: * If one is travelling across time-zones and the timestamps in the log appear out of order. * Comparing dates with `datediff` gives an offset by the time-zone This patch would reformat the time-stamp to a full ISO-8601 version. It includes the 'T' separating date and time including seconds. Old: [2019-03-04 16:15] New: [2019-03-04T16:15:45-05:00] Signed-off-by: Florian Wehner <florian(a)whnr.de> Signed-off-by: Allan McRae <allan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: build-aux/edit-script.sh.in | 4 -- configure.ac | 88 ++++++++++++++-------------------- doc/PKGBUILD.5.asciidoc | 25 +++++----- lib/libalpm/libalpm.pc.in | 4 +- lib/libalpm/log.c | 8 ++-- meson.build | 70 +++++++++++---------------- meson_options.txt | 3 -- scripts/Makefile.am | 4 -- scripts/libmakepkg/source/git.sh.in | 2 +- scripts/libmakepkg/util/compress.sh.in | 26 +++++----- scripts/makepkg.sh.in | 3 +- 11 files changed, 99 insertions(+), 138 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [PATCH] Clarify that build_options only apply during build()
by Que Quotion 19 Mar '19

19 Mar '19

From: Que Quotion <quequotion(a)gmail.com> The word 'build' can be vague: as a verb in can be used to describe the entire process of rendering a package tarball from a PKGBUILD, while on these lines it refers to a specific PKGBUILD function. We need to more effectively discourage packagers from doing things like generating makefiles in prepare(), where the scripts these options trigger are not called to set or unset their flags. Signed-off-by: Que Quotion <quequotion(a)gmail.com> --- doc/PKGBUILD.5.asciidoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/PKGBUILD.5.asciidoc b/doc/PKGBUILD.5.asciidoc index e8ce691f..416a86a7 100644 --- a/doc/PKGBUILD.5.asciidoc +++ b/doc/PKGBUILD.5.asciidoc @@ -296,23 +296,23 @@ underscore and the architecture name e.g., 'replaces_x86_64=()'. Compress man and info pages with gzip. *ccache*;; - Allow the use of ccache during build. More useful in its negative + Allow the use of ccache during build(). More useful in its negative form `!ccache` with select packages that have problems building with ccache. *distcc*;; - Allow the use of distcc during build. More useful in its negative + Allow the use of distcc during build(). More useful in its negative form `!distcc` with select packages that have problems building with distcc. *buildflags*;; Allow the use of user-specific buildflags (CPPFLAGS, CFLAGS, CXXFLAGS, - LDFLAGS) during build as specified in linkman:makepkg.conf[5]. More + LDFLAGS) during build() as specified in linkman:makepkg.conf[5]. More useful in its negative form `!buildflags` with select packages that have problems building with custom buildflags. *makeflags*;; - Allow the use of user-specific makeflags during build as specified + Allow the use of user-specific makeflags during build() as specified in linkman:makepkg.conf[5]. More useful in its negative form `!makeflags` with select packages that have problems building with custom makeflags such as `-j2` (or higher). -- 2.21.0

2 2

[pacman-dev] [PATCH 0/2] [WIP] Fix du filesize computation
by Santiago Torres 19 Mar '19

19 Mar '19

This patchset is a WIP that tries to address FS#61717[1]. This is my first patch for pacman, so I may be missing a couple of the heuristics of proper pacman develoment. Namely, I'm worried about the use of xargs (I went down this road because I wanted to have the minimal set of changes). I'm not sure what're the heuristics for what I can assume (e.g., can findutils be assumed to exist?). As a followup of this, I removed the configure.ac stuff related to DUPATH and DUFLAGS. I'm not sure if we'd want to add XARGSPATH or so. I opted to not do that just yet. Thanks, -Santiago. [1] https://bugs.archlinux.org/task/61717 Santiago Torres (2): Make makepkg compute sizes properly configure.ac: drop DU* config configure.ac | 10 ---------- scripts/makepkg.sh.in | 3 +-- 2 files changed, 1 insertion(+), 12 deletions(-) -- 2.21.0

4 9

[pacman-dev] [PATCH] Reformatting log timestamp to include time-zone
by Florian Wehner 19 Mar '19

19 Mar '19

The time logged is currently given as localtime without any time-zone information. This is confusing in various scenarios. Examples: * If one is travelling across time-zones and the timestamps in the log appear out of order. * Comparing dates with `datediff` gives an offset by the time-zone This patch would reformat the time-stamp to a full ISO-8601 version. It includes the 'T' separating date and time. This could be removed. Old: [2019-03-04 16:15] New: [2019-03-04T16:15-05:00] Signed-off-by: Florian Wehner <florian(a)whnr.de> --- lib/libalpm/log.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/lib/libalpm/log.c b/lib/libalpm/log.c index e46ad3c3..cf869a08 100644 --- a/lib/libalpm/log.c +++ b/lib/libalpm/log.c @@ -20,6 +20,7 @@ #include <stdio.h> #include <stdarg.h> +#include <stdlib.h> #include <errno.h> #include <syslog.h> @@ -37,12 +38,17 @@ static int _alpm_log_leader(FILE *f, const char *prefix) { time_t t = time(NULL); + int tz_h, tz_m; struct tm *tm = localtime(&t); + /* Calculate the timezone offset ±hh:mm */ + tz_h = tm->tm_gmtoff/3600; + tz_m = abs(tm->tm_gmtoff - (tz_h*3600))/60; + /* Use ISO-8601 date format */ - return fprintf(f, "[%04d-%02d-%02d %02d:%02d] [%s] ", + return fprintf(f, "[%04d-%02d-%02dT%02d:%02d%+03d:%02d] [%s] ", tm->tm_year + 1900, tm->tm_mon + 1, tm->tm_mday, - tm->tm_hour, tm->tm_min, prefix); + tm->tm_hour, tm->tm_min, tz_h, tz_m, prefix); } /** A printf-like function for logging. -- 2.21.0

4 8

[pacman-dev] New Optional Dependencies Aren't Logged.
by Ralph Corderoy 17 Mar '19

17 Mar '19

`pacman -Su' suggested an optional dependency of gst-plugins-bad. I looked to see what that was and didn't want it. Later, due to discussion on the local LUG IRC, I wanted to look up what package had the new optional depends but found /var/log/pacman.log didn't record it. Please consider logging them. Looking at the optional depends of all the upgraded packages, I think it was webkit2gtk, but if it were logged then I could check if there were any new optional depends in the more general case, e.g. looking back in time. -- Cheers, Ralph.

3 8

[pacman-dev] [PATCH] scripts: update gitignore for currently noisy files
by Eli Schwartz 13 Mar '19

13 Mar '19

Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- scripts/.gitignore | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/.gitignore b/scripts/.gitignore index b8bb1f83..c511f889 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -1,7 +1,9 @@ +.lib/ makepkg makepkg-template pacman-db-upgrade pacman-key +pkgdelta repo-add repo-elephant repo-remove -- 2.21.0

1 1

[pacman-dev] [PATCH] repo-add: use more libmakepkg to handle common compression routines
by Eli Schwartz 12 Mar '19

12 Mar '19

Currently the list of supported formats for an archive, is maintained in two places. And repo-add does not actually get updated. :( In the process, remove some of the logical duplication when calling bsdtar/compress_as. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- scripts/repo-add.sh.in | 27 +++++++++------------------ 1 file changed, 9 insertions(+), 18 deletions(-) diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index 57413df5..0f9628ad 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -44,6 +44,7 @@ CLEAN_LOCK=0 USE_COLOR='y' # Import libmakepkg +source "$LIBRARY"/util/compress.sh source "$LIBRARY"/util/message.sh # ensure we have a sane umask set @@ -188,21 +189,12 @@ verify_signature() { } verify_repo_extension() { - local repofile=$1 - - case $repofile in - *.db.tar.gz) TAR_OPT="z" ;; - *.db.tar.bz2) TAR_OPT="j" ;; - *.db.tar.xz) TAR_OPT="J" ;; - *.db.tar.zst) TAR_OPT="--zstd" ;; - *.db.tar.Z) TAR_OPT="Z" ;; - *.db.tar) TAR_OPT="" ;; + case $1 in + *.db.tar.*|*.db.tar) ;; *) error "$(gettext "'%s' does not have a valid database archive extension.")" \ - "$repofile" + "$1" exit 1 ;; esac - - printf '%s' "$TAR_OPT" } # write an entry to the pacman database @@ -513,7 +505,6 @@ rotate_db() { } create_db() { - TAR_OPT=$(verify_repo_extension "$REPO_DB_FILE") # $LOCKFILE is already guaranteed to be absolute so this is safe dirname=${LOCKFILE%/*} @@ -523,13 +514,13 @@ create_db() { tempname=$dirname/.tmp.$filename pushd "$tmpdir/$repo" >/dev/null - if ( shopt -s nullglob; files=(*); (( ${#files[*]} )) ); then - bsdtar -c${TAR_OPT}f "$tempname" * - else + local files=(*) + if [[ ${files[*]} = '*' ]]; then # we have no packages remaining? zip up some emptyness warning "$(gettext "No packages remain, creating empty database.")" - bsdtar -c${TAR_OPT}f "$tempname" -T /dev/null + files=(-T /dev/null) fi + bsdtar -cf - "${files[@]}" | compress_as "$REPO_DB_SUFFIX" > "$tempname" popd >/dev/null create_signature "$tempname" @@ -644,7 +635,7 @@ else LOCKFILE=$PWD/$REPO_DB_FILE.lck fi -verify_repo_extension "$REPO_DB_FILE" >/dev/null +verify_repo_extension "$REPO_DB_FILE" REPO_DB_PREFIX=${REPO_DB_FILE##*/} REPO_DB_PREFIX=${REPO_DB_PREFIX%.db.*} -- 2.21.0

3 4