October 2014 - Pacman-dev - lists.archlinux.org

[pacman-dev] [RFC] makepkg: introduce .SRCINFO files for source packages
by Dave Reisner 09 Nov '14

09 Nov '14

Similar to .PKGINFO, .SRCINFO provides structured metadata from the PKGBUILD to be included with source packages. The format is structured such that it contains a "pkgbase" and one to many "pkgname" sections. Each "pkgname" section represents an "output package", and inherits all of the attributes of the "pkgbase" section, and then can define their own additive fields. For example, a simple PKGBUILD: pkgbase=ponies pkgname=('applejack' 'pinkiepie') pkgver=1.2.3 pkgrel=1 arch=('x86_64' 'i686') depends=('friendship' 'magic') build() { ...; } package_applejack() { provides=('courage') ...; } package_pinkiepie() { provides=('laughter') ...; } Would yield the following .SRCINFO file: pkgbase = ponies pkgdesc = friendship is magic pkgver = 1.2.3 pkgrel = 1 arch = x86_64 arch = i686 depends = friendship depends = magic pkgname = applejack provides = courage pkgname = pinkiepie provides = laughter The code to generate this new file is taken a project which I've been incubating[0] under the guise of 'mkaurball', which creates .AURINFO files for the AUR. AURINFO is the exactly same file as .SRCINFO, but named as such to make it clear that this is specific to the AUR. Because we're parsing shell in the packaging functions rather than executing it, there *are* some limitations, but these only really crop up in more "exotic" PKGBUILDs. Smoketesting[1] for accuracy in the Arch repos yields 100% accuracy for [core] and [extra]. [community] clocks in at ~98% accuracy (.3% difference per PKGBUILD), largely due to silly haskell packages calling pacman from inside the PKGBUILD to determine dependencies. [multilib] currently shows about 92% accuracy -- a statistic which can be largely improved by utilizing the recently merged arch-specific attribute work. This is also a smaller repo so the numbers are somewhat inflated. In reality, this is only a .8% variance per PKGBUILD. Together, we can make PKGBUILD better. [0] https://github.com/falconindy/pkgbuild-introspection [1] https://github.com/falconindy/pkgbuild-introspection/blob/master/test/smoke… --- Based on Allan's recent email about the feature freeze being pushed back, I'm offering this up for discussion. scripts/makepkg.sh.in | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 8a67d94..0349e05 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -2031,6 +2031,99 @@ find_libprovides() { printf '%s\n' "${libprovides[@]}" } +srcinfo_open_section() { + printf '%s = %s\n' "$1" "$2" +} + +srcinfo_close_section() { + echo +} + +srcinfo_write_attr() { + # $1: attr name + # $2: attr values + + local attrname=$1 attrvalues=("${@:2}") + + # normalize whitespace, strip leading and trailing + attrvalues=("${attrvalues[@]//+([[:space:]])/ }") + attrvalues=("${attrvalues[@]#[[:space:]]}") + attrvalues=("${attrvalues[@]%[[:space:]]}") + + printf "\t$attrname = %s\n" "${attrvalues[@]}" +} + +pkgbuild_extract_to_srcinfo() { + # $1: pkgname + # $2: attr name + # $3: multivalued + + local pkgname=$1 attrname=$2 isarray=$3 outvalue= + + if pkgbuild_get_attribute "$pkgname" "$attrname" "$isarray" 'outvalue'; then + srcinfo_write_attr "$attrname" "${outvalue[@]}" + fi +} + +srcinfo_write_section_details() { + local attr package_arch a + local multivalued_arch_attrs=(source provides conflicts depends replaces + optdepends makedepends checkdepends) + + for attr in "${singlevalued[@]}"; do + pkgbuild_extract_to_srcinfo "$1" "$attr" 0 + done + + for attr in "${multivalued[@]}"; do + pkgbuild_extract_to_srcinfo "$1" "$attr" 1 + done + + pkgbuild_get_attribute "$1" 'arch' 1 'package_arch' + for a in "${package_arch[@]}"; do + # 'any' is special. there's no support for, e.g. depends_any. + [[ $a = any ]] && continue + + for attr in "${multivalued_arch_attrs[@]}"; do + pkgbuild_extract_to_srcinfo "$1" "${attr}_$a" 1 + done + done +} + +srcinfo_write_global() { + local singlevalued=(pkgdesc pkgver pkgrel epoch url install changelog) + local multivalued=(arch groups license checkdepends makedepends + depends optdepends provides conflicts replaces + noextract options backup + source {md5,sha{1,224,256,384,512}}sums) + + srcinfo_open_section 'pkgbase' "${pkgbase:-$pkgname}" + srcinfo_write_section_details '' + srcinfo_close_section +} + +srcinfo_write_package() { + local singlevalued=(pkgdesc url install changelog) + local multivalued=(arch groups license checkdepends depends optdepends + provides conflicts replaces options backup) + + srcinfo_open_section 'pkgname' "$1" + srcinfo_write_section_details "$1" + srcinfo_close_section +} + +write_srcinfo() { + local pkg + + printf "# Generated by makepkg %s\n" "$makepkg_version" + printf "# %s\n" "$(LC_ALL=C date -u)" + + srcinfo_write_global + + for pkg in "${pkgname[@]}"; do + srcinfo_write_package "$pkg" + done +} + write_pkginfo() { local builddate=$(date -u "+%s") if [[ -n $PACKAGER ]]; then @@ -2243,6 +2336,9 @@ create_srcpackage() { msg2 "$(gettext "Adding %s...")" "$BUILDSCRIPT" ln -s "${BUILDFILE}" "${srclinks}/${pkgbase}/${BUILDSCRIPT}" + msg2 "$(gettext "Generating %s file...")" .SRCINFO + write_srcinfo > "$srclinks/$pkgbase"/.SRCINFO + local file all_sources get_all_sources 'all_sources' -- 2.1.2

3 5

[pacman-dev] [PATCH] Exit with 0/SUCCESS if there are no packages to remove
by Eric Schultz 29 Oct '14

29 Oct '14

Previously the lack of candidate packages was considered an error and return 1/FAILURE but really this isn't an issue. Also, for systemd (and others) this flagged the instance as having failed for no good reason. Signed-off-by: Eric Schultz <eric(a)schultzter.ca> --- contrib/paccache.sh.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/paccache.sh.in b/contrib/paccache.sh.in index 71aee23..b3b553d 100644 --- a/contrib/paccache.sh.in +++ b/contrib/paccache.sh.in @@ -328,7 +328,7 @@ done if (( ! ${#candidates[*]} )); then msg 'no candidate packages found for pruning' - exit 1 + exit 0 fi # grab this prior to signature scavenging -- 2.1.2

1 0

[pacman-dev] [PATCH] makepkg: Use a recursive, shallow clone for git://
by Drew DeVault 29 Oct '14

29 Oct '14

makepkg: Use a recursive, shallow clone for git:// This clones git submodules, which are required to build most projects that use them. This change is harmless for packages that don't use them. This also modifies the initial clone to use --depth=1, which avoids checking out the entire history and saves some bandwidth and improves the time required to clone. Signed-off-by: Drew DeVault <sir(a)cmpwn.com> --- I'm still not used to submitting patches through a mailing list, so I apologize if I've done something wrong. Please don't hesitate to let me know if there's something I could do better. scripts/makepkg.sh.in | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 8a67d94..3b74d4d 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -542,7 +542,7 @@ download_git() { if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "git" - if ! git clone --mirror "$url" "$dir"; then + if ! git clone --depth=1 --mirror "$url" "$dir"; then error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "git" plain "$(gettext "Aborting...")" exit 1 @@ -612,6 +612,11 @@ extract_git() { plain "$(gettext "Aborting...")" exit 1 fi + if ! git submodule update --init --recursive ; then + error "$(gettext "Failure while initializing submodules of %s %s repo")" "${repo}" "git" + plain "$(gettext "Aborting...")" + exit 1 + fi fi popd &>/dev/null -- 2.1.2

6 10

[pacman-dev] [PATCH] Added option to return success when there are no candidates for removal
by Eric Schultz 28 Oct '14

28 Oct '14

Add the -n or --noneisok option to return success if there are no candidates for removal. This is useful when checking the exit code, like from a systemd unit, and it is okay when there are no packages to remove. Signed-off-by: Eric Schultz <eric(a)schultzter.ca> --- contrib/paccache.sh.in | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/contrib/paccache.sh.in b/contrib/paccache.sh.in index 71aee23..cf1ba15 100644 --- a/contrib/paccache.sh.in +++ b/contrib/paccache.sh.in @@ -24,7 +24,7 @@ declare -r myname='paccache' declare -r myver='@PACKAGE_VERSION@' declare -a cachedirs=() candidates=() cmdopts=() whitelist=() blacklist=() -declare -i delete=0 dryrun=0 filecount=0 move=0 needsroot=0 totalsaved=0 verbose=0 +declare -i delete=0 dryrun=0 filecount=0 move=0 needsroot=0 noneisok=0 totalsaved=0 verbose=0 declare delim=$'\n' keep=3 movedir= scanarch= USE_COLOR='y' @@ -192,6 +192,8 @@ Usage: ${myname} <operation> [options] [targets...] delimited. -k, --keep <num> keep "num" of each package in the cache (default: 3). --nocolor remove color from output. + -n, --noneisok returns 0 (no error) rather than 1 (failed) when there + are no candidate packages for removal -u, --uninstalled target uninstalled packages. -v, --verbose increase verbosity. specify up to 3 times. -z, --null use null delimiters for candidate names (only with -v @@ -205,9 +207,9 @@ version() { echo 'Copyright (C) 2011 Dave Reisner <dreisner(a)archlinux.org>' } -OPT_SHORT=':a:c:dfhi:k:m:rsuVvz' +OPT_SHORT=':a:c:dfhi:k:m:nrsuVvz' OPT_LONG=('arch:' 'cachedir:' 'dryrun' 'force' 'help' 'ignore:' 'keep:' 'move' - 'nocolor' 'remove' 'uninstalled' 'version' 'verbose' 'null') + 'nocolor' 'noneisok' 'remove' 'uninstalled' 'version' 'verbose' 'null') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then exit 1 @@ -249,6 +251,8 @@ while :; do shift ;; --nocolor) USE_COLOR='n' ;; + -n|--noneisok) + noneisok=1 ;; -m|--move) move=1 movedir=$2 shift ;; @@ -328,7 +332,11 @@ done if (( ! ${#candidates[*]} )); then msg 'no candidate packages found for pruning' - exit 1 + if (( noneisok )); then + exit 0 + else + exit 1 + fi fi # grab this prior to signature scavenging -- 2.1.2

2 1

Re: [pacman-dev] FS#42482 - [paccache] No candidates should not be an error
by Eric Schultz 28 Oct '14

28 Oct '14

> > ---------- Forwarded message ---------- > From: Florian Pritz <bluewind(a)xinu.at> > To: pacman-dev(a)archlinux.org > Cc: > Date: Tue, 28 Oct 2014 10:41:31 +0100 > Subject: Re: [pacman-dev] FS#42482 - [paccache] No candidates should not be an error > On 28.10.2014 03:44, Eric Schultz wrote: > > declare -a cachedirs=() candidates=() cmdopts=() whitelist=() blacklist=() > > -declare -i delete=0 dryrun=0 filecount=0 move=0 needsroot=0 totalsaved=0 > > verbose=0 > > +declare -i delete=0 dryrun=0 filecount=0 move=0 needsroot=0 noneisok=0 > > totalsaved=0 verbose=0 > > declare delim=$'\n' keep=3 movedir= scanarch= > > Dan tried to link this earlier but missed the l of html so here's the > link again: https://www.archlinux.org/pacman/submitting-patches.html > > Please use git-send-email and nothing else (especially don't paste a > patch in gmail) because that is bound to break formatting as can be seen > above (no indentation, wrapped lines). > > Also if you have any comments to add that are not part of a commit use > git send-email --annotate and add them between the --- marker and the > diffstat. > > BTT: I also don't see why paccache should exit 1. Looks like this was in > the initial version Dave wrote so maybe he can chime in? > Ok, I'll try to figure-out configuring git send-email given my network constraints. As for whether no candidates should just exit 0 I would agree. However since paccache with it's current behaviour is out in the wild I figured adding a flag that provides alternate behaviour would be more prudent than changing the current behaviour (in case anyone relies on it). I'll happily go back and change just one character in the original version and re-submit if you all agree that's what's to be done. -- Headed for the second star to the right and straight on 'til morning... Eric Schultz (aka schultzter) Read more at 2FatDads

1 0

Re: [pacman-dev] FS#42482 - [paccache] No candidates should not be an error
by Eric Schultz 28 Oct '14

28 Oct '14

I've written a patch for this feature request, I hope this is okay. >From c332ee7030fba07e6efe8e6635d3d1648a97ba13 Mon Sep 17 00:00:00 2001 From: Eric Schultz <eric(a)schultzter.ca> Date: Mon, 27 Oct 2014 22:24:23 -0400 Subject: [PATCH] Added option to return success when there are no candidates for removal Add the -n or --noneisok option to return success if there are no candidates for removal. This is useful when checking the exit code, like from a systemd unit, and it is okay when there are no packages to remove. Signed-off-by: Eric Schultz <eric(a)schultzter.ca> --- contrib/paccache.sh.in | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/contrib/paccache.sh.in b/contrib/paccache.sh.in index 71aee23..cf1ba15 100644 --- a/contrib/paccache.sh.in +++ b/contrib/paccache.sh.in @@ -24,7 +24,7 @@ declare -r myname='paccache' declare -r myver='@PACKAGE_VERSION@' declare -a cachedirs=() candidates=() cmdopts=() whitelist=() blacklist=() -declare -i delete=0 dryrun=0 filecount=0 move=0 needsroot=0 totalsaved=0 verbose=0 +declare -i delete=0 dryrun=0 filecount=0 move=0 needsroot=0 noneisok=0 totalsaved=0 verbose=0 declare delim=$'\n' keep=3 movedir= scanarch= USE_COLOR='y' @@ -192,6 +192,8 @@ Usage: ${myname} <operation> [options] [targets...] delimited. -k, --keep <num> keep "num" of each package in the cache (default: 3). --nocolor remove color from output. + -n, --noneisok returns 0 (no error) rather than 1 (failed) when there + are no candidate packages for removal -u, --uninstalled target uninstalled packages. -v, --verbose increase verbosity. specify up to 3 times. -z, --null use null delimiters for candidate names (only with -v @@ -205,9 +207,9 @@ version() { echo 'Copyright (C) 2011 Dave Reisner <dreisner(a)archlinux.org>' } -OPT_SHORT=':a:c:dfhi:k:m:rsuVvz' +OPT_SHORT=':a:c:dfhi:k:m:nrsuVvz' OPT_LONG=('arch:' 'cachedir:' 'dryrun' 'force' 'help' 'ignore:' 'keep:' 'move' - 'nocolor' 'remove' 'uninstalled' 'version' 'verbose' 'null') + 'nocolor' 'noneisok' 'remove' 'uninstalled' 'version' 'verbose' 'null') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then exit 1 @@ -249,6 +251,8 @@ while :; do shift ;; --nocolor) USE_COLOR='n' ;; + -n|--noneisok) + noneisok=1 ;; -m|--move) move=1 movedir=$2 shift ;; @@ -328,7 +332,11 @@ done if (( ! ${#candidates[*]} )); then msg 'no candidate packages found for pruning' - exit 1 + if (( noneisok )); then + exit 0 + else + exit 1 + fi fi # grab this prior to signature scavenging -- 2.1.2 -- Headed for the second star to the right and straight on 'til morning... Eric Schultz <http://schultzter.ca> (aka schultzter <http://twitter.com/schultzter>) Read more at 2FatDads <http://2fatdads.com/author/schultzter/>

3 2

[pacman-dev] FS#42482 - [paccache] No candidates should not be an error
by Eric Schultz 27 Oct '14

27 Oct '14

Good evening... I've implemented this feature request in paccache (as a flag that turns off error reporting when there's no candidates), and I've been using it locally without any issues. I was wondering how I should share the change - should I post the git diff here? Attach the changed paccache.sh.in file? Submit a git pull request? Thanks, -- Headed for the second star to the right and straight on 'til morning... Eric Schultz <http://schultzter.ca> (aka schultzter <http://twitter.com/schultzter>) Read more at 2FatDads <http://2fatdads.com/author/schultzter/>

2 1

[pacman-dev] [PATCH 1/6] pmdb: include directories in package file lists
by Andrew Gregory 23 Oct '14

23 Oct '14

Needed for checking directories with PKG_FILES. Signed-off-by: Andrew Gregory <andrew.gregory.8(a)gmail.com> --- test/pacman/pmdb.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/pacman/pmdb.py b/test/pacman/pmdb.py index 60e0e9c..aa14dff 100644 --- a/test/pacman/pmdb.py +++ b/test/pacman/pmdb.py @@ -172,7 +172,7 @@ def db_read(self, name): if line == "%FILES%": while line: line = fd.readline().strip("\n") - if line and line[-1] != "/": + if line: pkg.files.append(line) if line == "%BACKUP%": pkg.backup = _getsection(fd) -- 2.1.2

2 12

[pacman-dev] [PATCH] makepkg: always look for sources in source=()
by Dave Reisner 18 Oct '14

18 Oct '14

This regression snuck in during some reviewing of 963f7fe02fcb14 (arch-specific sources). We must always check the source=() array for sources. --- scripts/makepkg.sh.in | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 8a67d94..86f8a77 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1309,6 +1309,8 @@ verify_integrity_sums() { have_sources() { local a + (( ${#source[*]} )) && return 0 + case $1 in all) for a in "${arch[@]}"; do @@ -1316,9 +1318,7 @@ have_sources() { done ;; *) - if (( ${#source[*]} )) || array_build _ source_"$CARCH"; then - return 0 - fi + array_build _ source_"$CARCH" && return 0 ;; esac -- 2.1.2

1 0

[pacman-dev] Pacman-4.2 release plans
by Allan McRae 17 Oct '14

17 Oct '14

Hi all, We release pacman-4.1 ages ago and there are many bug fixes sitting on master that I am too lazy to backport. So I would like to see a push towards pacman-4.2. I propose the following. 2014-08-01: Feature freeze 2014-08-15: Code freeze 2014-08-31: Release I have the glibc release scheduled for the end of July, so that is where the feature freeze comes from. We have a number of bugs in the tracker and patches in patchwork fixing bugs or providing some polishing that need dealt with. The two (or more) week gap before the code freeze is dedicated to dealing with those. In preparation, can bugs be filed for anything that needs done before 4.2? The major thing that I know needs finished is figuring out how to adjust databases that have packages installed using the old directory symlink method. Our current roadmap (with several bugs that will be pushed further back) is here: https://bugs.archlinux.org/roadmap/proj3 Cheers, Allan @Lukáš: note the VCS incremental builds is listed in the roadmap so your patches will be dealt with soon! @Andrew: Do you need your ini/config reading patches in, or would it be OK to hold back until the next release (which I intend to focus only on adding hooks support and call pacman-5.0).

4 4