December 2018 - Arch-projects - lists.archlinux.org

[arch-projects] [devtools] [PATCH 1/1] arch-nspawn: get all mirrors from host
by Christian Hesse 22 Mar '19

22 Mar '19

From: Christian Hesse <mail(a)eworm.de> Now that pacconf gives us all mirrors we can use them, instead of just the first one. Signed-off-by: Christian Hesse <mail(a)eworm.de> --- arch-nspawn.in | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/arch-nspawn.in b/arch-nspawn.in index b9c846e..661bbbd 100644 --- a/arch-nspawn.in +++ b/arch-nspawn.in @@ -60,17 +60,19 @@ fi pacconf_cmd=$(command -v pacman-conf || command -v pacconf) # shellcheck disable=2016 -host_mirror=$($pacconf_cmd --repo extra Server 2> /dev/null | head -1 | sed -r 's#(.*/)extra/os/.*#\1$repo/os/$arch#') +host_mirrors=($($pacconf_cmd --repo extra Server 2> /dev/null | sed -r 's#(.*/)extra/os/.*#\1$repo/os/$arch#')) # shellcheck disable=2016 -[[ $host_mirror == *file://* ]] && host_mirror_path=$(echo "$host_mirror" | sed -r 's#file://(/.*)/\$repo/os/\$arch#\1#g') # {{{ functions build_mount_args() { declare -g mount_args=() - if [[ -n $host_mirror_path ]]; then - mount_args+=("--bind-ro=$host_mirror_path") - fi + for host_mirror in "${host_mirrors[@]}"; do + if [[ $host_mirror == *file://* ]]; then + host_mirror_path=$(echo "$host_mirror" | sed -r 's#file://(/.*)/\$repo/os/\$arch#\1#g') + mount_args+=("--bind-ro=$host_mirror_path") + fi + done mount_args+=("--bind=${cache_dirs[0]}") @@ -81,7 +83,10 @@ build_mount_args() { copy_hostconf () { cp -a /etc/pacman.d/gnupg "$working_dir/etc/pacman.d" - echo "Server = $host_mirror" >"$working_dir/etc/pacman.d/mirrorlist" + + for host_mirror in "${host_mirrors[@]}"; do + echo "Server = $host_mirror" + done >"$working_dir/etc/pacman.d/mirrorlist" [[ -n $pac_conf ]] && cp "$pac_conf" "$working_dir/etc/pacman.conf" [[ -n $makepkg_conf ]] && cp "$makepkg_conf" "$working_dir/etc/makepkg.conf"

1 1

[arch-projects] [dbscripts] [PATCH 1/3] testing2x: be more generic and accept multiple testing repos
by Eli Schwartz 10 Jan '19

10 Jan '19

Currently multilib is a second-class citizen the way it is lumped into community, and dbscripts cannot even keep track of whether it constitutes a testing repo. Teach config to track both testing and staging repos just like the stable ones, and teach testing2x to search through TESTING_REPOS to determine which one it is operating on. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- config | 3 ++- config.local.svn-community | 3 ++- config.local.svn-packages | 3 ++- test/lib/common.bash | 3 ++- testing2x | 26 ++++++++++++++++---------- 5 files changed, 24 insertions(+), 14 deletions(-) diff --git a/config b/config index 1cfe11f4..4c76558b 100644 --- a/config +++ b/config @@ -4,7 +4,8 @@ FTP_BASE="/srv/ftp" PKGREPOS=() PKGPOOL='' SRCPOOL='' -TESTING_REPO='' +STAGING_REPOS=() +TESTING_REPOS=() STABLE_REPOS=() # VCS backend diff --git a/config.local.svn-community b/config.local.svn-community index 5d61b5ea..493f34e3 100644 --- a/config.local.svn-community +++ b/config.local.svn-community @@ -5,7 +5,8 @@ PKGPOOL='pool/community' SRCPOOL='sources/community' SVNREPO='file:///srv/repos/svn-community/svn' SVNUSER='svn-community' -TESTING_REPO='community-testing' +STAGING_REPOS=('community-staging' 'multilib-staging') +TESTING_REPOS=('community-testing' 'multilib-testing') STABLE_REPOS=('community') CLEANUP_DESTDIR="/srv/repos/svn-community/package-cleanup" diff --git a/config.local.svn-packages b/config.local.svn-packages index 34aab35c..567d002e 100644 --- a/config.local.svn-packages +++ b/config.local.svn-packages @@ -5,7 +5,8 @@ PKGPOOL='pool/packages' SRCPOOL='sources/packages' SVNREPO='file:///srv/repos/svn-packages/svn' SVNUSER='svn-packages' -TESTING_REPO='testing' +STAGING_REPOS=('staging') +TESTING_REPOS=('testing') STABLE_REPOS=('core' 'extra') CLEANUP_DESTDIR="/srv/repos/svn-packages/package-cleanup" diff --git a/test/lib/common.bash b/test/lib/common.bash index bc2b4e6d..a92a01ee 100644 --- a/test/lib/common.bash +++ b/test/lib/common.bash @@ -103,7 +103,8 @@ setup() { PKGREPOS=('core' 'extra' 'testing') PKGPOOL='pool/packages' SRCPOOL='sources/packages' - TESTING_REPO='testing' + STAGING_REPOS=('staging') + TESTING_REPOS=('testing') STABLE_REPOS=('core' 'extra') CLEANUP_DESTDIR="${TMP}/package-cleanup" SOURCE_CLEANUP_DESTDIR="${TMP}/source-cleanup" diff --git a/testing2x b/testing2x index 14970a9f..215be39c 100755 --- a/testing2x +++ b/testing2x @@ -10,7 +10,7 @@ fi # Lock everything to reduce possibility of interfering task between the different repo-updates script_lock -for repo in "${TESTING_REPO}" "${STABLE_REPOS[@]}"; do +for repo in "${TESTING_REPOS[@]}" "${STABLE_REPOS[@]}"; do for pkgarch in "${ARCHES[@]}"; do repo_lock "${repo}" "${pkgarch}" || exit 1 done @@ -18,30 +18,36 @@ done declare -A pkgs +testing_repo= for pkgbase in "$@"; do for pkgarch in "${ARCHES[@]}"; do - vcsrepo_from="$(find_repo_for_package "${pkgbase}" "${pkgarch[@]}" "${TESTING_REPO}")" + vcsrepo_from="$(find_repo_for_package "${pkgbase}" "${pkgarch[@]}" "${TESTING_REPOS[@]}")" vcsrepo_to="$(find_repo_for_package "${pkgbase}" "${pkgarch[@]}" "${STABLE_REPOS[@]}")" + repo_from=${vcsrepo_from%-@(any|${pkgarch})} + repo_to="${vcsrepo_to%-@(any|${pkgarch})}" + if [[ -z ${testing_repo} ]]; then + testing_repo=${repo_from} + elif [[ ${testing_repo} != ${repo_from} ]]; then + die "Cannot move packages from multiple repos at a time: %s" "${testing_repo} ${repo_from}" + fi if [[ ${vcsrepo_from} && ${vcsrepo_to} ]]; then - move_to="${vcsrepo_to%-@(any|${pkgarch})}" - pkgs[${move_to}]+="${pkgbase} " + pkgs[${repo_to}]+="${pkgbase} " break fi done - [[ ${vcsrepo_from} ]] || die "%s not found in [%s]" "$pkgbase" "$TESTING_REPO" + [[ ${vcsrepo_from} ]] || die "%s not found in any of these repos: %s" "${pkgbase}" "${TESTING_REPOS[@]}" [[ ${vcsrepo_to} ]] || die "%s not found in any of these repos: %s" "$pkgbase" "${STABLE_REPOS[*]}" done -for pkgarch in "${ARCHES[@]}"; do - repo_unlock "${TESTING_REPO}" "${pkgarch}" -done -for repo in "${STABLE_REPOS[@]}"; do +for repo in "${TESTING_REPOS[@]}" "${STABLE_REPOS[@]}"; do for pkgarch in "${ARCHES[@]}"; do repo_unlock "${repo}" "${pkgarch}" done +done +for repo in "${STABLE_REPOS[@]}"; do if [[ -n ${pkgs[${repo}]} ]]; then - "$(dirname "$(readlink -e "$0")")/db-move" "${TESTING_REPO}" "${repo}" ${pkgs[${repo}]} + "$(dirname "$(readlink -e "$0")")/db-move" "${testing_repo}" "${repo}" ${pkgs[${repo}]} fi done -- 2.19.2

1 4

[arch-projects] [dbscripts] [PATCH 1/4] tests: make dummy copies of all pkgpool packages in the test environment
by Eli Schwartz 09 Jan '19

09 Jan '19

Prerequisite for reproducible archives of packages. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- test/lib/common.bash | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/test/lib/common.bash b/test/lib/common.bash index bc2b4e6d..3dda5f62 100644 --- a/test/lib/common.bash +++ b/test/lib/common.bash @@ -102,6 +102,7 @@ setup() { SVNREPO="file://${TMP}/svn-packages-repo" PKGREPOS=('core' 'extra' 'testing') PKGPOOL='pool/packages' + EXTRA_PKGPOOLS=('pool/community') SRCPOOL='sources/packages' TESTING_REPO='testing' STABLE_REPOS=('core' 'extra') @@ -123,9 +124,18 @@ eot mkdir -p "${TMP}/ftp/${r}/os/${a}" done done - mkdir -p "${TMP}/ftp/${PKGPOOL}" + mkdir -p "${TMP}/ftp/${PKGPOOL}"{,-reproducible} + mkdir -p "${TMP}/ftp/${EXTRA_PKGPOOLS[0]}" mkdir -p "${TMP}/ftp/${SRCPOOL}" + # make dummy packages for "reproducibility" + comm -12 <(pacman -Sql core extra | sort -u) <(pacman -Qq | sort -u) | pacman -Sddp - | while read -r line; do + touch "${FTP_BASE}/${PKGPOOL}/${line##*/}"{,.sig} + done + comm -12 <(pacman -Sql community | sort -u) <(pacman -Qq | sort -u) | pacman -Sddp - | while read -r line; do + touch "${FTP_BASE}/${EXTRA_PKGPOOLS[0]}/${line##*/}"{,.sig} + done + svnadmin create "${TMP}/svn-packages-repo" svn checkout -q "file://${TMP}/svn-packages-repo" "${TMP}/svn-packages-copy" } -- 2.19.2

3 18

[arch-projects] [devtools] [PATCH v2] Expand check_root keepenv variables
by foxboron＠archlinux.org 21 Dec '18

21 Dec '18

From: Morten Linderud <foxboron(a)archlinux.org> Les us source makepkg.conf settings from the environemnt. This also includes `GNUPGHOME` which is present in `makechrootpkg`, but not included in archbuild. Signed-off-by: Morten Linderud <foxboron(a)archlinux.org> --- archbuild.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/archbuild.in b/archbuild.in index 1e5b582..bd5706d 100644 --- a/archbuild.in +++ b/archbuild.in @@ -39,7 +39,7 @@ while getopts 'hcr:' arg; do esac done -check_root SOURCE_DATE_EPOCH +check_root SOURCE_DATE_EPOCH,SRCDEST,SRCPKGDEST,PKGDEST,LOGDEST,MAKEFLAGS,PACKAGER,GNUPGHOME # Pass all arguments after -- right to makepkg makechrootpkg_args+=("${@:$OPTIND}") -- 2.20.1

1 0

[arch-projects] [mkinitcpio] [PATCH] --rsyncable option
by Charles Duffy 16 Dec '18

16 Dec '18

At present, mkinitcpio outputs are not particularly stable: Generating two consecutive initramfs images from the same inputs can generate archives which are quite entirely unlike each other from the perspective of binary diff tools (rsync, casync, rdiff-backup, xdelta, etc). Making some adjustments to compression and archival can greatly reduce the size of this delta. A ticket with a proposed patch (adding a --rsyncable option to mkinitcpio) has been opened at https://bugs.archlinux.org/task/56128 -- however, as the advice there has been to consult this mailing list, it may no longer be open as of this writing. :) The change consists of the following: - Sort the list of files (in the C locale) before passing it to cpio. - When --rsyncable is passed, set all datestamps to the UNIX epoch before generating the cpio archive. - When --rsyncable is passed, pass the --reproducible option to cpio (this creates a dependency on GNU cpio 2.12, prevents device numbers from being included in the archive, and renumbers inode numbers to be consistent when file ordering is unchanged). - When --rsyncable is passed, pass the --rsyncable and --no-name options to gzip (the former tells it to reset the compression table on occasion, sacrificing compression ratio to prevent the effect from changes from cascading throughout too much of the stream; the latter prevents name and timestamp data from being stored in the gzip header itself). A version of the patch squashing together the two commits already present on the ticket is attached.

1 1

[arch-projects] [namcap] FS#34300 False positive on Mach-O universal binary
by Michael Straube 03 Dec '18

03 Dec '18

Hi, namcap does not distinguish java class files from Mach-O files since both use the same byte magic CAFEBABE. See FS#34300 Implementing this would only need a test if the value of the byte at offset 7 is greater than 0x30. See: https://github.com/file/file/blob/master/magic/Magdir/cafebabe I think having Mach-O files in a package is a _very_ rare case. But on the other hand implementing would be no big deal. Would a patch for this be accepted, or is it not worth? Regards, Michael

1 0

[arch-projects] [namcap] [PATCH] namcap-tags: Add back missing dependency reason tags (FS#56898)
by Michael Straube 03 Dec '18

03 Dec '18

There are two missing tags that are needed in rules added (back) in commit e385ac93a354 (Restore and refactor accidentally removed tests). Add back the missing tags. Signed-off-by: Michael Straube <michael.straube(a)posteo.de> --- namcap-tags | 2 ++ 1 file changed, 2 insertions(+) diff --git a/namcap-tags b/namcap-tags index 17b9506..70dabb2 100644 --- a/namcap-tags +++ b/namcap-tags @@ -101,6 +101,8 @@ depends-by-namcap-sight depends=(%s) :: Depends as namcap sees them: depends=(%s dconf-needed-for-glib-schemas :: needed for glib schemas glib2-needed-for-gio-modules :: needed for GIO modules hicolor-icon-theme-needed-for-hicolor-dir :: needed for hicolor theme hierarchy +shared-mime-info-needed :: needed for update-mime-database +desktop-file-utils-needed :: needed for update-desktop-database kdebase-runtime-needed %s :: needed for programs %s java-runtime-needed %s :: found class files %s libraries-needed %s %s :: libraries %s needed in files %s -- 2.19.2

2 2

[arch-projects] [namcap] [PATCH v3] makedepends: Make VCS matching more robust
by Michael Straube 02 Dec '18

02 Dec '18

If a VCS source is renamed using the "::" syntax the makedepends are not detected. If there are files starting with <name of VCS binary> in the source array false positives are produced. See the gitlab package for example. Make the matching more robust to avoid such issues. Signed-off-by: Michael Straube <michael.straube(a)posteo.de> --- v1 -> v2 Make it also work for e.g. git:// v2 -> v3 Return early if no vcs sources. Namcap/rules/makedepends.py | 15 ++++++++++++--- Namcap/tests/pkgbuild/test_makedepends.py | 15 +++++++++++---- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/Namcap/rules/makedepends.py b/Namcap/rules/makedepends.py index 2a0ceaa..53676c2 100644 --- a/Namcap/rules/makedepends.py +++ b/Namcap/rules/makedepends.py @@ -54,10 +54,19 @@ class VCSMakedepends(PkgbuildRule): 'svn' : 'subversion', } missing = [] + protocols = set() - for v in vcs: - if not any(s.startswith(v) for s in pkginfo["source"]): - continue + for s in pkginfo["source"]: + p = s.split("::", 1)[-1] + p = p.split("://", 1)[0] + p = p.split("+", 1)[0] + if p in vcs: + protocols.add(p) + + if not protocols: + return + + for v in protocols: d = vcs[v] if 'makedepends' not in pkginfo: missing.append(d) diff --git a/Namcap/tests/pkgbuild/test_makedepends.py b/Namcap/tests/pkgbuild/test_makedepends.py index 78c476a..d443b2b 100644 --- a/Namcap/tests/pkgbuild/test_makedepends.py +++ b/Namcap/tests/pkgbuild/test_makedepends.py @@ -76,10 +76,10 @@ depends=() makedepends=() license=('GPL') options=('!libtool') -source=(bzr+https://ftp.example.com/pub/mypackage - git+https://ftp.example.com/pub/mypackage - hg+https://ftp.example.com/pub/mypackage - svn+https://ftp.example.com/pub/mypackage) +source=(name::bzr+https://example.com/pub/mypackage + name::git://example.com/pub/mypackage + hg+https://example.com/pub/mypackage + svn://example.com/pub/mypackage) md5sums=('abcdefabcdef12345678901234567890') build() { @@ -104,4 +104,11 @@ package() { set(("missing-vcs-makedeps %s", i) for i in makedeps)) self.assertEqual(r.infos, []) + def test_example2(self): + # Example 2 + r = self.run_on_pkg(self.pkgbuild1 + 'source=(gitsomething)') + self.assertEqual(r.errors, []) + self.assertEqual(r.warnings, []) + self.assertEqual(r.infos, []) + # vim: set ts=4 sw=4 noet: -- 2.19.2

1 0

[arch-projects] [namcap] [PATCH v2] makedepends: Make VCS matching more robust
by Michael Straube 02 Dec '18

02 Dec '18

If a VCS source is renamed using the "::" syntax the makedepends are not detected. If there are files starting with <name of VCS binary> in the source array false positives are produced. See the gitlab package for example. Make the matching more robust to avoid such issues. Signed-off-by: Michael Straube <michael.straube(a)posteo.de> --- v1 -> v2 Make it also work for e.g. git:// Namcap/rules/makedepends.py | 10 +++++++++- Namcap/tests/pkgbuild/test_makedepends.py | 15 +++++++++++---- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/Namcap/rules/makedepends.py b/Namcap/rules/makedepends.py index 2a0ceaa..65747c4 100644 --- a/Namcap/rules/makedepends.py +++ b/Namcap/rules/makedepends.py @@ -54,9 +54,17 @@ class VCSMakedepends(PkgbuildRule): 'svn' : 'subversion', } missing = [] + protocols = set() + + for s in pkginfo["source"]: + p = s.split("::", 1)[-1] + p = p.split("://", 1)[0] + p = p.split("+", 1)[0] + if p in vcs: + protocols.add(p) for v in vcs: - if not any(s.startswith(v) for s in pkginfo["source"]): + if not v in protocols: continue d = vcs[v] if 'makedepends' not in pkginfo: diff --git a/Namcap/tests/pkgbuild/test_makedepends.py b/Namcap/tests/pkgbuild/test_makedepends.py index 78c476a..d443b2b 100644 --- a/Namcap/tests/pkgbuild/test_makedepends.py +++ b/Namcap/tests/pkgbuild/test_makedepends.py @@ -76,10 +76,10 @@ depends=() makedepends=() license=('GPL') options=('!libtool') -source=(bzr+https://ftp.example.com/pub/mypackage - git+https://ftp.example.com/pub/mypackage - hg+https://ftp.example.com/pub/mypackage - svn+https://ftp.example.com/pub/mypackage) +source=(name::bzr+https://example.com/pub/mypackage + name::git://example.com/pub/mypackage + hg+https://example.com/pub/mypackage + svn://example.com/pub/mypackage) md5sums=('abcdefabcdef12345678901234567890') build() { @@ -104,4 +104,11 @@ package() { set(("missing-vcs-makedeps %s", i) for i in makedeps)) self.assertEqual(r.infos, []) + def test_example2(self): + # Example 2 + r = self.run_on_pkg(self.pkgbuild1 + 'source=(gitsomething)') + self.assertEqual(r.errors, []) + self.assertEqual(r.warnings, []) + self.assertEqual(r.infos, []) + # vim: set ts=4 sw=4 noet: -- 2.19.2

2 2

[arch-projects] [namcap] [PATCH] makedepends: Make VCS matching more robust
by Michael Straube 02 Dec '18

02 Dec '18

If a VCS source is renamed using the "::" syntax the makedepends are not detected. If there are files starting with <name of VCS binary> in the source array false positives are produced. See the gitlab package for example. Make the matching more robust to avoid such issues. Signed-off-by: Michael Straube <michael.straube(a)posteo.de> --- Perhaps there is a more elegant way? Namcap/rules/makedepends.py | 2 +- Namcap/tests/pkgbuild/test_makedepends.py | 9 ++++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Namcap/rules/makedepends.py b/Namcap/rules/makedepends.py index 2a0ceaa..710e969 100644 --- a/Namcap/rules/makedepends.py +++ b/Namcap/rules/makedepends.py @@ -56,7 +56,7 @@ class VCSMakedepends(PkgbuildRule): missing = [] for v in vcs: - if not any(s.startswith(v) for s in pkginfo["source"]): + if not any(s.split("::")[-1].startswith(v + '+') for s in pkginfo["source"]): continue d = vcs[v] if 'makedepends' not in pkginfo: diff --git a/Namcap/tests/pkgbuild/test_makedepends.py b/Namcap/tests/pkgbuild/test_makedepends.py index 78c476a..0309016 100644 --- a/Namcap/tests/pkgbuild/test_makedepends.py +++ b/Namcap/tests/pkgbuild/test_makedepends.py @@ -77,7 +77,7 @@ makedepends=() license=('GPL') options=('!libtool') source=(bzr+https://ftp.example.com/pub/mypackage - git+https://ftp.example.com/pub/mypackage + some_name::git+https://ftp.example.com/pub/mypackage hg+https://ftp.example.com/pub/mypackage svn+https://ftp.example.com/pub/mypackage) md5sums=('abcdefabcdef12345678901234567890') @@ -104,4 +104,11 @@ package() { set(("missing-vcs-makedeps %s", i) for i in makedeps)) self.assertEqual(r.infos, []) + def test_example2(self): + # Example 2 + r = self.run_on_pkg(self.pkgbuild1 + 'source=(gitsomething)') + self.assertEqual(r.errors, []) + self.assertEqual(r.warnings, []) + self.assertEqual(r.infos, []) + # vim: set ts=4 sw=4 noet: -- 2.19.2

2 2