- Arch-projects - lists.archlinux.org

[arch-projects] [dbscripts] [GIT] Official repo DB scripts tag parabola/20171111 created. 20111105-451-ge397326
by eschwartz＠archlinux.org 11 Jan '19

11 Jan '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Official repo DB scripts". The tag, parabola/20171111 has been created at e3973262cca737d8c174b6292f64de6b1baeb6e6 (commit) - Log ----------------------------------------------------------------- commit e3973262cca737d8c174b6292f64de6b1baeb6e6 Author: Isaac David <isacdaavid(a)isacdaavid.info> Date: Sat Nov 11 11:44:16 2017 -0600 db-import-pkg: move all global setup code to a single place ----------------------------------------------------------------------- hooks/post-receive -- Official repo DB scripts

1 0

[arch-projects] [dbscripts] [GIT] Official repo DB scripts tag parabola/20180502 created. 20111105-561-g26c101c
by eschwartz＠archlinux.org 11 Jan '19

11 Jan '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Official repo DB scripts". The tag, parabola/20180502 has been created at 26c101cd9b966764d61a8496ad8801f83f9b8f7e (commit) - Log ----------------------------------------------------------------- commit 26c101cd9b966764d61a8496ad8801f83f9b8f7e Author: Luke Shumaker <lukeshu(a)parabola.nu> Date: Wed May 2 11:23:23 2018 -0400 db-import-pkg: Tidy calling `wc -l` ----------------------------------------------------------------------- hooks/post-receive -- Official repo DB scripts

1 0

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20180603-18-gd5970df
by eschwartz＠archlinux.org 11 Jan '19

11 Jan '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Official repo DB scripts". The branch, master has been updated via d5970df3a7051b7ceb5d69a345c763e9b5ed8052 (commit) via 8804e63f4414fdeef7f677c8b87a7acda052fd64 (commit) via 863818f2b2bcd00660299c5e609e5cf31d5213d0 (commit) from 948a181f5f09ca1cc03d450081f296222ca26d2f (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit d5970df3a7051b7ceb5d69a345c763e9b5ed8052 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Dec 2 12:07:05 2018 -0500 db-move: also block moving package from staging to extra without handling testing Don't allow anomalous testing packages floating around after a rebuild which are older than stable. commit 8804e63f4414fdeef7f677c8b87a7acda052fd64 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Dec 2 11:28:20 2018 -0500 db-update: die when trying to update a package without updating a pending rebuild A semi-common pattern is for one maintainer to stage a rebuild of a package due to e.g. cascading repository-wide python/boost/whatever rebuilds, and then for the original maintainer of the package to not notice and update the package in the stable repo, leaving an out of date rebuild in staging or testing. Then the the out of date package gets moved and ends up breaking things, possibly via a package downgrade, possibly via breaking compatibility with a much more targeted rebuild uploaded all at once. Ultimately, Things Happen™ and the repository hierarchy gets broken. Prevent this by enforcing for all packages that exist in multiple levels of the repo: staging -> testing -> stable That updates to one must come with an update to all the others. commit 863818f2b2bcd00660299c5e609e5cf31d5213d0 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Dec 2 12:28:36 2018 -0500 testing2x: be more generic and accept multiple testing repos Currently multilib is a second-class citizen the way it is lumped into community, and dbscripts cannot even keep track of whether it constitutes a testing repo. Teach config to track both testing and staging repos just like the stable ones, and teach testing2x to search through TESTING_REPOS to determine which one it is operating on. ----------------------------------------------------------------------- Summary of changes: config | 3 ++- config.local.svn-community | 3 ++- config.local.svn-packages | 3 ++- db-functions | 25 +++++++++++++++++++++++++ db-move | 9 +++++++++ db-update | 3 +++ test/cases/db-move.bats | 14 ++++++++++++++ test/cases/db-update.bats | 27 +++++++++++++++++++++++++++ test/lib/common.bash | 5 +++-- testing2x | 26 ++++++++++++++++---------- 10 files changed, 103 insertions(+), 15 deletions(-) hooks/post-receive -- Official repo DB scripts

1 0

[arch-projects] [dbscripts] [PATCH 1/3] testing2x: be more generic and accept multiple testing repos
by Eli Schwartz 10 Jan '19

10 Jan '19

Currently multilib is a second-class citizen the way it is lumped into community, and dbscripts cannot even keep track of whether it constitutes a testing repo. Teach config to track both testing and staging repos just like the stable ones, and teach testing2x to search through TESTING_REPOS to determine which one it is operating on. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- config | 3 ++- config.local.svn-community | 3 ++- config.local.svn-packages | 3 ++- test/lib/common.bash | 3 ++- testing2x | 26 ++++++++++++++++---------- 5 files changed, 24 insertions(+), 14 deletions(-) diff --git a/config b/config index 1cfe11f4..4c76558b 100644 --- a/config +++ b/config @@ -4,7 +4,8 @@ FTP_BASE="/srv/ftp" PKGREPOS=() PKGPOOL='' SRCPOOL='' -TESTING_REPO='' +STAGING_REPOS=() +TESTING_REPOS=() STABLE_REPOS=() # VCS backend diff --git a/config.local.svn-community b/config.local.svn-community index 5d61b5ea..493f34e3 100644 --- a/config.local.svn-community +++ b/config.local.svn-community @@ -5,7 +5,8 @@ PKGPOOL='pool/community' SRCPOOL='sources/community' SVNREPO='file:///srv/repos/svn-community/svn' SVNUSER='svn-community' -TESTING_REPO='community-testing' +STAGING_REPOS=('community-staging' 'multilib-staging') +TESTING_REPOS=('community-testing' 'multilib-testing') STABLE_REPOS=('community') CLEANUP_DESTDIR="/srv/repos/svn-community/package-cleanup" diff --git a/config.local.svn-packages b/config.local.svn-packages index 34aab35c..567d002e 100644 --- a/config.local.svn-packages +++ b/config.local.svn-packages @@ -5,7 +5,8 @@ PKGPOOL='pool/packages' SRCPOOL='sources/packages' SVNREPO='file:///srv/repos/svn-packages/svn' SVNUSER='svn-packages' -TESTING_REPO='testing' +STAGING_REPOS=('staging') +TESTING_REPOS=('testing') STABLE_REPOS=('core' 'extra') CLEANUP_DESTDIR="/srv/repos/svn-packages/package-cleanup" diff --git a/test/lib/common.bash b/test/lib/common.bash index bc2b4e6d..a92a01ee 100644 --- a/test/lib/common.bash +++ b/test/lib/common.bash @@ -103,7 +103,8 @@ setup() { PKGREPOS=('core' 'extra' 'testing') PKGPOOL='pool/packages' SRCPOOL='sources/packages' - TESTING_REPO='testing' + STAGING_REPOS=('staging') + TESTING_REPOS=('testing') STABLE_REPOS=('core' 'extra') CLEANUP_DESTDIR="${TMP}/package-cleanup" SOURCE_CLEANUP_DESTDIR="${TMP}/source-cleanup" diff --git a/testing2x b/testing2x index 14970a9f..215be39c 100755 --- a/testing2x +++ b/testing2x @@ -10,7 +10,7 @@ fi # Lock everything to reduce possibility of interfering task between the different repo-updates script_lock -for repo in "${TESTING_REPO}" "${STABLE_REPOS[@]}"; do +for repo in "${TESTING_REPOS[@]}" "${STABLE_REPOS[@]}"; do for pkgarch in "${ARCHES[@]}"; do repo_lock "${repo}" "${pkgarch}" || exit 1 done @@ -18,30 +18,36 @@ done declare -A pkgs +testing_repo= for pkgbase in "$@"; do for pkgarch in "${ARCHES[@]}"; do - vcsrepo_from="$(find_repo_for_package "${pkgbase}" "${pkgarch[@]}" "${TESTING_REPO}")" + vcsrepo_from="$(find_repo_for_package "${pkgbase}" "${pkgarch[@]}" "${TESTING_REPOS[@]}")" vcsrepo_to="$(find_repo_for_package "${pkgbase}" "${pkgarch[@]}" "${STABLE_REPOS[@]}")" + repo_from=${vcsrepo_from%-@(any|${pkgarch})} + repo_to="${vcsrepo_to%-@(any|${pkgarch})}" + if [[ -z ${testing_repo} ]]; then + testing_repo=${repo_from} + elif [[ ${testing_repo} != ${repo_from} ]]; then + die "Cannot move packages from multiple repos at a time: %s" "${testing_repo} ${repo_from}" + fi if [[ ${vcsrepo_from} && ${vcsrepo_to} ]]; then - move_to="${vcsrepo_to%-@(any|${pkgarch})}" - pkgs[${move_to}]+="${pkgbase} " + pkgs[${repo_to}]+="${pkgbase} " break fi done - [[ ${vcsrepo_from} ]] || die "%s not found in [%s]" "$pkgbase" "$TESTING_REPO" + [[ ${vcsrepo_from} ]] || die "%s not found in any of these repos: %s" "${pkgbase}" "${TESTING_REPOS[@]}" [[ ${vcsrepo_to} ]] || die "%s not found in any of these repos: %s" "$pkgbase" "${STABLE_REPOS[*]}" done -for pkgarch in "${ARCHES[@]}"; do - repo_unlock "${TESTING_REPO}" "${pkgarch}" -done -for repo in "${STABLE_REPOS[@]}"; do +for repo in "${TESTING_REPOS[@]}" "${STABLE_REPOS[@]}"; do for pkgarch in "${ARCHES[@]}"; do repo_unlock "${repo}" "${pkgarch}" done +done +for repo in "${STABLE_REPOS[@]}"; do if [[ -n ${pkgs[${repo}]} ]]; then - "$(dirname "$(readlink -e "$0")")/db-move" "${TESTING_REPO}" "${repo}" ${pkgs[${repo}]} + "$(dirname "$(readlink -e "$0")")/db-move" "${testing_repo}" "${repo}" ${pkgs[${repo}]} fi done -- 2.19.2

1 4

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20180603-15-g948a181
by eschwartz＠archlinux.org 10 Jan '19

10 Jan '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Official repo DB scripts". The branch, master has been updated via 948a181f5f09ca1cc03d450081f296222ca26d2f (commit) from f11a038c43270a70eafdba34ff33e134b6726a04 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 948a181f5f09ca1cc03d450081f296222ca26d2f Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Jan 10 10:59:25 2019 -0500 Fix db-archive not running at the correct time. At the point when it was being run, the signature was not yet moved to PKGPOOL. ----------------------------------------------------------------------- Summary of changes: db-update | 2 +- test/cases/db-update.bats | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) hooks/post-receive -- Official repo DB scripts

1 0

[arch-projects] [dbscripts] [PATCH] Fix db-archive not running at the correct time.
by Eli Schwartz 10 Jan '19

10 Jan '19

At the point when it was being run, the signature was not yet moved to PKGPOOL. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- Discovered while debugging the verify branch. db-update | 2 +- test/cases/db-update.bats | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/db-update b/db-update index 3a66c859..4848fef0 100755 --- a/db-update +++ b/db-update @@ -86,13 +86,13 @@ for repo in "${repos[@]}"; do # any packages might have been moved by the previous run if [[ -f ${pkg} ]]; then mv "${pkg}" "$FTP_BASE/${PKGPOOL}" - "$(dirname "$(readlink -e "${BASH_SOURCE[0]}")")/db-archive" "${FTP_BASE}/${PKGPOOL}/${pkg##*/}" fi ln -s "../../../${PKGPOOL}/${pkgfile}" "$FTP_BASE/$repo/os/${pkgarch}" # also move signatures if [[ -f ${pkg}.sig ]]; then mv "${pkg}.sig" "$FTP_BASE/${PKGPOOL}" fi + "$(dirname "$(readlink -e "${BASH_SOURCE[0]}")")/db-archive" "${FTP_BASE}/${PKGPOOL}/${pkg##*/}" if [[ -f $FTP_BASE/${PKGPOOL}/${pkgfile}.sig ]]; then ln -s "../../../${PKGPOOL}/${pkgfile}.sig" "$FTP_BASE/$repo/os/${pkgarch}" fi diff --git a/test/cases/db-update.bats b/test/cases/db-update.bats index bc978302..e1782a59 100644 --- a/test/cases/db-update.bats +++ b/test/cases/db-update.bats @@ -91,6 +91,7 @@ load ../lib/common releasePackage extra pkg-any-a db-update [[ -f ${ARCHIVE_BASE}/packages/p/pkg-any-a/pkg-any-a-1-1-any${PKGEXT} ]] + [[ -f ${ARCHIVE_BASE}/packages/p/pkg-any-a/pkg-any-a-1-1-any${PKGEXT}.sig ]] } @test "update same any package to same repository fails" { -- 2.20.1

1 0

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch verify-update-plus-rebuild created. 20180603-17-g77c7269
by eschwartz＠archlinux.org 10 Jan '19

10 Jan '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Official repo DB scripts". The branch, verify-update-plus-rebuild has been created at 77c72694e608decd63978f3ae6d941487476ef98 (commit) - Log ----------------------------------------------------------------- commit 77c72694e608decd63978f3ae6d941487476ef98 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Dec 2 12:07:05 2018 -0500 db-move: also block moving package from staging to extra without handling testing Don't allow anomalous testing packages floating around after a rebuild which are older than stable. commit aeeaac2e9979c8bf51f1e3401cc3c3f94fba7571 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Dec 2 11:28:20 2018 -0500 db-update: die when trying to update a package without updating a pending rebuild A semi-common pattern is for one maintainer to stage a rebuild of a package due to e.g. cascading repository-wide python/boost/whatever rebuilds, and then for the original maintainer of the package to not notice and update the package in the stable repo, leaving an out of date rebuild in staging or testing. Then the the out of date package gets moved and ends up breaking things, possibly via a package downgrade, possibly via breaking compatibility with a much more targeted rebuild uploaded all at once. Ultimately, Things Happen™ and the repository hierarchy gets broken. Prevent this by enforcing for all packages that exist in multiple levels of the repo: staging -> testing -> stable That updates to one must come with an update to all the others. commit 4a0f58b1eb729755bc2e5c2f68968c41bb859b76 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Dec 2 12:28:36 2018 -0500 testing2x: be more generic and accept multiple testing repos Currently multilib is a second-class citizen the way it is lumped into community, and dbscripts cannot even keep track of whether it constitutes a testing repo. Teach config to track both testing and staging repos just like the stable ones, and teach testing2x to search through TESTING_REPOS to determine which one it is operating on. ----------------------------------------------------------------------- hooks/post-receive -- Official repo DB scripts

1 0

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20180603-14-gf11a038
by eschwartz＠archlinux.org 10 Jan '19

10 Jan '19

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Official repo DB scripts". The branch, master has been updated via f11a038c43270a70eafdba34ff33e134b6726a04 (commit) via 490b2d11c87580eb3a954f655911074dca87029d (commit) from 549fe0017f44031b24fb5322726721b95cca395b (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit f11a038c43270a70eafdba34ff33e134b6726a04 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Nov 22 15:44:42 2018 -0500 Add reproducible archive of packages. Whenever adding new package files to the pool of distributed packages, copy the file into a longterm archive. This is the first step to merging the functionality of archivetools, as this implements the shared pool while also guaranteeing that all packages are archived at the time of entry rather than once per day if they still exist. commit 490b2d11c87580eb3a954f655911074dca87029d Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Nov 22 21:46:38 2018 -0500 tests: make dummy archive copies of all packages in the test environment Prerequisite for reproducible archives of packages. ----------------------------------------------------------------------- Summary of changes: config | 2 ++ db-archive | 21 +++++++++++++++++++++ db-functions | 39 +++++++++++++++++++++++++++++++-------- db-update | 5 +++++ test/cases/db-update.bats | 6 ++++++ test/lib/common.bash | 10 ++++++++++ 6 files changed, 75 insertions(+), 8 deletions(-) create mode 100755 db-archive hooks/post-receive -- Official repo DB scripts

1 0

[arch-projects] [dbscripts] [PATCH 1/4] tests: make dummy copies of all pkgpool packages in the test environment
by Eli Schwartz 09 Jan '19

09 Jan '19

Prerequisite for reproducible archives of packages. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- test/lib/common.bash | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/test/lib/common.bash b/test/lib/common.bash index bc2b4e6d..3dda5f62 100644 --- a/test/lib/common.bash +++ b/test/lib/common.bash @@ -102,6 +102,7 @@ setup() { SVNREPO="file://${TMP}/svn-packages-repo" PKGREPOS=('core' 'extra' 'testing') PKGPOOL='pool/packages' + EXTRA_PKGPOOLS=('pool/community') SRCPOOL='sources/packages' TESTING_REPO='testing' STABLE_REPOS=('core' 'extra') @@ -123,9 +124,18 @@ eot mkdir -p "${TMP}/ftp/${r}/os/${a}" done done - mkdir -p "${TMP}/ftp/${PKGPOOL}" + mkdir -p "${TMP}/ftp/${PKGPOOL}"{,-reproducible} + mkdir -p "${TMP}/ftp/${EXTRA_PKGPOOLS[0]}" mkdir -p "${TMP}/ftp/${SRCPOOL}" + # make dummy packages for "reproducibility" + comm -12 <(pacman -Sql core extra | sort -u) <(pacman -Qq | sort -u) | pacman -Sddp - | while read -r line; do + touch "${FTP_BASE}/${PKGPOOL}/${line##*/}"{,.sig} + done + comm -12 <(pacman -Sql community | sort -u) <(pacman -Qq | sort -u) | pacman -Sddp - | while read -r line; do + touch "${FTP_BASE}/${EXTRA_PKGPOOLS[0]}/${line##*/}"{,.sig} + done + svnadmin create "${TMP}/svn-packages-repo" svn checkout -q "file://${TMP}/svn-packages-repo" "${TMP}/svn-packages-copy" } -- 2.19.2

3 18

[arch-projects] [namcap] [PATCH] recognize arch-specific variables even if packages is for only one architecture
by arch＠eckner.net 09 Jan '19

09 Jan '19

From: Erich Eckner <arch(a)eckner.net> Architecture-specific variables should be useable as soon as a package is not built for 'any' architecture. With the deprecation of i686, this becomes relevant, as many packages are now only built for x86_64. However, the source (and checksums) may be _not_ architecture-agnostic, e.g. should _not_ be listed as source= but source_x86_64=. This patch silences the extra-var-begins-without-underscore message which is currently emitted in that case. Signed-off-by: Erich Eckner <arch(a)eckner.net> --- Namcap/rules/extravars.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Namcap/rules/extravars.py b/Namcap/rules/extravars.py index ba4df2d..81084f8 100644 --- a/Namcap/rules/extravars.py +++ b/Namcap/rules/extravars.py @@ -31,7 +31,7 @@ class package(PkgbuildRule): 'pkgbase', 'pkgver', 'pkgrel', 'epoch', 'pkgdesc', 'groups', 'url', 'install', 'changelog', 'options', 'validpgpkeys'] + carch_vars - if 'arch' in pkginfo and len(pkginfo["arch"]) >= 2: + if 'arch' in pkginfo and pkginfo['arch'] != ['any']: stdvars.extend(v+'_'+a for v,a in product(carch_vars, pkginfo["arch"])) for varname in pkginfo["setvars"]: if varname.islower() and varname not in stdvars \ -- 2.20.1

1 0