[arch-projects] [dbscripts] [PATCH 1/2] Make tests much more portable and usable
Using 'sudo' in a test? Not on my machine, I don't care how secure it should be, it is totally unnecessary, not to mention the annoyance of password prompts, needing the chroot setups, etc. Instead, use makepkg directly along with some very vanilla makepkg.conf config files to do the work. This makes for a more reproducable environment as far as makepkg settings goes.
Also adjust the package building code in common.inc to be a bit more straightforward. Remove the need for three seperate loops, and only ever copy packages in one direction (from the local test directory to the /dev/shm/ staging area).
With these changes, I didn't have too many problems running the test suite locally on my box after cloning the repository.
Signed-off-by: Dan McGee dan@archlinux.org --- test/lib/common.inc | 54 ++++++++++++++++++++------------------- test/lib/makepkg.i686.conf | 10 +++++++ test/lib/makepkg.x86_64.conf | 10 +++++++ test/test.d/db-update.sh | 6 +++- test/test.d/pool-transition.sh | 3 +- test/test.d/testing2x.sh | 3 +- 6 files changed, 56 insertions(+), 30 deletions(-) create mode 100644 test/lib/makepkg.i686.conf create mode 100644 test/lib/makepkg.x86_64.conf
diff --git a/test/lib/common.inc b/test/lib/common.inc index eb46508..a2ba1e0 100644 --- a/test/lib/common.inc +++ b/test/lib/common.inc @@ -10,39 +10,39 @@ oneTimeSetUp() { local pkgname local pkgarch local pkgversion - local build + origindir="$(dirname ${BASH_SOURCE[0]})/../packages" pkgdir="$(mktemp -d /dev/shm/$(basename $0).XXXXXXXXXX)" - cp -Lr $(dirname ${BASH_SOURCE[0]})/../packages/* "${pkgdir}" - msg 'Building packages...' - for d in "${pkgdir}"/*; do - pushd $d >/dev/null - pkgname=($(. PKGBUILD; echo ${pkgname[@]})) - pkgarch=($(. PKGBUILD; echo ${arch[@]})) - pkgversion=$(. PKGBUILD; echo $(get_full_version ${epoch:-0} ${pkgver} ${pkgrel})) - - build=true - for a in ${pkgarch[@]}; do - for p in ${pkgname[@]}; do - [ ! -f ${p}-${pkgversion}-${a}${PKGEXT} ] && build=false - done - done
- if ! ${build}; then + if [ -f ${origindir}/packages-built ]; then + msg 'Using existing packages.' + else + msg 'Building packages...' + for d in "${origindir}"/*; do + pushd $d >/dev/null + pkgname=($(. PKGBUILD; echo ${pkgname[@]})) + pkgarch=($(. PKGBUILD; echo ${arch[@]})) + pkgversion=$(. PKGBUILD; echo $(get_full_version ${epoch:-0} ${pkgver} ${pkgrel})) + + PKGDEST=$(pwd) + export PKGDEST + echo $PKGDEST if [ "${pkgarch[0]}" == 'any' ]; then - sudo extra-x86_64-build || die 'extra-x86_64-build failed' + MAKEPKG_CONF="$(dirname ${BASH_SOURCE[0]})/makepkg.i686.conf" + makepkg --force --config "$MAKEPKG_CONF" else for a in ${pkgarch[@]}; do - sudo extra-${a}-build || die "extra-${a}-build failed" + MAKEPKG_CONF="$(dirname ${BASH_SOURCE[0]})/makepkg.$a.conf" + makepkg --force --config "$MAKEPKG_CONF" done fi - for a in ${pkgarch[@]}; do - for p in ${pkgname[@]}; do - cp ${p}-${pkgversion}-${a}${PKGEXT} $(dirname ${BASH_SOURCE[0]})/../packages/$(basename ${d}) - done - done - fi - popd >/dev/null - done + popd >/dev/null + done + + touch ${origindir}/packages-built + fi + + cp -Lr "${origindir}"/* "${pkgdir}" + rm "${pkgdir}"/packages-built }
oneTimeTearDown() { @@ -229,3 +229,5 @@ checkRemovedAnyPackage() { [ -d "${TMP}/svn-packages-copy/${pkgbase}/repos/${repo}-any" ] \ && fail "svn-packages-copy/${pkgbase}/repos/${repo}-any should not exist" } + +# vim: syntax=sh: diff --git a/test/lib/makepkg.i686.conf b/test/lib/makepkg.i686.conf new file mode 100644 index 0000000..6fae31e --- /dev/null +++ b/test/lib/makepkg.i686.conf @@ -0,0 +1,10 @@ +# /etc/makepkg.conf + +CARCH="i686" + +BUILDENV=(fakeroot !distcc color !ccache check !sign) +OPTIONS=(strip docs libtool emptydirs zipman purge) + +INTEGRITY_CHECK=(md5) +PKGEXT='.pkg.tar.xz' +SRCEXT='.src.tar.gz' diff --git a/test/lib/makepkg.x86_64.conf b/test/lib/makepkg.x86_64.conf new file mode 100644 index 0000000..cdee1d7 --- /dev/null +++ b/test/lib/makepkg.x86_64.conf @@ -0,0 +1,10 @@ +# /etc/makepkg.conf + +CARCH="x86_64" + +BUILDENV=(fakeroot !distcc color !ccache check !sign) +OPTIONS=(strip docs libtool emptydirs zipman purge) + +INTEGRITY_CHECK=(md5) +PKGEXT='.pkg.tar.xz' +SRCEXT='.src.tar.gz' diff --git a/test/test.d/db-update.sh b/test/test.d/db-update.sh index e38c328..5db276b 100755 --- a/test/test.d/db-update.sh +++ b/test/test.d/db-update.sh @@ -82,7 +82,8 @@ testUpdateAnyPackage() { pushd "${TMP}/svn-packages-copy/pkg-any-a/trunk/" >/dev/null sed 's/pkgrel=1/pkgrel=2/g' -i PKGBUILD svn commit -q -m"update pkg to pkgrel=2" >/dev/null - sudo extra-i686-build >/dev/null 2>&1 + MAKEPKG_CONF="${curdir}/../lib/makepkg.i686.conf" + makepkg --force --config "$MAKEPKG_CONF" mv pkg-any-a-1-2-any.pkg.tar.xz "${pkgdir}/pkg-any-a/" popd >/dev/null
@@ -100,7 +101,8 @@ testUpdateAnyPackageToDifferentRepositoriesAtOnce() { pushd "${TMP}/svn-packages-copy/pkg-any-a/trunk/" >/dev/null sed 's/pkgrel=1/pkgrel=2/g' -i PKGBUILD svn commit -q -m"update pkg to pkgrel=2" >/dev/null - sudo extra-i686-build >/dev/null 2>&1 + MAKEPKG_CONF="${curdir}/../lib/makepkg.i686.conf" + makepkg --force --config "$MAKEPKG_CONF" mv pkg-any-a-1-2-any.pkg.tar.xz "${pkgdir}/pkg-any-a/" popd >/dev/null
diff --git a/test/test.d/pool-transition.sh b/test/test.d/pool-transition.sh index 5873f00..545c2a5 100755 --- a/test/test.d/pool-transition.sh +++ b/test/test.d/pool-transition.sh @@ -63,7 +63,8 @@ testUpdateAnyPackageWithoutPool() { pushd "${TMP}/svn-packages-copy/${pkgname}/trunk/" >/dev/null sed 's/pkgrel=1/pkgrel=2/g' -i PKGBUILD svn commit -q -m"update pkg to pkgrel=2" >/dev/null - sudo extra-i686-build >/dev/null 2>&1 + MAKEPKG_CONF="${curdir}/../lib/makepkg.i686.conf" + makepkg --force --config "$MAKEPKG_CONF" mv "${pkg2}" "${pkgdir}/${pkgname}/" popd >/dev/null
diff --git a/test/test.d/testing2x.sh b/test/test.d/testing2x.sh index eda6cd6..bcc1b9d 100755 --- a/test/test.d/testing2x.sh +++ b/test/test.d/testing2x.sh @@ -10,7 +10,8 @@ testTesting2xAnyPackage() { pushd "${TMP}/svn-packages-copy/pkg-any-a/trunk/" >/dev/null sed 's/pkgrel=1/pkgrel=2/g' -i PKGBUILD svn commit -q -m"update pkg to pkgrel=2" >/dev/null - sudo extra-i686-build >/dev/null 2>&1 + MAKEPKG_CONF="${curdir}/../lib/makepkg.i686.conf" + makepkg --force --config "$MAKEPKG_CONF" mv pkg-any-a-1-2-any.pkg.tar.xz "${pkgdir}/pkg-any-a/" popd >/dev/null
Precondition: trunk/ directory never has any subdirectories. This is true across our entire svn-packages repository, although I did not check svn-community. The following command was used:
find -mindepth 3 -type d ! -wholename '*/.svn*' -wholename *trunk*
This should really help a variety of things- reduce the number of commits, reduce the number of emails to arch-commits, make mass package moves faster, etc.
The entire dbscripts test suite still passes after this change.
Signed-off-by: Dan McGee dan@archlinux.org --- db-move | 13 ++++++++++--- 1 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/db-move b/db-move index ff1b0e2..e3467e8 100755 --- a/db-move +++ b/db-move @@ -78,11 +78,18 @@ for pkgbase in ${args[@]:2}; do pkgver=$(. "${svnrepo_from}/PKGBUILD"; echo $(get_full_version ${epoch:-0} ${pkgver} ${pkgrel}))
if [ -d "${svnrepo_to}" ]; then - /usr/bin/svn rm --force -q "${svnrepo_to}" - /usr/bin/svn commit -q "${WORKDIR}/svn/${pkgbase}" -m "$(basename $0): ${pkgbase} removed by $(id -un) for move to [${repo_to}] (${pkgarch})" + for file in $(svn ls "${svnrepo_to}"); do + svn rm -q "$file" + done + else + mkdir "${svnrepo_to}" + svn add "${svnrepo_to}" fi
- /usr/bin/svn mv -q -r HEAD "${svnrepo_from}" "${svnrepo_to}" + for file in $(svn ls "${svnrepo_from}"); do + svn mv -q -r HEAD "${svnrepo_from}/$file" "${svnrepo_to}/" + done + /usr/bin/svn rm --force -q "${svnrepo_from}" /usr/bin/svn commit -q "${WORKDIR}/svn/${pkgbase}" -m "$(basename $0): moved ${pkgbase} from [${repo_from}] to [${repo_to}] (${pkgarch})"
for pkgname in ${pkgnames[@]}; do
On Tue, 28 Jun 2011 21:58:14 -0500, Dan McGee wrote:
Using 'sudo' in a test? Not on my machine, I don't care how secure it should be, it is totally unnecessary, not to mention the annoyance of password prompts, needing the chroot setups, etc. Instead, use makepkg directly along with some very vanilla makepkg.conf config files to do the work. This makes for a more reproducable environment as far as makepkg settings goes.
I prefer using devtools here instead of duplicating tools. This is only a test suite and packages need to be built only once.
This way we also make sure we get the same packages as a packager would get.
Also your approach would need gcc-multilib to be able to build 32bit packages.
On Mon, Jul 25, 2011 at 1:48 PM, Pierre Schmitz pierre@archlinux.de wrote:
On Tue, 28 Jun 2011 21:58:14 -0500, Dan McGee wrote:
Using 'sudo' in a test? Not on my machine, I don't care how secure it should be, it is totally unnecessary, not to mention the annoyance of password prompts, needing the chroot setups, etc. Instead, use makepkg directly along with some very vanilla makepkg.conf config files to do the work. This makes for a more reproducable environment as far as makepkg settings goes.
I prefer using devtools here instead of duplicating tools. This is only a test suite and packages need to be built only once.
What tools are we duplicating?
If the tests aren't self contained, they don't get run. It took me an hour or more to get these running, and that is straight up unacceptable.
I will *not* ever give a script sudo access on my machine. Ever. Nor will most other people writing improvements to these tests and scripts.
This way we also make sure we get the same packages as a packager would get.
The same as makepkg? I don't understand at all what you mean by this and how using makepkg directly is not the same as a packager would get (if it isn't, then devtools is busted, and we aren't attempting to test devtools here).
Also your approach would need gcc-multilib to be able to build 32bit packages.
Well for starters your test suite won't even run on x86 because of this, and the fact that you try to build 64 bit packages. But more importantly, where does gcc-multilib even come into play here? You aren't even building a real package, and there is no source code!
-- Pierre Schmitz, https://users.archlinux.de/~pierre
participants (3)
-
Dan McGee
-
Dan McGee
-
Pierre Schmitz