September 2013 - Pacman-dev - lists.archlinux.org

[pacman-dev] Please make the colourised output exactly like that of "pacman-color"
by Xavion 29 Apr '16

29 Apr '16

Hi All, It's great that "pacman" finally has native colour support in v4.1.0. It's just a shame that it's not as good as that of "pacman-color". Without meaning any offence, it falls way short of the mark IMHO. I'm hoping you can just replicate the colourisation of "pacman-color". For your convenience, I've attached the relevant files you'd need. I'm only requesting this because "pacman-color" no longer exists. -- Regards, Xavion.

10 60

[pacman-dev] [PATCH v2] Extend database upgrade script to handle alpm db version 9
by Allan McRae 20 Aug '14

20 Aug '14

Signed-off-by: Allan McRae <allan(a)archlinux.org> --- v2: Use awk to do symlink to actual directory replacement As an example of what changes get made: diff -Naur local.tmp/glibc-2.18-4/files local/glibc-2.18-4/files --- local.tmp/glibc-2.18-4/files 2013-09-18 18:45:35.747068765 +1000 +++ local/glibc-2.18-4/files 2013-09-18 18:51:29.633643945 +1000 @@ -4,8 +4,8 @@ etc/locale.gen etc/nscd.conf etc/rpc -lib/ -lib/libc.a +usr/lib/ +usr/lib/libc.a usr/ usr/bin/ usr/bin/catchsegv Note that now usr/lib/ is present twice in the file (may not always be the case) and these entries are out of alphabetical order. Both these are cosmetic to pacman. scripts/pacman-db-upgrade.sh.in | 59 ++++++++++++++++++++++++++++++++++------- 1 file changed, 49 insertions(+), 10 deletions(-) diff --git a/scripts/pacman-db-upgrade.sh.in b/scripts/pacman-db-upgrade.sh.in index a1630c5..d6f4169 100644 --- a/scripts/pacman-db-upgrade.sh.in +++ b/scripts/pacman-db-upgrade.sh.in @@ -109,18 +109,57 @@ fi # do not let pacman run while we do this touch "$lockfile" -# pacman-3.4 to 3.5 upgrade - merge depends into desc -if [[ $(find "$dbroot"/local -name depends) ]]; then - msg "$(gettext "Pre-3.5 database format detected - upgrading...")" - for i in "$dbroot"/local/*; do - if [[ -f "$i"/depends ]]; then - cat "$i"/depends >> "$i"/desc - rm "$i"/depends - fi - done - msg "$(gettext "Done.")" +if [[ -f "${dbroot}"/local/.alpm_db_version ]]; then + db_version=$(cat "${dbroot}"/local/.alpm_db_version) fi +if [[ -z "$db_version" ]]; then + # pacman-3.4 to 3.5 upgrade - merge depends into desc + if [[ $(find "$dbroot"/local -name depends) ]]; then + msg "$(gettext "Pre-3.5 database format detected - upgrading...")" + for i in "$dbroot"/local/*; do + if [[ -f "$i"/depends ]]; then + cat "$i"/depends >> "$i"/desc + rm "$i"/depends + fi + done + msg "$(gettext "Done.")" + fi + + # pacman 4.1 to 4.2 upgrade - remove directory symlink support + dirlist=() + + unset GREP_OPTIONS + while IFS= read -r dir; do + dirlist+=("/${dir%/}") + done < <(grep -h '/$' "$dbroot"/local/*/files | sort -u) + + mapfile -t dirlist < <(find "${dirlist[@]}" -maxdepth 0 -type l) + + if [[ ${#dirlist[@]} != 0 ]]; then + msg "$(gettext "Pre-4.2 database format detected - upgrading...")" + for dir in "${dirlist[@]}"; do + realdir="$(cd $dir; pwd -P)" + for f in "$dbroot"/local/*/files; do + awk -v "dir=${dir#/}/" -v "realdir=${realdir#/}/" ' + BEGIN { + i = length(dir) + 1 + } + { + if (index($0, dir) == 1) { + printf("%s%s\n", realdir, substr($0, i)) + } else { + print + } + }' $f > $f.tmp + mv $f.tmp $f + done + done + fi +fi + +echo "9" > "$dbroot"/local/.alpm_db_version + # remove the lock file rm -f "$lockfile" -- 1.8.4

2 9

[pacman-dev] Time for a 4.1.1 release?
by Allan McRae 31 Oct '13

31 Oct '13

It looks like the bug reports for 4.1 have stopped, so I guess we should make a maintenance release. There are a couple of new strings to be translated and a couple of issues reported with translations, so I will need to get those on transifex. See: here for a list of bugs reported/fixed: https://bugs.archlinux.org/index.php?do=index&tasks=&project=3&due=111&stat… The PKGBUILD proto files could have the prepare() function added just to be more complete and fix the remaining bug is someone wants to volunteer to do that... Anything I have missed? A bug not assigned to 4.1.1 that should be fixed? Allan

4 7

[pacman-dev] [PATCH] deps.c: check for indirect deps when ordering
by Andrew Gregory 18 Oct '13

18 Oct '13

On upgrades, indirect dependencies were not being detected if there was a dependency in between them that was not part of the transaction. For example, with the dependency chain: pkg1 -> pkg2 -> pkg3, if pkg1 and pkg3 are being upgraded but not pkg2 pacman would not order pkg1 and pkg3 properly. This was particularly problematic when replacements were involved because the replaced package(s) would be removed at the start of the transaction. If an install script required the replacer and lacked a direct dependency, it could fail. Fixes FS#32764. Partially fixes FS#23011. Signed-off-by: Andrew Gregory <andrew.gregory.8(a)gmail.com> --- The need to keep state in order to avoid getting stuck in cyclic dependencies makes this a little ugly (hence the wrapper function). If anybody has a better idea for avoiding those loops, I'd love to clean it up. lib/libalpm/deps.c | 101 +++++++++++++++++++++++++++++++++------- test/pacman/tests/replace100.py | 2 - test/pacman/tests/upgrade100.py | 44 +++++++++++++++++ 3 files changed, 127 insertions(+), 20 deletions(-) create mode 100644 test/pacman/tests/upgrade100.py diff --git a/lib/libalpm/deps.c b/lib/libalpm/deps.c index 96c49a9..adcdd3a 100644 --- a/lib/libalpm/deps.c +++ b/lib/libalpm/deps.c @@ -65,8 +65,8 @@ void _alpm_depmiss_free(alpm_depmissing_t *miss) FREE(miss); } -/* Does pkg1 depend on pkg2, ie. does pkg2 satisfy a dependency of pkg1? */ -static int _alpm_dep_edge(alpm_pkg_t *pkg1, alpm_pkg_t *pkg2) +/** Check if pkg2 satisfies a dependency of pkg1 */ +static int _alpm_pkg_depends_on(alpm_pkg_t *pkg1, alpm_pkg_t *pkg2) { alpm_list_t *i; for(i = alpm_pkg_get_depends(pkg1); i; i = i->next) { @@ -77,6 +77,84 @@ static int _alpm_dep_edge(alpm_pkg_t *pkg1, alpm_pkg_t *pkg2) return 0; } +static alpm_pkg_t *find_dep_satisfier(alpm_list_t *pkgs, alpm_depend_t *dep) +{ + alpm_list_t *i; + + for(i = pkgs; i; i = i->next) { + alpm_pkg_t *pkg = i->data; + if(_alpm_depcmp(pkg, dep)) { + return pkg; + } + } + return NULL; +} + +/** Check if pkg2 is anywhere in pkg1's dependency tree. + * @param pkg1 + * @param pkg2 + * @param targets if a package in this list is an intermediate dependency + * between pkg1 and pkg2, the pkg1 -> pkg2 dependency will not be + * reported + * @param ignore packages which should not be recursively checked for + * intermediate dependencies. Used internally for state to avoid + * getting stuck on cyclic dependencies. + * @return 1 if pkg2 is found in pkg1's dependency tree + */ +static int _alpm_pkg_in_dep_tree(alpm_pkg_t *pkg1, alpm_pkg_t *pkg2, + alpm_list_t *targets, alpm_list_t **ignore) +{ + alpm_list_t *i, *pkgs = alpm_db_get_pkgcache(pkg1->handle->db_local); + + if(_alpm_pkg_depends_on(pkg1, pkg2)) { + return 1; + } + + *ignore = alpm_list_add(*ignore, pkg1); + + /* pkg1 does not directly depend on pkg2, but if this is an upgrade + * operation there may be an indirect dependency through an installed + * dependency not part of the current transaction */ + for(i = alpm_pkg_get_depends(pkg1); i; i = i->next) { + alpm_depend_t *dep = i->data; + alpm_pkg_t *lpkg = find_dep_satisfier(pkgs, dep); + + if(!lpkg) { + continue; + } else if(alpm_list_find(targets, lpkg, _alpm_pkg_cmp)) { + /* lpkg's upgrade is part of the transaction, any dependency will be + * detected separately as pkg1 -> lpkg and lpkg -> pkg2 */ + continue; + } else if(alpm_list_find(*ignore, lpkg, _alpm_pkg_cmp)) { + /* we've already checked lpkg, move on */ + continue; + } else if(_alpm_pkg_in_dep_tree(lpkg, pkg2, targets, ignore)) { + /* we have an indirect dependency: pkg1 -> lpkg -> ... -> pkg2 */ + return 1; + } + } + + return 0; +} + +/** Check if pkg2 is anywhere in pkg1's dependency tree. + * Wrapper for _alpm_pkg_in_dep_tree to handle creating and destroying state. + * @param pkg1 + * @param pkg2 + * @param targets if a package in this list is an intermediate dependency + * between pkg1 and pkg2, the pkg1 -> pkg2 dependency will not be + * reported + * @return 1 if pkg2 is found in pkg1's dependency tree + */ +static int _alpm_dep_edge(alpm_pkg_t *pkg1, alpm_pkg_t *pkg2, + alpm_list_t *targets) +{ + alpm_list_t *ignore = NULL; + int ret = _alpm_pkg_in_dep_tree(pkg1, pkg2, targets, &ignore); + alpm_list_free(ignore); + return ret; +} + /* Convert a list of alpm_pkg_t * to a graph structure, * with a edge for each dependency. * Returns a list of vertices (one vertex = one package) @@ -101,7 +179,7 @@ static alpm_list_t *dep_graph_init(alpm_list_t *targets) for(j = vertices; j; j = j->next) { alpm_graph_t *vertex_j = j->data; alpm_pkg_t *p_j = vertex_j->data; - if(_alpm_dep_edge(p_i, p_j)) { + if(_alpm_dep_edge(p_i, p_j, targets)) { vertex_i->children = alpm_list_add(vertex_i->children, vertex_j); } @@ -229,19 +307,6 @@ static void release_filtered_depend(alpm_depend_t *dep, int nodepversion) } } -static alpm_pkg_t *find_dep_satisfier(alpm_list_t *pkgs, alpm_depend_t *dep) -{ - alpm_list_t *i; - - for(i = pkgs; i; i = i->next) { - alpm_pkg_t *pkg = i->data; - if(_alpm_depcmp(pkg, dep)) { - return pkg; - } - } - return NULL; -} - /** Find a package satisfying a specified dependency. * The dependency can include versions with depmod operators. * @param pkgs an alpm_list_t* of alpm_pkg_t where the satisfier will be searched @@ -517,7 +582,7 @@ static int can_remove_package(alpm_db_t *db, alpm_pkg_t *pkg, /* see if other packages need it */ for(i = _alpm_db_get_pkgcache(db); i; i = i->next) { alpm_pkg_t *lpkg = i->data; - if(_alpm_dep_edge(lpkg, pkg) && !alpm_pkg_find(targets, lpkg->name)) { + if(_alpm_pkg_depends_on(lpkg, pkg) && !alpm_pkg_find(targets, lpkg->name)) { return 0; } } @@ -549,7 +614,7 @@ int _alpm_recursedeps(alpm_db_t *db, alpm_list_t *targs, int include_explicit) alpm_pkg_t *pkg = i->data; for(j = _alpm_db_get_pkgcache(db); j; j = j->next) { alpm_pkg_t *deppkg = j->data; - if(_alpm_dep_edge(pkg, deppkg) + if(_alpm_pkg_depends_on(pkg, deppkg) && can_remove_package(db, deppkg, targs, include_explicit)) { alpm_pkg_t *copy; _alpm_log(db->handle, ALPM_LOG_DEBUG, "adding '%s' to the targets\n", diff --git a/test/pacman/tests/replace100.py b/test/pacman/tests/replace100.py index f94ca0f..dcb7111 100644 --- a/test/pacman/tests/replace100.py +++ b/test/pacman/tests/replace100.py @@ -41,5 +41,3 @@ self.addrule("PKG_VERSION=kernel26|2.6.37.1-1") self.addrule("FILE_EXIST=sbin/blkid") self.addrule("FILE_EXIST=foundit") - -self.expectfailure = True diff --git a/test/pacman/tests/upgrade100.py b/test/pacman/tests/upgrade100.py new file mode 100644 index 0000000..3f7a021 --- /dev/null +++ b/test/pacman/tests/upgrade100.py @@ -0,0 +1,44 @@ +self.description = "Indirect dependency ordering (FS#32764)" + +lp1 = pmpkg("fcitx-gtk2", "1.0-1"); +lp1.depends = ["gtk2"] +self.addpkg2db("local", lp1) + +lp2 = pmpkg("gtk2", "1.0-1"); +lp2.depends = ["pango"] +self.addpkg2db("local", lp2) + +lp3 = pmpkg("pango", "1.0-1"); +lp3.depends = ["harfbuzz"] +self.addpkg2db("local", lp3) + +lp4 = pmpkg("harfbuzz", "1.0-1"); +lp4.depends = ["icu"] +self.addpkg2db("local", lp4) + +lp5 = pmpkg("icu", "1.0-1"); +self.addpkg2db("local", lp5) + +sp1 = pmpkg("fcitx-gtk2", "1.0-2"); +sp1.depends = ["gtk2"] +sp1.install['post_upgrade'] = "[ -f bin/harfbuzz ] && echo > found_harfbuzz" +self.addpkg2db("sync", sp1) + +sp4 = pmpkg("harfbuzz", "1.0-2"); +sp4.depends = ["icu"] +sp4.files = ["bin/harfbuzz"] +sp4.install['post_upgrade'] = "[ -f bin/icu ] && echo > found_icu" +self.addpkg2db("sync", sp4) + +sp5 = pmpkg("icu", "1.0-2"); +sp5.files = ["bin/icu"] +self.addpkg2db("sync", sp5) + +self.args = "-Su" + +self.addrule("PACMAN_RETCODE=0") +self.addrule("PKG_VERSION=fcitx-gtk2|1.0-2") +self.addrule("PKG_VERSION=harfbuzz|1.0-2") +self.addrule("PKG_VERSION=icu|1.0-2") +self.addrule("FILE_EXIST=found_harfbuzz") +self.addrule("FILE_EXIST=found_icu") -- 1.8.2.3

3 5

[pacman-dev] A makepkg test suite
by Ashley Whetter 09 Oct '13

09 Oct '13

I've recently come across issue FS#15645 ([makepkg] needs a test suite) and I'd like to have a bash at implementing it. It seems like a good idea to get peoples ideas and thoughts first so what do you think? My first reaction is to do something similar to the pacman tests and just write a custom framework in python. Also have the output the same as the pacman tests. I think keeping the idea an environment (like in the pacman tests) seems a bit heavy handed because the only configuration done is in makepkg.conf and we can probably fake a pacman environment using a stub pacman. Or maybe we could utilise the pacman test environments! Unit testing should be easy enough because we can just call specific functions in makepkg and check the output. For functional testing I'm not really sure. As for acceptance testing we can just put different flag combinations in different test environments, and check the return code. Check the .PKGINFO. Maybe that files were put where they should be in pkg and src, as well as in the package itself. Have some PKGBUILDs that fail for various reasons like syntax, a dependency package doesn't exist. Make sure missing {,make,check}dependencies are installed. Any suggestions or ideas? Ashley

8 39

[pacman-dev] pacman-disowned
by Jeremy Heiner 06 Oct '13

06 Oct '13

Greetings, everyone! My name is Jeremy. Pleased to make your acquaintance. The "Pacman Tips" wiki page provides a pacman-disowned shell script and suggests running it periodically. I completely agree: I have found it both helpful and educational. I've played around with the script, trying to squeeze more information into its output. Just so you can get a sense of what direction my exploration lead me, I've attached my scala implementation. To be clear: I am *not* suggesting this scala program as a replacement for pacman-disowned. I can't even guarantee that my scala program does anything useful, and I eschew any liability should you ever run it :). I've only included it in case you're wondering where my head is at. What do I mean by "squeeze more info in"? Well, it would be fairly easy to tweak the shell script to list missing files (even though you can get that info using "-Qk"). So my scala program outputs a line for each file that indicates if it is missing or if it is not owned by a package: it combines the info from both "-Qk" and pacman-disowned. Anyway, I've been thinking about what it would take to bake this functionality into pacman itself. Now I'm certain many of you are already objecting: "it's better to layer this on top than to bake it in". And I concede that is a powerful objection. Still, I can't see any harm in exploring the idea. I'm benefiting by learning about the insides of pacman, even if my code is not adopted. I must also admit that I have not got to the point of implementing my scala stuff in C yet. I do have five-ish patches ready to post here, but they are all about preparing for eventually adding that functionality. It is possible that I might never post a patch that attains my end goal, but still some of these preliminary patches might prove useful. The first clump of patches I will post add unit tests for "-Qk". My ideas will (eventually) require big changes to check.c, and I wouldn't want to break anything there. This involved adding a new "hook" mechanism to the test framework, so that a package could be installed, then mangled by removing items, all in the generation phase so that the "-Qk" output could be verified in the test phase. Thanks in advance for your time evaluating my offerings, and for any feedback you have, Jeremy

9 28

[pacman-dev] [PATCH] Make 'SRCDEST', 'PKGDEST' and 'PKGSRCDEST' respect the 'BUILDDIR'
by Techlive Zheng 03 Oct '13

03 Oct '13

--- A task has also been added to the bugtracker: https://bugs.archlinux.org/task/35498 scripts/makepkg.sh.in | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index b85b87e..c27c74d 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -2790,7 +2790,15 @@ if [[ $BUILDDIR = "$startdir" ]]; then else srcdir="$BUILDDIR/$pkgbase/src" pkgdirbase="$BUILDDIR/$pkgbase/pkg" - + if [[ $SRCDEST = "$startdir" ]]; then + SRCDEST="$BUILDDIR/$pkgbase" + fi + if [[ $PKGDEST = "$startdir" ]]; then + PKGDEST="$BUILDDIR/$pkgbase" + fi + if [[ $SRCPKGDEST = "$startdir" ]]; then + SRCPKGDEST="$BUILDDIR/$pkgbase" + fi fi # set pkgdir to something "sensible" for (not recommended) use during build() -- 1.8.2.3

3 8

[pacman-dev] [PATCH] TESTS: add missing tests
by Andrew Gregory 03 Oct '13

03 Oct '13

Signed-off-by: Andrew Gregory <andrew.gregory.8(a)gmail.com> --- If anybody's interested, I have a script that can be used in a post-{commit,merge,applypatch} hook to check for unlisted tests here: https://github.com/andrewgregory/bin/blob/master/checktestlist test/pacman/tests/TESTS | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/test/pacman/tests/TESTS b/test/pacman/tests/TESTS index 2b47244..6652041 100644 --- a/test/pacman/tests/TESTS +++ b/test/pacman/tests/TESTS @@ -48,6 +48,8 @@ test/pacman/tests/fileconflict023.py \ test/pacman/tests/fileconflict024.py \ test/pacman/tests/fileconflict025.py \ test/pacman/tests/fileconflict030.py \ +test/pacman/tests/fileconflict031.py \ +test/pacman/tests/fileconflict032.py \ test/pacman/tests/ignore001.py \ test/pacman/tests/ignore002.py \ test/pacman/tests/ignore003.py \ @@ -158,6 +160,8 @@ test/pacman/tests/sync043.py \ test/pacman/tests/sync044.py \ test/pacman/tests/sync045.py \ test/pacman/tests/sync050.py \ +test/pacman/tests/sync051.py \ +test/pacman/tests/sync052.py \ test/pacman/tests/sync100.py \ test/pacman/tests/sync1000.py \ test/pacman/tests/sync1003.py \ -- 1.8.4

2 3

[pacman-dev] Fwd: [PATCH] RFC: support incremental build with VCS sources.
by Lukas Jirkovsky 01 Oct '13

01 Oct '13

Accidentally sent this to a wrong address... I hope forwarding this won't break the patch. --- Hi, This patch adds incremental build support for the packages with VCS sources. It is not complete work (only GIT is supported), but a mere request for comments. If there is a chance that something like this will be accepted, I will extend the support to other VCS, too. Rationale: When someone uses a VCS package, there is a high chance that such user wants to follow the development more closely. This means that the user will be rebuilding the package often. However, rebuilding the whole package takes a lot of time. This can be somewhat mitigated by using ccache. However, the greatest speedup can be achieved by incremental build. The problem is that current VCS support makes this pretty much imposible by removing the cloned sources before building. This patch adds a new option --incremental. Using this option forces makepkg to only update the cloned source directory instead of removing it and cloning anew. This allows make to use the incremental build feature. See also FS#35050 I've been using this patch for a few days with kdevplatform-git and kdevelop-git (not the ones from AUR) at it works great. It already saved me a lot of time, while using nice and clean PKGBUILD (so far I have been using the old-style VCS packages that clone sources in their build() function to achive similar functionality). Any comments are welcome. Lukas scripts/makepkg.sh.in | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 67ec240..2ac813a 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -66,6 +66,7 @@ FORCE=0 INFAKEROOT=0 GENINTEG=0 HOLDVER=0 +VCSINCREMENTAL=0 SKIPCHECKSUMS=0 SKIPPGPCHECK=0 INSTALL=0 @@ -580,12 +581,20 @@ extract_git() { msg2 "$(gettext "Creating working copy of %s %s repo...")" "${repo}" "git" pushd "$srcdir" &>/dev/null - rm -rf "${dir##*/}" - if ! git clone "$dir"; then - error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" - plain "$(gettext "Aborting...")" - exit 1 + if [[ VCSINCREMENTAL -ne 0 && -d "${dir##*/}" ]]; then + pushd `pwd` &>/dev/null + cd_safe "${dir##*/}" + git pull + popd &>/dev/null + else + rm -rf "${dir##*/}" + + if ! git clone "$dir"; then + error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" + plain "$(gettext "Aborting...")" + exit 1 + fi fi cd_safe "${dir##*/}" @@ -607,7 +616,7 @@ extract_git() { fi if [[ -n $ref ]]; then - if ! git checkout -b makepkg $ref; then + if ! git checkout -B makepkg $ref; then error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" plain "$(gettext "Aborting...")" exit 1 @@ -2510,6 +2519,7 @@ usage() { printf -- "$(gettext " --check Run the %s function in the %s")\n" "check()" "$BUILDSCRIPT" printf -- "$(gettext " --config <file> Use an alternate config file (instead of '%s')")\n" "$confdir/makepkg.conf" printf -- "$(gettext " --holdver Do not update VCS sources")\n" + printf -- "$(gettext " --incremental Incremental build with VCS sources (only updates build dir)")\n" printf -- "$(gettext " --key <key> Specify a key to use for %s signing instead of the default")\n" "gpg" printf -- "$(gettext " --nocheck Do not run the %s function in the %s")\n" "check()" "$BUILDSCRIPT" printf -- "$(gettext " --noprepare Do not run the %s function in the %s")\n" "prepare()" "$BUILDSCRIPT" @@ -2554,7 +2564,7 @@ ARGLIST=("$@") # Parse Command Line Options. OPT_SHORT="AcdefFghiLmop:rRsSV" OPT_LONG=('allsource' 'asroot' 'check' 'clean' 'config:' 'force' 'geninteg' - 'help' 'holdver' 'ignorearch' 'install' 'key:' 'log' 'nobuild' 'nocolor' + 'help' 'holdver' 'incremental' 'ignorearch' 'install' 'key:' 'log' 'nobuild' 'nocolor' 'nocheck' 'nodeps' 'noextract' 'noprepare' 'nosign' 'pkg:' 'repackage' 'rmdeps' 'sign' 'skipchecksums' 'skipinteg' 'skippgpcheck' 'source' 'syncdeps' 'verifysource' 'version') @@ -2589,6 +2599,7 @@ while true; do -F) INFAKEROOT=1 ;; -g|--geninteg) GENINTEG=1 ;; --holdver) HOLDVER=1 ;; + --incremental) VCSINCREMENTAL=1 ;; -i|--install) INSTALL=1 ;; --key) shift; GPGKEY=$1 ;; -L|--log) LOGGING=1 ;; -- 1.8.4

3 6

[pacman-dev] [PATCH 1] Simplify pmtest.generate and .run API (removed pacman parameter).
by Jeremy Heiner 01 Oct '13

01 Oct '13

Previously pmtest.__init__ received and cached the pmenv.root, and generate and run received the pmenv.pacman. Now __init__ receives and caches the pmenv, allowing any method to fetch root, pacman, or other pmenv goodies directly instead of needing the extra parameter. Moved the "which fakeroot" and "which fakechroot" logic from pmtest up into pmenv. Repeating the probes and warnings for each test was a bit redundant. Signed-off-by: Jeremy Heiner <ScalaProtractor at gmail> --- test/pacman/pmenv.py | 28 +++++++++++++++-- test/pacman/pmrule.py | 10 +++--- test/pacman/pmtest.py | 87 +++++++++++++++++++++++---------------------------- 3 files changed, 70 insertions(+), 55 deletions(-) diff --git a/test/pacman/pmenv.py b/test/pacman/pmenv.py index f358285..3cfcfa5 100644 --- a/test/pacman/pmenv.py +++ b/test/pacman/pmenv.py @@ -21,6 +21,7 @@ import pmtest import tap +import util class pmenv(object): @@ -43,6 +44,27 @@ def __init__(self, root = "root"): "nolog": 0 } + if os.geteuid() != 0: + self.fakeroot = util.which("fakeroot") + if not self.fakeroot: + tap.diag("WARNING: fakeroot not found!") + else: + self.fakeroot = "fakeroot" + + self.fakechroot = util.which("fakechroot") + if not self.fakechroot: + tap.diag("WARNING: fakechroot not found!") + else: + self.fakechroot = "fakechroot" + + def cmdroot(self): + cmd = [] + if self.fakeroot: + cmd.append(self.fakeroot) + if self.fakechroot: + cmd.append(self.fakechroot) + return cmd + def __str__(self): return "root = %s\n" \ "pacman = %s" \ @@ -53,7 +75,7 @@ def addtest(self, testcase): """ if not os.path.isfile(testcase): raise IOError("test file %s not found" % testcase) - test = pmtest.pmtest(testcase, self.root) + test = pmtest.pmtest(testcase, self) self.testcases.append(test) def run(self): @@ -64,8 +86,8 @@ def run(self): tap.diag("Running '%s'" % t.testname) t.load() - t.generate(self.pacman) - t.run(self.pacman) + t.generate() + t.run() tap.diag("==> Checking rules") tap.todo = t.expectfailure diff --git a/test/pacman/pmrule.py b/test/pacman/pmrule.py index c97a158..718c87c 100644 --- a/test/pacman/pmrule.py +++ b/test/pacman/pmrule.py @@ -56,7 +56,7 @@ def check(self, test): if test.retcode != int(key): success = 0 elif case == "OUTPUT": - logfile = os.path.join(test.root, util.LOGFILE) + logfile = os.path.join(test.env.root, util.LOGFILE) if not os.access(logfile, os.F_OK): tap.diag("LOGFILE not found, cannot validate 'OUTPUT' rule") success = 0 @@ -112,7 +112,7 @@ def check(self, test): tap.diag("PKG rule '%s' not found" % case) success = -1 elif kind == "FILE": - filename = os.path.join(test.root, key) + filename = os.path.join(test.env.root, key) if case == "EXIST": if not os.path.isfile(filename): success = 0 @@ -152,7 +152,7 @@ def check(self, test): tap.diag("FILE rule '%s' not found" % case) success = -1 elif kind == "DIR": - filename = os.path.join(test.root, key) + filename = os.path.join(test.env.root, key) if case == "EXIST": if not os.path.isdir(filename): success = 0 @@ -160,7 +160,7 @@ def check(self, test): tap.diag("DIR rule '%s' not found" % case) success = -1 elif kind == "LINK": - filename = os.path.join(test.root, key) + filename = os.path.join(test.env.root, key) if case == "EXIST": if not os.path.islink(filename): success = 0 @@ -168,7 +168,7 @@ def check(self, test): tap.diag("LINK rule '%s' not found" % case) success = -1 elif kind == "CACHE": - cachedir = os.path.join(test.root, util.PM_CACHEDIR) + cachedir = os.path.join(test.env.root, util.PM_CACHEDIR) if case == "EXISTS": pkg = test.findpkg(key, value, allow_local=True) if not pkg or not os.path.isfile( diff --git a/test/pacman/pmtest.py b/test/pacman/pmtest.py index b343d55..63d25e8 100644 --- a/test/pacman/pmtest.py +++ b/test/pacman/pmtest.py @@ -35,20 +35,20 @@ class pmtest(object): """Test object """ - def __init__(self, name, root): + def __init__(self, name, env): self.name = name self.testname = os.path.basename(name).replace('.py', '') - self.root = root + self.env = env self.cachepkgs = True def __str__(self): return "name = %s\n" \ "testname = %s\n" \ - "root = %s" % (self.name, self.testname, self.root) + "root = %s" % (self.name, self.testname, self.env.root) def addpkg2db(self, treename, pkg): if not treename in self.db: - self.db[treename] = pmdb.pmdb(treename, self.root) + self.db[treename] = pmdb.pmdb(treename, self.env.root) self.db[treename].pkgs.append(pkg) def addpkg(self, pkg): @@ -83,7 +83,7 @@ def load(self): self.args = "" self.retcode = 0 self.db = { - "local": pmdb.pmdb("local", self.root) + "local": pmdb.pmdb("local", self.env.root) } self.localpkgs = [] self.createlocalpkgs = False @@ -104,37 +104,40 @@ def load(self): else: raise IOError("file %s does not exist!" % self.name) - def generate(self, pacman): + def generate(self): tap.diag("==> Generating test environment") + root = self.env.root + pacman = self.env.pacman + # Cleanup leftover files from a previous test session - if os.path.isdir(self.root): - shutil.rmtree(self.root) - vprint("\t%s" % self.root) + if os.path.isdir(root): + shutil.rmtree(root) + vprint("\t%s" % root) # Create directory structure vprint(" Creating directory structure:") - dbdir = os.path.join(self.root, util.PM_SYNCDBPATH) - cachedir = os.path.join(self.root, util.PM_CACHEDIR) - syncdir = os.path.join(self.root, util.SYNCREPO) - tmpdir = os.path.join(self.root, util.TMPDIR) - logdir = os.path.join(self.root, os.path.dirname(util.LOGFILE)) - etcdir = os.path.join(self.root, os.path.dirname(util.PACCONF)) - bindir = os.path.join(self.root, "bin") + dbdir = os.path.join(root, util.PM_SYNCDBPATH) + cachedir = os.path.join(root, util.PM_CACHEDIR) + syncdir = os.path.join(root, util.SYNCREPO) + tmpdir = os.path.join(root, util.TMPDIR) + logdir = os.path.join(root, os.path.dirname(util.LOGFILE)) + etcdir = os.path.join(root, os.path.dirname(util.PACCONF)) + bindir = os.path.join(root, "bin") ldconfig = os.path.basename(pacman["ldconfig"]) - ldconfigdir = os.path.join(self.root, os.path.dirname(pacman["ldconfig"][1:])) + ldconfigdir = os.path.join(root, os.path.dirname(pacman["ldconfig"][1:])) shell = pacman["scriptlet-shell"][1:] - shelldir = os.path.join(self.root, os.path.dirname(shell)) + shelldir = os.path.join(root, os.path.dirname(shell)) sys_dirs = [dbdir, cachedir, syncdir, tmpdir, logdir, etcdir, bindir, ldconfigdir, shelldir] for sys_dir in sys_dirs: if not os.path.isdir(sys_dir): - vprint("\t%s" % sys_dir[len(self.root)+1:]) + vprint("\t%s" % sys_dir[len(root)+1:]) os.makedirs(sys_dir, 0755) # Only the dynamically linked binary is needed for fakechroot shutil.copy("/bin/sh", bindir) if shell != "bin/sh": - shutil.copy("/bin/sh", os.path.join(self.root, shell)) + shutil.copy("/bin/sh", os.path.join(root, shell)) shutil.copy(os.path.join(util.SELFPATH, "ldconfig.stub"), os.path.join(ldconfigdir, ldconfig)) ld_so_conf = open(os.path.join(etcdir, "ld.so.conf"), "w") @@ -142,7 +145,7 @@ def generate(self, pacman): # Configuration file vprint(" Creating configuration file") - util.mkcfgfile(util.PACCONF, self.root, self.option, self.db) + util.mkcfgfile(util.PACCONF, root, self.option, self.db) # Creating packages vprint(" Creating package archives") @@ -174,24 +177,24 @@ def generate(self, pacman): vprint(" Populating file system") for f in self.filesystem: vprint("\t%s" % f) - util.mkfile(self.root, f, f) - path = os.path.join(self.root, f) + util.mkfile(root, f, f) + path = os.path.join(root, f) if os.path.isfile(path): os.utime(path, (355, 355)) for pkg in self.db["local"].pkgs: vprint("\tinstalling %s" % pkg.fullname()) - pkg.install_package(self.root) + pkg.install_package(root) # Done. vprint(" Taking a snapshot of the file system") - for roots, dirs, files in os.walk(self.root): + for roots, dirs, files in os.walk(root): for i in files: filename = os.path.join(roots, i) - f = pmfile.PacmanFile(self.root, filename.replace(self.root + "/", "")) + f = pmfile.PacmanFile(root, filename.replace(root + "/", "")) self.files.append(f) vprint("\t%s" % f.name) - def run(self, pacman): + def run(self): if os.path.isfile(util.PM_LOCK): tap.bail("\tERROR: another pacman session is on-going -- skipping") return @@ -199,19 +202,9 @@ def run(self, pacman): tap.diag("==> Running test") vprint("\tpacman %s" % self.args) - cmd = [] - if os.geteuid() != 0: - fakeroot = util.which("fakeroot") - if not fakeroot: - tap.diag("WARNING: fakeroot not found!") - else: - cmd.append("fakeroot") - - fakechroot = util.which("fakechroot") - if not fakechroot: - tap.diag("WARNING: fakechroot not found!") - else: - cmd.append("fakechroot") + root = self.env.root + pacman = self.env.pacman + cmd = self.env.cmdroot() if pacman["gdb"]: cmd.extend(["libtool", "execute", "gdb", "--args"]) @@ -223,17 +216,17 @@ def run(self, pacman): "--show-reachable=yes", "--suppressions=%s" % suppfile]) cmd.extend([pacman["bin"], - "--config", os.path.join(self.root, util.PACCONF), - "--root", self.root, - "--dbpath", os.path.join(self.root, util.PM_DBPATH), - "--cachedir", os.path.join(self.root, util.PM_CACHEDIR)]) + "--config", os.path.join(root, util.PACCONF), + "--root", root, + "--dbpath", os.path.join(root, util.PM_DBPATH), + "--cachedir", os.path.join(root, util.PM_CACHEDIR)]) if not pacman["manual-confirm"]: cmd.append("--noconfirm") if pacman["debug"]: cmd.append("--debug=%s" % pacman["debug"]) cmd.extend(shlex.split(self.args)) if not (pacman["gdb"] or pacman["valgrind"] or pacman["nolog"]): - output = open(os.path.join(self.root, util.LOGFILE), 'w') + output = open(os.path.join(root, util.LOGFILE), 'w') else: output = None vprint("\trunning: %s" % " ".join(cmd)) @@ -242,7 +235,7 @@ def run(self, pacman): # archives are made available more easily. time_start = time.time() self.retcode = subprocess.call(cmd, stdout=output, stderr=output, - cwd=os.path.join(self.root, util.TMPDIR), env={'LC_ALL': 'C'}) + cwd=os.path.join(root, util.TMPDIR), env={'LC_ALL': 'C'}) time_end = time.time() vprint("\ttime elapsed: %.2fs" % (time_end - time_start)) @@ -256,7 +249,7 @@ def run(self, pacman): tap.diag("\tERROR: %s not removed" % util.PM_LOCK) os.unlink(util.PM_LOCK) # Look for a core file - if os.path.isfile(os.path.join(self.root, util.TMPDIR, "core")): + if os.path.isfile(os.path.join(root, util.TMPDIR, "core")): tap.diag("\tERROR: pacman dumped a core file") def check(self): -- 1.8.4

2 2