March 2009 - Pacman-dev - lists.archlinux.org

[pacman-dev] [PATCH] Revert "popen does NOT require /bin/sh in a subchroot"
by Xavier Chantry 15 Mar '09

15 Mar '09

This reverts commit 9558639d8009483fbf422b138d020745986f82f1. This change was wrong, popen does require /bin/sh in a subchroot. 1) pacman -S lilo -r root Notice no error 2) rm root/bin/sh ; pacman -S lilo -r root Notice an error : error: scriptlet failed to execute correctly Actually, we already get an explicit error here, when popen is run, so there is no need to check for bin/sh explicitely. Besides this check was problematic in some cases. For example, bash itself has a scriptlet, but only post_install and post_upgrade, no pre_install and pre_upgrade. However, since bash has a scriptlet, runscriptlet will also be called before bash is installed. It won't do anything since the scriptlet has no pre_install function. But if we keep the check, we will still get "error : no /bin/sh". Conflicts: lib/libalpm/trans.c Signed-off-by: Xavier Chantry <shiningxc(a)gmail.com> --- lib/libalpm/trans.c | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) diff --git a/lib/libalpm/trans.c b/lib/libalpm/trans.c index 4b83119..32c6d90 100644 --- a/lib/libalpm/trans.c +++ b/lib/libalpm/trans.c @@ -443,13 +443,6 @@ int _alpm_runscriptlet(const char *root, const char *installfn, return(0); } - /* NOTE: popen will use the PARENT's /bin/sh, not the chroot's */ - if(access("/bin/sh", X_OK)) { - /* not found */ - _alpm_log(PM_LOG_ERROR, _("No /bin/sh in parent environment, aborting scriptlet\n")); - return(0); - } - /* creates a directory in $root/tmp/ for copying/extracting the scriptlet */ snprintf(tmpdir, PATH_MAX, "%stmp/", root); if(access(tmpdir, F_OK) != 0) { -- 1.6.2

3 2

[pacman-dev] [PATCH] Only check checksums if there is a source file
by Allan McRae 14 Mar '09

14 Mar '09

Signed-off-by: Allan McRae <allan(a)archlinux.org> --- scripts/makepkg.sh.in | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 2ff98f1..3ff256f 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -547,6 +547,8 @@ generate_checksums() { } check_checksums() { + [ ${#source[@]} -eq 0 ] && return 0 + if [ ! $(type -p openssl) ]; then error "$(gettext "Cannot find openssl.")" exit 1 # $E_MISSING_PROGRAM -- 1.6.2

1 0

[pacman-dev] delay writes a fsync
by Allan McRae 14 Mar '09

14 Mar '09

Hi, You have probably heard about this: http://linux.slashdot.org/article.pl?sid=09/03/11/2031231 Here is the link to the bug report that describes the cause of the problem: https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54 A key part of the bug report comment... Applications are expected to use fsync() or fdatasync(), and if that impacts their performance too much, to use a single berkdb or other binary database file, and not do something stupid with hundreds of tiny text files that only hold a few bytes of data in each text file. Hmmm... something sound familiar there? So what should we do? Maybe, add some sort of syncing after a db write. But then after each file or just at the end of the transaction? Allan

8 19

[pacman-dev] delta support in libalpm
by Henning Garus 14 Mar '09

14 Mar '09

Hi, I have been looking through the current delta implementation in libalpm and have put some thought into changing makepkg/repo-add to support delta creation. However, I'm running into some problems, mostly due to md5sums and gzip. The current implementation works as follows. On a sync operation it is checked, whether a valid delta path exists and if the summed filesize of the deltas is smaller than the filesize of the whole download. When this is the case the deltas are downloaded and applied to the old file. After that the patched file is treated as if it was downloaded normally, this includes a check of the md5sum. Gzip files have a header, that has a timestamp, which will screw with this md5sum. When a patch is applied to a gzipped file by xdelta, xdelta will unzip the file, apply the patch and then rezip the file. The author of xdelta was obviously aware of the problems with the timestamp, because he decided to leave it empty. The same can be achieved by the -n option of gzip. But there comes the next problem, xdelta uses zlib for compression, gzip implements compression itself. And files created by gzip can differ from files created by zlib. Bsdtar uses zlib as well, but writes the timestamp and there is no option to prevent this (at least none that I can see). There are four ways around this, that I can think of: 1. create the package, then create the delta, apply the delta to the old version, remove the original new package and present the patched package as output I think this sucks, this ties delta creation to makepkg (more about that later) and has an incredibly huge and useless overhead (countless unzips and rezips and applying the patch). 2. create the package, but don't compress it with bsdtar, use gzip -n instead. This means we have to use gzip again, in libalpm, when we apply the delta. Seems better than 1, but makes makepkg and libalpm rely on gzip. Not sure if this is a good thing, especially for libalpm. 3. save the md5sums of the unzipped tars in the synchdb and change libalpm to check those Seems reasonable, but I don't see a way to do this with libarchive, so this would require using zlib directly and pacman would lose the ability to handle to handle tar.bz2 4. Skip checking the md5sum for deltas OK during the initial synch, as long as we trust xdelta to do its job (the md5sums of both the old and the new file are in the delta file). But the created package will have the wrong md5sum and can't be used to reinstall, etc. which makes this look like a bad idea. In a previous mail Xavier toyed with the idea to put delta creation into repo-add, I have given this some thought, as it seems nice in principle, but there are drawbacks. For Arch this would mean creating deltas on Gerolde, which seems to be fairly strained already, according to the dev list. Furthermore this introduces some new variables to repo-add (at least repo location and an output location) this would be manageable, but doesn't look very nice. Delta creation in makepkg seems somehow ok (its already in there after all). But what I would really like is a separate tool for delta creation, which would allow the separation of building packages and creating deltas and setting up a separated delta server. This leaves us with options 2 and 3 and I am not really sure, which way to go. looking forward to your comments

6 40

[pacman-dev] Checking for VSC/SCM tools in makepkg
by Allan McRae 09 Mar '09

09 Mar '09

Hi, Am I missing something here or should these sorts of tests cause a failure: devel_check() { ... # Also do a brief check to make sure we have the VCS tool available. ... [ $(type -p darcs) ] || return 0 ... [ $(type -p svn) ] || return 0 etc. At the moment, these tests just seem useless to me. Making this an error would require a decent change in makepkg as at the moment devel_check() is called before dependency resolution is done. In fact, I don't think I have ever managed to build an SCM package in a clean chroot and this may be the cause. If no-one points out something obvious I am missing, then I will file a bug report so this is not forgotten. Allan

2 4

[pacman-dev] Useless comments...
by Allan McRae 09 Mar '09

09 Mar '09

Hi, if (you don't want to read a semi rant) exit 0; I have noticed a number of useless comments in makepkg and have decided that they are worse than no comments. Examples: 1) This inspired the rant! download_sources # we can only check checksums if we have all files check_checksums Huh, why are we not checking for all files then? Because download_sources exits when it fails... 2) and there are a lot of these: # fix flyspray feature request #2978 # fix flyspray bug #5923 # Fixes FS#10039 # fix flyspray #6246 #fix flyspray feature request #5223 # fix flyspray bug #5973 and I am guilty here... but I was at the airport with no internet access so I had no idea if these are important. And most of these appeared to be non-obscure features/fixes so did not need a comment justifying their inclusion. 3) overly obvious comments # do we have a changelog? if [ -f "$startdir/ChangeLog" ]; then If you do not understand that test, then leave the code alone... I will patch these out eventually but I like to share my frustration. I am just a generous guy! Allan

4 3

[pacman-dev] [PATCH] Remove duplicates from the unresolvable list before prompting user
by bji-keyword-pacman.3644cb＠www.ischo.com 08 Mar '09

08 Mar '09

From: Bryan Ischo <bji-keyword-pacman.3644cb(a)www.ischo.com> Signed-off-by: Bryan Ischo <bji-keyword-pacman.3644cb(a)www.ischo.com> --- lib/libalpm/sync.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/lib/libalpm/sync.c b/lib/libalpm/sync.c index fca96d8..cac6783 100644 --- a/lib/libalpm/sync.c +++ b/lib/libalpm/sync.c @@ -444,6 +444,7 @@ int _alpm_sync_prepare(pmtrans_t *trans, pmdb_t *db_local, alpm_list_t *dbs_sync /* If there were unresolvable top-level packages, prompt the user to see if they'd like to ignore them rather than failing the sync */ if(unresolvable != NULL) { + unresolvable = alpm_list_remove_dupes(unresolvable); int remove_unresolvable = 0; QUESTION(handle->trans, PM_TRANS_CONV_REMOVE_PKGS, unresolvable, NULL, NULL, &remove_unresolvable); -- 1.6.1.3

1 0

[pacman-dev] [PATCH] Look in target-list first to resolve dependencies
by bji-keyword-pacman.3644cb＠www.ischo.com 08 Mar '09

08 Mar '09

From: Bryan Ischo <bji-keyword-pacman.3644cb(a)www.ischo.com> This fixes a bug introduced by my previous changes which changes the behavior of IgnorePkg/IgnoreGroup to allow the user to remove unresolvable packages from the transaction. The bug is that the target-list was no longer being consulted first to resolve dependencies, which means that if two packages in the sync database satisfied a dependency, and the user explicitly requested one of those two packages in the sync, the other package was still being pulled in. A new test was added, sync993.py, to verify the desired behavior. Signed-off-by: Bryan Ischo <bji-keyword-pacman.3644cb(a)www.ischo.com> --- lib/libalpm/deps.c | 20 ++++++++++++-------- lib/libalpm/deps.h | 3 ++- lib/libalpm/sync.c | 7 ++++--- pactest/tests/sync993.py | 20 ++++++++++++++++++++ 4 files changed, 38 insertions(+), 12 deletions(-) create mode 100644 pactest/tests/sync993.py diff --git a/lib/libalpm/deps.c b/lib/libalpm/deps.c index 694e5be..0faa221 100644 --- a/lib/libalpm/deps.c +++ b/lib/libalpm/deps.c @@ -582,8 +582,8 @@ pmpkg_t *_alpm_resolvedep(pmdepend_t *dep, alpm_list_t *dbs, * unmodified by this function */ int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, pmpkg_t *pkg, - alpm_list_t **packages, alpm_list_t *remove, - alpm_list_t **data) + alpm_list_t *preferred, alpm_list_t **packages, + alpm_list_t *remove, alpm_list_t **data) { alpm_list_t *i, *j; alpm_list_t *targ; @@ -620,17 +620,21 @@ int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, pmpkg_t *pkg, if(_alpm_find_dep_satisfier(*packages, missdep)) { continue; } - /* find a satisfier package in the given repositories */ - pmpkg_t *spkg = _alpm_resolvedep(missdep, dbs_sync, *packages, 0); + /* check if one of the packages in the [preferred] list already satisfies this dependency */ + pmpkg_t *spkg = _alpm_find_dep_satisfier(preferred, missdep); + if(!spkg) { + /* find a satisfier package in the given repositories */ + spkg = _alpm_resolvedep(missdep, dbs_sync, *packages, 0); + } if(!spkg) { pm_errno = PM_ERR_UNSATISFIED_DEPS; char *missdepstring = alpm_dep_compute_string(missdep); _alpm_log(PM_LOG_WARNING, _("cannot resolve \"%s\", a dependency of \"%s\"\n"), - missdepstring, tpkg->name); + missdepstring, tpkg->name); free(missdepstring); if(data) { pmdepmissing_t *missd = _alpm_depmiss_new(miss->target, - miss->depend, miss->causingpkg); + miss->depend, miss->causingpkg); if(missd) { *data = alpm_list_add(*data, missd); } @@ -640,9 +644,9 @@ int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, pmpkg_t *pkg, alpm_list_free_inner(deps, (alpm_list_fn_free)_alpm_depmiss_free); alpm_list_free(deps); return(-1); - } else { + } else { _alpm_log(PM_LOG_DEBUG, "pulling dependency %s (needed by %s)\n", - alpm_pkg_get_name(spkg), alpm_pkg_get_name(tpkg)); + alpm_pkg_get_name(spkg), alpm_pkg_get_name(tpkg)); *packages = alpm_list_add(*packages, spkg); } } diff --git a/lib/libalpm/deps.h b/lib/libalpm/deps.h index eb1400b..94847d8 100644 --- a/lib/libalpm/deps.h +++ b/lib/libalpm/deps.h @@ -49,7 +49,8 @@ alpm_list_t *_alpm_sortbydeps(alpm_list_t *targets, int reverse); void _alpm_recursedeps(pmdb_t *db, alpm_list_t *targs, int include_explicit); pmpkg_t *_alpm_resolvedep(pmdepend_t *dep, alpm_list_t *dbs, alpm_list_t *excluding, int prompt); int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, pmpkg_t *pkg, - alpm_list_t **packages, alpm_list_t *remove, alpm_list_t **data); + alpm_list_t *preferred, alpm_list_t **packages, alpm_list_t *remove, + alpm_list_t **data); int _alpm_dep_edge(pmpkg_t *pkg1, pmpkg_t *pkg2); pmdepend_t *_alpm_splitdep(const char *depstring); pmpkg_t *_alpm_find_dep_satisfier(alpm_list_t *pkgs, pmdepend_t *dep); diff --git a/lib/libalpm/sync.c b/lib/libalpm/sync.c index fca96d8..b504a42 100644 --- a/lib/libalpm/sync.c +++ b/lib/libalpm/sync.c @@ -396,7 +396,7 @@ static int compute_download_size(pmpkg_t *newpkg) int _alpm_sync_prepare(pmtrans_t *trans, pmdb_t *db_local, alpm_list_t *dbs_sync, alpm_list_t **data) { - alpm_list_t *deps = NULL; + alpm_list_t *deps = NULL, *preferred = NULL; alpm_list_t *list = NULL, *remove = NULL; /* allow checkdeps usage with trans->packages */ alpm_list_t *unresolvable = NULL; alpm_list_t *i, *j; @@ -422,19 +422,20 @@ int _alpm_sync_prepare(pmtrans_t *trans, pmdb_t *db_local, alpm_list_t *dbs_sync EVENT(trans, PM_TRANS_EVT_RESOLVEDEPS_START, NULL, NULL); _alpm_log(PM_LOG_DEBUG, "resolving target's dependencies\n"); - /* build remove list for resolvedeps */ + /* build remove list and preferred list for resolvedeps */ for(i = trans->packages; i; i = i->next) { pmsyncpkg_t *sync = i->data; for(j = sync->removes; j; j = j->next) { remove = alpm_list_add(remove, j->data); } + preferred = alpm_list_add(preferred, sync->pkg); } /* Resolve packages in the transaction one at a time, in addtion building up a list of packages which could not be resolved. */ for(i = trans->packages; i; i = i->next) { pmpkg_t *pkg = ((pmsyncpkg_t *) i->data)->pkg; - if(_alpm_resolvedeps(db_local, dbs_sync, pkg, &list, remove, data) == -1) { + if(_alpm_resolvedeps(db_local, dbs_sync, pkg, preferred, &list, remove, data) == -1) { unresolvable = alpm_list_add(unresolvable, pkg); } /* Else, [list] now additionally contains [pkg] and all of its diff --git a/pactest/tests/sync993.py b/pactest/tests/sync993.py new file mode 100644 index 0000000..82cc803 --- /dev/null +++ b/pactest/tests/sync993.py @@ -0,0 +1,20 @@ +self.description = "Choose a dependency satisfier (target-list vs. database)" + +sp1 = pmpkg("pkg1") +sp1.depends = ["dep"] + +sp2 = pmpkg("pkg2") +sp2.provides = ["dep"] + +sp3 = pmpkg("pkg3") +sp3.provides = ["dep"] + +for p in sp1, sp2, sp3: + self.addpkg2db("sync", p) + +self.args = "-S pkg1 pkg3" + +self.addrule("PACMAN_RETCODE=0") +self.addrule("PKG_EXIST=pkg1") +self.addrule("!PKG_EXIST=pkg2") +self.addrule("PKG_EXIST=pkg3") -- 1.6.1.3

1 0

[pacman-dev] [PATCH] Fixed a memory leak when unresolved packages are removed from transaction
by bji-keyword-pacman.3644cb＠www.ischo.com 07 Mar '09

07 Mar '09

From: Bryan Ischo <bji-keyword-pacman.3644cb(a)www.ischo.com> Signed-off-by: Bryan Ischo <bji-keyword-pacman.3644cb(a)www.ischo.com> --- lib/libalpm/sync.c | 21 ++++++++++++++++++++- 1 files changed, 20 insertions(+), 1 deletions(-) diff --git a/lib/libalpm/sync.c b/lib/libalpm/sync.c index fca96d8..fe4eded 100644 --- a/lib/libalpm/sync.c +++ b/lib/libalpm/sync.c @@ -431,7 +431,10 @@ int _alpm_sync_prepare(pmtrans_t *trans, pmdb_t *db_local, alpm_list_t *dbs_sync } /* Resolve packages in the transaction one at a time, in addtion - building up a list of packages which could not be resolved. */ + building up a list of packages which could not be resolved. + The list which is created by this loop is used later to replace + trans->packages; the unresolvable packages which were in the old + trans->packages list but not in the new list are freed up later. */ for(i = trans->packages; i; i = i->next) { pmpkg_t *pkg = ((pmsyncpkg_t *) i->data)->pkg; if(_alpm_resolvedeps(db_local, dbs_sync, pkg, &list, remove, data) == -1) { @@ -500,6 +503,22 @@ int _alpm_sync_prepare(pmtrans_t *trans, pmdb_t *db_local, alpm_list_t *dbs_sync } } alpm_list_free(sortlist); + + /* Destroy all packages which are in trans->packages but not in + newpkgs before replacing trans->packages with newpkgs */ + for(i = trans->packages; i; i = i->next) { + pmsyncpkg_t *spkg = (pmsyncpkg_t *) i->data; + for (j = newpkgs; j; j = j->next) { + if(_alpm_pkg_cmp(spkg->pkg, ((pmsyncpkg_t *) j->data)->pkg) == 0) { + spkg = NULL; + break; + } + } + if (spkg != NULL) { + _alpm_sync_free(spkg); + } + } + alpm_list_free(trans->packages); trans->packages = newpkgs; -- 1.6.1.3

3 2

[pacman-dev] [PATCH] Fix error handling in _alpm_resolvedeps
by Nagy Gabor 07 Mar '09

07 Mar '09

>From 816e0ec31278ff9e94b0d8dcdea24dadb5e4fa1b Mon Sep 17 00:00:00 2001 From: Nagy Gabor <ngaba(a)bibl.u-szeged.hu> Date: Fri, 6 Mar 2009 17:02:19 +0100 Subject: [PATCH] Fix error handling in _alpm_resolvedeps Now resolvedeps is just a helper function for sync_prepare, so we set pm_errno in sync_prepare. We free *data list, when user decided to remove unresolvable targets. Signed-off-by: Nagy Gabor <ngaba(a)bibl.u-szeged.hu> --- lib/libalpm/deps.c | 1 - lib/libalpm/sync.c | 7 ++++++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/lib/libalpm/deps.c b/lib/libalpm/deps.c index 36f4d35..7a46692 100644 --- a/lib/libalpm/deps.c +++ b/lib/libalpm/deps.c @@ -635,7 +635,6 @@ int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, pmpkg_t *pkg, /* find a satisfier package in the given repositories */ pmpkg_t *spkg = _alpm_resolvedep(missdep, dbs_sync, *packages, 0); if(!spkg) { - pm_errno = PM_ERR_UNSATISFIED_DEPS; char *missdepstring = alpm_dep_compute_string(missdep); _alpm_log(PM_LOG_WARNING, _("cannot resolve \"%s\", a dependency of \"%s\"\n"), missdepstring, tpkg->name); diff --git a/lib/libalpm/sync.c b/lib/libalpm/sync.c index 84a18e2..14cf42e 100644 --- a/lib/libalpm/sync.c +++ b/lib/libalpm/sync.c @@ -456,9 +456,14 @@ int _alpm_sync_prepare(pmtrans_t *trans, pmdb_t *db_local, alpm_list_t *dbs_sync dependency-reordered list below */ alpm_list_free(unresolvable); unresolvable = NULL; + if(data) { + alpm_list_free_inner(*data, (alpm_list_fn_free)_alpm_depmiss_free); + alpm_list_free(*data); + *data = NULL; + } } else { - /* pm_errno is set by resolvedeps */ + pm_errno = PM_ERR_UNSATISFIED_DEPS; ret = -1; goto cleanup; } -- 1.6.0.3

1 1