- Pacman-dev - lists.archlinux.org

[pacman-dev] Draft pacman-3.5.0 NEWS
by Allan McRae 27 Feb '11

27 Feb '11

For comment. I have not written the epoch stuff yet... VERSION DESCRIPTION ----------------------------------------------------------------------------- 3.5.0 - database format changes - sync databases are read directly from the repo database tarball (FS#8586, FS#20233) - local database "depends" files have been merged into the "desc" file - pacman-db-upgrade script provided to update the local database format - sync database extension is .db (without compression suffix) - requires repo-add from pacman-3.4+ - epoch - TODO!!! - disk space checking - check available space before installing packages (FS#11639) - enabled by the "CheckSpace" option in pacman.conf - attempt to stop install if we hit an extraction issue (FS#7692, FS#22034) - improved groups/providers selction dialog (FS#19704, FS#19853) - more fine grained controll of ignorine dependency resolution - -Sd to ignore dependency versions only - -Sdd to ignore all dependency information - clean-up of --help output (FS#19526) - CleanMethod for package cache cleaning can use both KeepInstalled and KeepCurrent simultaneously - various speed-ups: - improved internal storage of the package cache - faster pkgname/depends searches - use openssl crypto functions if available - makepkg: - add support for running testsuites in a check() function (FS#15145) - controlled by BUILDENV option 'check' in makepkg.conf which may be overridden by --check/--nocheck on the commandline - extract any file bsdtar recognizes - STRIP_DIRS has been removed in favour of stripping all recognized files - improve $srcdir/$pkgdir check to reduce false positives - $pkgname can be used in split package() functions (FS#22174) - added '!buildflags' option to allow unsetting CFLAGS/LDFLAGS/etc - repo-add: - added -f/--files to create files database (FS#11302) - pactree: rewritten in C using libalpm

2 1

[pacman-dev] Removing empty directories
by Allan McRae 27 Feb '11

27 Feb '11

I just noticed this: > pacman -Qk | grep "^warning" warning: filesystem: /opt/ (No such file or directory) warning: hal: /usr/share/hal/fdi/information/20thirdparty/ (No such file or directory) warning: hal-info: /usr/share/hal/fdi/information/20thirdparty/ (No such file or directory) warning: hicolor-icon-theme: /usr/share/icons/hicolor/256x256/apps/ (No such file or directory) warning: network-manager-applet: /etc/gconf/schemas/ (No such file or directory) All these directories are empty in their package. They appear to have been removed when I uninstalled a different package that had files in that directory. Maybe we need some more checking before removing an empty directory. Allan

4 8

[pacman-dev] [PATCH 1/2] pactest for removing a required empty directory
by Allan McRae 27 Feb '11

27 Feb '11

Signed-off-by: Allan McRae <allan(a)archlinux.org> --- test/pacman/tests/remove070.py | 21 +++++++++++++++++++++ 1 files changed, 21 insertions(+), 0 deletions(-) create mode 100644 test/pacman/tests/remove070.py diff --git a/test/pacman/tests/remove070.py b/test/pacman/tests/remove070.py new file mode 100644 index 0000000..e0587e1 --- /dev/null +++ b/test/pacman/tests/remove070.py @@ -0,0 +1,21 @@ +self.description = "Remove a package with an empty directory needed by another package" + +p1 = pmpkg("pkg1") +p1.files = ["bin/pkg1", "opt/"] + +p2 = pmpkg("pkg2") +p2.files = ["bin/pkg2", "opt/"] + +for p in p1, p2: + self.addpkg2db("local", p) + +self.args = "-R %s" % p1.name + +self.addrule("PACMAN_RETCODE=0") +self.addrule("!PKG_EXIST=pkg1") +self.addrule("PKG_EXIST=pkg2") +self.addrule("!FILE_EXIST=bin/pkg1") +self.addrule("FILE_EXIST=bin/pkg2") +self.addrule("FILE_EXIST=opt/") + +self.expectfailure = True -- 1.7.4.1

1 0

[pacman-dev] [PATCH] Fix double close of the lock file
by Jonathan Conder 27 Feb '11

27 Feb '11

According to FOPEN(3), using fclose on an fdopen'd file stream also closes the underlying file descriptor. This happened in _alpm_lckmk (util.c), which meant that when alpm_trans_release closed it again, the log file (which reused the original file descriptor) was closed instead. Signed-off-by: Jonathan Conder <jonno.conder(a)gmail.com> --- lib/libalpm/handle.c | 1 - lib/libalpm/handle.h | 2 +- lib/libalpm/trans.c | 13 +++++-------- lib/libalpm/util.c | 7 +++---- lib/libalpm/util.h | 2 +- 5 files changed, 10 insertions(+), 15 deletions(-) diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c index aa34cf4..8f44e94 100644 --- a/lib/libalpm/handle.c +++ b/lib/libalpm/handle.c @@ -49,7 +49,6 @@ pmhandle_t *_alpm_handle_new() ALPM_LOG_FUNC; CALLOC(handle, 1, sizeof(pmhandle_t), RET_ERR(PM_ERR_MEMORY, NULL)); - handle->lckfd = -1; return(handle); } diff --git a/lib/libalpm/handle.h b/lib/libalpm/handle.h index 1834994..6884666 100644 --- a/lib/libalpm/handle.h +++ b/lib/libalpm/handle.h @@ -34,7 +34,7 @@ typedef struct _pmhandle_t { pmdb_t *db_local; /* local db pointer */ alpm_list_t *dbs_sync; /* List of (pmdb_t *) */ FILE *logstream; /* log file stream pointer */ - int lckfd; /* lock file descriptor if one exists */ + FILE *lckstream; /* lock file stream pointer if one exists */ pmtrans_t *trans; /* callback functions */ diff --git a/lib/libalpm/trans.c b/lib/libalpm/trans.c index 02612ec..e13a7b2 100644 --- a/lib/libalpm/trans.c +++ b/lib/libalpm/trans.c @@ -73,8 +73,8 @@ int SYMEXPORT alpm_trans_init(pmtransflag_t flags, /* lock db */ if(!(flags & PM_TRANS_FLAG_NOLOCK)) { - handle->lckfd = _alpm_lckmk(); - if(handle->lckfd == -1) { + handle->lckstream = _alpm_lckmk(); + if(handle->lckstream == NULL) { RET_ERR(PM_ERR_HANDLE_LOCK, -1); } } @@ -260,12 +260,9 @@ int SYMEXPORT alpm_trans_release() /* unlock db */ if(!nolock_flag) { - if(handle->lckfd != -1) { - int fd; - do { - fd = close(handle->lckfd); - } while(fd == -1 && errno == EINTR); - handle->lckfd = -1; + if(handle->lckstream != NULL) { + fclose(handle->lckstream); + handle->lckstream = NULL; } if(_alpm_lckrm()) { _alpm_log(PM_LOG_WARNING, _("could not remove lock file %s\n"), diff --git a/lib/libalpm/util.c b/lib/libalpm/util.c index 32eaa44..4d152ee 100644 --- a/lib/libalpm/util.c +++ b/lib/libalpm/util.c @@ -197,7 +197,7 @@ char *_alpm_strtrim(char *str) } /* Create a lock file */ -int _alpm_lckmk() +FILE *_alpm_lckmk() { int fd; char *dir, *ptr; @@ -220,10 +220,9 @@ int _alpm_lckmk() fprintf(f, "%ld\n", (long)getpid()); fflush(f); fsync(fd); - fclose(f); - return(fd); + return(f); } - return(-1); + return(NULL); } /* Remove a lock file */ diff --git a/lib/libalpm/util.h b/lib/libalpm/util.h index 8a3154a..edd51d5 100644 --- a/lib/libalpm/util.h +++ b/lib/libalpm/util.h @@ -62,7 +62,7 @@ int _alpm_makepath(const char *path); int _alpm_makepath_mode(const char *path, mode_t mode); int _alpm_copyfile(const char *src, const char *dest); char *_alpm_strtrim(char *str); -int _alpm_lckmk(); +FILE *_alpm_lckmk(); int _alpm_lckrm(); int _alpm_unpack_single(const char *archive, const char *prefix, const char *fn); int _alpm_unpack(const char *archive, const char *prefix, alpm_list_t *list, int breakfirst); -- 1.7.4

3 6

[pacman-dev] Style questions and concerns
by Dan McGee 27 Feb '11

27 Feb '11

I've noticed these things a lot in recent patches, so let the discussion commence. 1) typedef-ed structs. This is just a "copy the rest of them" habit, but I really feel we should stop doing this, only typedef-ing when we are making a public-facing type that we don't want to expose the internal contents of. Everything else should remain a struct type, and referred to as such, no typedef and no confusion that the fields are accessible. 2) return(value) style. This is a rule set in stone in the coding style doc but people are continuously breaking it. To be fair, it is silly, but mixing styles sucks way worse. I propose switching this, *BUT* all current patches need to follow current style, and if we switch, it should happen post-3.5.0 and in one commit, not this piecemeal junk. 3) C99. Several patches are introducing things that go further down this road. I don't think they should all be avoided but I'm not sure GCC 3.X (Cygwin is still on this?) supports all of these. * Variable declarations not at the start of a block. I'm not a huge fan of this, as I think having them at the start of the block helps clarify scope a lot better. * Dynamically sized array declarations, e.g. data[numcpus]. I think this needs to be avoided, unfortunately, but it isn't hard to just use MALLOC/FREE in this case. * C99 structure initialization (saw it in the parallelize patches). Please chip in and discuss. -Dan

5 5

[pacman-dev] Paralellising integrity checks
by Tavian Barnes 26 Feb '11

26 Feb '11

Integrity checking in pacman seems to be a CPU-bound embarrassingly parallel task, so I'd like to spread it out over every available core to speed it up. To me it looks like both the delta and regular package integrity checking loops in lib/libalpm/sync.c could be parallelised. But I've never messed with the pacman code before, so I have a couple questions about your best practices: - There doesn't seem to be any threading code in pacman/libalpm currently; would using pthreads directly be okay, or should a threading abstraction layer be created? - There's no portable way to get the number of available cores. Where does platform-specific code go in libalpm? The way to do it on Linux is with sched_getaffinity(); sysconf(_SC_NPROCESSORS_ONLN) is almost as good and works on BSD too I believe. Also, I guess a major question would be: would this be likely to be merged? It's not a major issue, but it's also not a very large change. Every time there's a KDE update or something similarly large I find myself wishing that all 2/4/24 of my cores were being used for the the integrity checks. -- Tavian Barnes

4 10

[pacman-dev] License for new contributions?
by edmeister46＠hushmail.com 26 Feb '11

26 Feb '11

Hello pacman team! I've been following development for quite some time, and would like to submit my package signing patches for review. However, since some of the files are entirely new, they would have a license header. I would like to know under what license should I release my work. I bring this up because during this time I overlooked the inclusion of the rankmirrors script, which I've now noticed to be GPL v3 code. Should my files be GPL v2 or v3? Thank you for your time! Ed

2 1

[pacman-dev] Parallelize integrity checks
by Tavian Barnes 25 Feb '11

25 Feb '11

The following patches add a new _alpm_for_each_cpu() internal API function, and use it to spread out the workload of integrity checking deltas and packages over the available CPUs. I didn't end up dividing the packages by size, because the logic gets uglier for what is probably marginal benefit. So each thread gets roughly the same number of packages to check. It currently uses sysconf(_SC_NPROCESSORS_ONLN) if available to count the number of CPUs. Support for detecting the number of CPUs on other platforms should probably be added by people who can test on other platforms, but the git and x264 source code are both good examples.

4 6

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v3.4.3-363-g3149a45
by dan＠archlinux.org 25 Feb '11

25 Feb '11

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 3149a45bfbf05a02155f17bbd10fa58ec24f5655 (commit) via 1fcc49675640acc14ee50549d0143c5b8164bdf8 (commit) via eefe8c83644892b963b1b4e5fbe297fa4be1f119 (commit) via ab49bf6fa9df6daf7210c28ddfcf013ee9fe4386 (commit) via 2f96764058b89ff32c928a736308c5095f1ff764 (commit) via d4d304cdb7b59e3b5ab7d5825404593a3476f127 (commit) via 7c14e48776cf82b521cc8b7dad05892a81b8659b (commit) via 6735807c0f5ec5464f2977da96b8981e2321940a (commit) from cfa760203295e99f65234896036f854b5047c90e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3149a45bfbf05a02155f17bbd10fa58ec24f5655 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 25 09:48:20 2011 -0600 Add directory name to ownership error message If you were doing a -Qo via xargs, it is at least nice to see what input caused the error message to occur rather than having a bunch of plain messages. This matches the style when we can't find the owner of a file. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 1fcc49675640acc14ee50549d0143c5b8164bdf8 Author: Dave Reisner <d(a)falconindy.com> Date: Fri Feb 25 09:22:09 2011 -0500 alpm: alpm_db_get_pkgcache_list => alpm_db_get_pkgcache This avoids needless breakage of the public API. Signed-off-by: Dave Reisner <d(a)falconindy.com> Signed-off-by: Dan McGee <dan(a)archlinux.org> commit eefe8c83644892b963b1b4e5fbe297fa4be1f119 Author: Dave Reisner <d(a)falconindy.com> Date: Fri Feb 25 09:11:37 2011 -0500 alpm: remove public visibility of pmpkghash_t There's no API functions exposed which allow manipulation of this type, so remove it from public view. Also, rename the public and private alpm_db_get_pkgcache symbol to alpm_db_get_pkgcache_has. Signed-off-by: Dave Reisner <d(a)falconindy.com> Signed-off-by: Dan McGee <dan(a)archlinux.org> commit ab49bf6fa9df6daf7210c28ddfcf013ee9fe4386 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 25 07:57:41 2011 -0600 Add test case for util-linux/util-linux-ng name switch and deps This case currently fails, but highlights a failure in our install process I experienced the other day. Because we don't do replacement uninstalls inline with the rest of the upgrade uninstalls, we can have a time on our system where a critical package is not installed. I hope no one ever renames glibc. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 2f96764058b89ff32c928a736308c5095f1ff764 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 25 07:55:16 2011 -0600 Continue resolving dependencies rather than bailing on first error This allows error messages emitted by the frontend to be a bit more descriptive and not have the annoying "well why didn't you tell me that the first time" problem. If a package had multiple missing deps, we would bail on the first one before rather than finish processing all missing dependencies, and only print one error message. Instead, continue through this entire set of missing deps and append all eventual errors. The added pactest tests this case, as the to be installed package has two missing dependencies. However, pactest does not actually test or see the difference in output from before and after, so it passes in both cases, but it is clearly visible in the logs. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit d4d304cdb7b59e3b5ab7d5825404593a3476f127 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 25 07:51:58 2011 -0600 Various small spelling fixes and small tweaks Nothing that changes behavior here. Spelling fixes and pushing a variable down to the scope it is used. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 7c14e48776cf82b521cc8b7dad05892a81b8659b Author: Dan McGee <dan(a)archlinux.org> Date: Thu Feb 24 09:38:59 2011 -0600 Mark log callback format string const Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 6735807c0f5ec5464f2977da96b8981e2321940a Author: Dan McGee <dan(a)archlinux.org> Date: Tue Feb 22 16:48:51 2011 -0600 Remove trans->skip_add This is old code that has since gone stale; we no longer ever add anything to this list so no need to keep it around and check the contents during extraction. Signed-off-by: Dan McGee <dan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: lib/libalpm/add.c | 8 --- lib/libalpm/alpm.h | 8 +-- lib/libalpm/conflict.c | 2 +- lib/libalpm/db.c | 33 +++-------- lib/libalpm/db.h | 4 +- lib/libalpm/deps.c | 61 +++++++++++-------- lib/libalpm/log.c | 4 +- lib/libalpm/log.h | 2 +- lib/libalpm/package.c | 2 +- lib/libalpm/pkghash.h | 2 + lib/libalpm/remove.c | 6 +- lib/libalpm/sync.c | 10 ++-- lib/libalpm/trans.c | 1 - lib/libalpm/trans.h | 1 - src/pacman/callback.c | 2 +- src/pacman/callback.h | 2 +- src/pacman/deptest.c | 2 +- src/pacman/query.c | 8 +- src/pacman/sync.c | 10 ++-- src/util/cleanupdelta.c | 4 +- src/util/pactree.c | 4 +- src/util/testdb.c | 6 +- src/util/testpkg.c | 2 +- test/pacman/tests/replace100.py | 45 ++++++++++++++ test/pacman/tests/sync134.py | 2 +- test/pacman/tests/{upgrade081.py => upgrade077.py} | 9 ++- 26 files changed, 135 insertions(+), 105 deletions(-) create mode 100644 test/pacman/tests/replace100.py copy test/pacman/tests/{upgrade081.py => upgrade077.py} (51%) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] (no subject)
by Dave Reisner 25 Feb '11

25 Feb '11

The ensuing patches revert Allan's breakage^Wchanges to the public API with regard to the introduction of pmpkghash_t. Since we haven't exposed any public facing API calls which can manipulate a pmpkghash_t, there's not much point in exposing the type along with it. alpm_db_get_pkgcache_list is renamed back to alpm_db_get_pkgcache, and the calls which return a pmpkghash_t are explicitly renamed alpm_db_get_pkgcache_hash. dave

3 5