December 2008 - Pacman-dev - lists.archlinux.org

[pacman-dev] Pacman contributors/developers (was: Fix bug 9395)
by Dan McGee 31 Dec '08

31 Dec '08

On Wed, Dec 31, 2008 at 2:55 PM, Bryan Ischo <bji-keyword-pacman.3644cb(a)www.ischo.com> wrote: > This sounds like a good idea. But it would involve greater changes to > libalpm then I felt comfortable doing. Unfortunately, when one is just a > patch contributor rather than a real developer, one feels the need to keep > all changes as minimal as possible, instead of reaching into lots of parts > of the code and doing cleanups and refactoring and such. But I would be > happy to do some of that, once I feel more comfortable with the code base. Just wanted to set the record straight here- if you are a patch contributor, you are a developer in my mind. I don't think the old timers or new timers are perfectly happy with the shape libalpm is in, so feel free to hack things up if it results in long term improvement. We definitely have done a lot of refactoring and cleanup to the code, but there is no shortage of places that could still use a good rework or rewrite. Obviously it is best if you can separate your work into patch chunks that are easier to digest, but sometimes that just isn't possible. GIT makes it real easy to go back and refactor your refactorings as you can easily mix and match your patches. -Dan

1 0

[pacman-dev] small pacman code clarification
by Allan McRae 31 Dec '08

31 Dec '08

Hi, I am fixing FS#12263 (problem with umask and db creation), which only requires moving the creation of the db directory for the package before the package archive starts getting extracted. This occurs in the if loop in libalpm/add.c at line 699. However, I am confused about that if loop. The archive is extraction is in a loop: if(!(trans->flags & PM_TRANS_FLAG_DBONLY)) { Why is there a need for that if? As far as I can tell, PM_TRANS_FLAG_DBONLY only happens with a -Rk operation. Is this here in case we want to add the -k flag to Sync operations? Or am I missing its point completely... Cheers, Allan

2 1

[pacman-dev] AUR pacman package fixes bug 9395 - comments?
by Bryan Ischo 31 Dec '08

31 Dec '08

Hi all. I'm sorry I didn't post to this list earlier about this issue; I kind of forgot that there was a pacman-dev list and I added my comments to the bug, not realizing that I should post here too. Anyway, I have created a patch for pacman 3.2.1 which changes the behavior of pacman when it encounters unresolvable dependencies during a sync. The current behavior is that any time a package to be upgraded has a dependency that cannot be resolved, pacman asks if the user would like to ignore the missing dependency and install the package anyway. If the user answers yes, the sync proceeds, resulting in broken dependencies. If the user answers no, then the sync fails immediately. This situation can be quite common if you use the IgnorePkg/IgnoreGroup feature, because any package which depends on an upgrade to an ignored package cannot be sync'd, and thus the user is forced to ignore more and more packages to prevent sync -Su from being impossible to complete. My patch causes pacman to, instead of issuing a force/fail question when an unresolvable dependency is discovered, keep track of which packages that it would upgrade cannot be upgraded for this reason, and issue a single question to the user at the end of the package resolution step, of this form: :: the following packages cannot be upgraded due to unresolvable dependencies: package1, package2, package3, etc. Do you want to skip these packages for this upgrade? [Y/n] If the user answers yes, then the offending packages will simply be removed from the sync operation and the remainder of the sync can occur as normal. If the user answers no, then the sync cannot proceed, and it fails. So this changes the "force/fail" question into a "skip/fail" question, which I think is better. In fact, I can't think of any disadvantages to doing it this way - can you? I don't have a patch to submit for this; instead I've created a patch as part of an AUR submission that produces a new version of pacman with the behavior I have described. I think this is an easier way for users to test this feature. But I would be more than happy to submit a patch to this list directly for this issue if it is the preferred means of getting this change into the official pacman sources. A question I have though, is exactly what should be patched? The NEWS file, configure.ac, etc? Or just the code? I ask because my patch introduces what I believe to be an incompatibility in libalpm with the previous version (because a new "question" has been added, that amounts to an addition to the API), which requires a new libalpm version number, so there are lots of files in the pacman source that need to be patched to effect this. Should I submit a patch to this list that takes pacman-3.2.1 to pacman-3.3.0 (with libalpm 4.0)? Or should I just submit the actual header and source file changes (plus man page and other documentation changes, I am sure) in a patch to this list? Any questions, concerns, comments, or other feedback about this patch would be most appreciated. The bug in question is at: http://bugs.archlinux.org/task/9395 The AUR package that I have created is at: http://aur.archlinux.org/packages.php?ID=11182 Thanks, and best wishes, Bryan

3 3

[pacman-dev] alpm_db_get_pkgcache and alpm_db_get_grpcache rename
by Sebastian Nowicki 30 Dec '08

30 Dec '08

Hi, Recently the alpm_db_get_{pkg,grp}cache() functions were renamed to alpm_db_get_{pkg,grp}cache()[1]. Since this is a public API change, shouldn't the old functions perhaps simply be deprecated and "alias" the new functions, instead of dropping them completely? I'd imagine this would potentially break some front-ends if libalpm gets updated, as was the case with a project of mine. [1]: commit: d05882db9e417244fa580c4697b45333faffcc79

4 5

[pacman-dev] small libdownload patch
by Johannes Krampf 29 Dec '08

29 Dec '08

Hi, I've found a small compatibility problem and static checking a minor buffer overflow in libdownload. Please excuse if this should already be fixed in git. Here's the patch, <stdint.h> is included for uintptr_t and fscanf writes a trailing \0, therefore requiring 1025 bytes in the worst case: diff -Naurd libdownload-1.3/common.c libdownload-1.3-mod/common.c --- libdownload-1.3/common.c Thu May 24 04:30:59 2007 +++ libdownload-1.3-mod/common.c Mon Nov 10 18:51:23 2008 @@ -42,6 +42,7 @@ #include <netdb.h> #include <pwd.h> #include <stdarg.h> +#include <stdint.h> #include <stdlib.h> #include <stdio.h> #include <string.h> @@ -634,7 +635,7 @@ { static char word[1024]; - if (fscanf(f, " %1024s ", word) != 1) + if (fscanf(f, " %1023s ", word) != 1) return (NULL); return (word); } kind regards Johannes

4 7

[pacman-dev] Aqpm, and some help needed
by Dario Freddi 29 Dec '08

29 Dec '08

Hello guys, I'm introducing you Aqpm, a QT/C++ wrapper to libalpm. I did this since Shaman backend was starting to get too much bloated, and since I needed to use Alpm in some other apps (and in Shaman 2 too). What's cool about it: I aim at a strong binary compatibility. This way, even when alpm introduces some binary-incompatible changes, modifying Aqpm internals will be enough to get all applications relying on Aqpm work again without a rebuild. This way we'll also avoid bad situation like Shaman crack this summer. It also wraps around the callback system and implements an easier Queue. It has solid internal threading where needed, and I'm planning to make it support PolicyKit, so that I'll also get rid of the problem of running Shaman with suid and other apps using Alpm running as root (even though Policykit could be directly added to alpm itself, I don't know). Enough talking, it's here: http://github.com/drf/aqpm/tree/master. If you want to test it, I moved Shaman to github and I have a branch there working with aqpm: http://github.com/drf/shaman1/tree/aqpm-work . Sure, it has some regressions, but really small stuff. Please note that Aqpm at the current status supports latest git revision of alpm, so it is not compilable with current stable pacman. I am writing you also to ask you for some help. I have one problem since libalpm 3, namely the download progress callback streaming always 0 in the "total" parameter. I got almost crazy over it, and I still can't find a reason why it does not work. I verified that when Alpm streams the callback, the value is correct, though when it gets received it becomes 0. So I kindly ask you a small review of my code, since I really don't know where to crash my head. It is the last missing bit to aqpm to work 100%, so I'd really appreciate if you could give some help. The relevant code is at: http://github.com/drf/aqpm/tree/master/trunk/libaqpm/Backend.cpp line 797-1010 (Transaction thread, but in that file you'll find every action aqpm does) http://github.com/drf/aqpm/tree/master/trunk/libaqpm/callbacks.cpp Callbacks are here. Also some feedbacks/suggestions would be highly appreciated Thanks and Cheers Dario

3 4

[pacman-dev] [PATCH] makepkg: move BUILDSCRIPT from makepkg.conf
by Allan McRae 27 Dec '08

27 Dec '08

Commit 4b183bf9 moved makepkg.conf sourcing to after the parsing of options, broking the -p option and --help output. The solution is to move BUILDSCRIPT out of makepkg.conf. This patch moves the definition BUILDSCRIPT back to makepkg itself and adds configure option to allow easy changing of this value during build time. Signed-off-by: Allan McRae <allan(a)archlinux.org> --- configure.ac | 9 +++++++++ etc/makepkg.conf.in | 3 +-- scripts/Makefile.am | 2 ++ scripts/makepkg.sh.in | 1 + 4 files changed, 13 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index c7841b8..5f1a153 100644 --- a/configure.ac +++ b/configure.ac @@ -88,6 +88,11 @@ AC_ARG_WITH(db-ext, AS_HELP_STRING([--with-db-ext=ext], [set the file extension used by the database]), [DBEXT=$withval], [DBEXT=.db.tar.gz]) +# Help line for buildscript filename +AC_ARG_WITH(buildscript, + AS_HELP_STRING([--with-buildscript=name], [set the build script name used by makepkg]), + [BUILDSCRIPT=$withval], [BUILDSCRIPT=PKGBUILD]) + # Help line for libdownload/libfetch AC_ARG_ENABLE(internal-download, AS_HELP_STRING([--disable-internal-download], [do not build with libdownload/libfetch support]), @@ -314,6 +319,9 @@ AC_DEFINE_UNQUOTED([SRCEXT], "$SRCEXT", [The file extension used by pacman sourc # Set database file extension AC_SUBST(DBEXT) AC_DEFINE_UNQUOTED([DBEXT], "$DBEXT", [The file extension used by pacman databases]) +# Set makepkg build script name +AC_SUBST(BUILDSCRIPT) +AC_DEFINE_UNQUOTED([BUILDSCRIPT], "$BUILDSCRIPT", [The build script name used by makepkg]) # Configuration files AC_CONFIG_FILES([ @@ -362,6 +370,7 @@ ${PACKAGE_NAME}: package extension : ${PKGEXT} source pkg extension : ${SRCEXT} database extension : ${DBEXT} + build script name : ${BUILDSCRIPT} Compilation options: Run make in doc/ dir : ${wantdoc} diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index 87b115c..9872d5d 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -95,13 +95,12 @@ PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod) #PACKAGER="John Doe <john(a)doe.com>" ######################################################################### -# BUILDSCRIPT/EXTENSION DEFAULTS +# EXTENSION DEFAULTS ######################################################################### # # WARNING: Do NOT modify these variables unless you know what you are # doing. # -BUILDSCRIPT='PKGBUILD' PKGEXT='@PKGEXT@' SRCEXT='@SRCEXT@' diff --git a/scripts/Makefile.am b/scripts/Makefile.am index 014ae87..334b79a 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -37,9 +37,11 @@ edit = sed \ -e 's|@PACKAGE_BUGREPORT[@]|$(PACKAGE_BUGREPORT)|g' \ -e 's|@PACKAGE_NAME[@]|$(PACKAGE_NAME)|g' \ -e 's|@DBEXT[@]|$(DBEXT)|g' \ + -e 's|@BUILDSCRIPT[@]|$(BUILDSCRIPT)|g' \ -e 's|@SIZECMD[@]|$(SIZECMD)|g' \ -e 's|@configure_input[@]|Generated from $@.in; do not edit by hand.|g' + ## All the scripts depend on Makefile so that they are rebuilt when the ## prefix etc. changes. Use chmod -w to prevent people from editing the ## wrong file by accident. diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index a999e0b..eae2c31 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -38,6 +38,7 @@ export COMMAND_MODE='legacy' myver='@PACKAGE_VERSION@' confdir='@sysconfdir@' +BUILDSCRIPT='@BUILDSCRIPT@' startdir="$PWD" srcdir="$startdir/src" pkgdir="$startdir/pkg" -- 1.6.0.6

1 0

[pacman-dev] [PATCH] makepkg: quote all uses of BUILDSCRIPT
by Allan McRae 26 Dec '08

26 Dec '08

Allows specifying alternative build script with spaces in name Signed-off-by: Allan McRae <allan(a)archlinux.org> --- scripts/makepkg.sh.in | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 1dc6415..a999e0b 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1205,9 +1205,9 @@ devel_update() { if [ "$newpkgver" != "" ]; then if [ "$newpkgver" != "$pkgver" ]; then if [ -f "./$BUILDSCRIPT" ]; then - sed -i "s/^pkgver=[^ ]*/pkgver=$newpkgver/" ./$BUILDSCRIPT - sed -i "s/^pkgrel=[^ ]*/pkgrel=1/" ./$BUILDSCRIPT - source $BUILDSCRIPT + sed -i "s/^pkgver=[^ ]*/pkgver=$newpkgver/" "./$BUILDSCRIPT" + sed -i "s/^pkgrel=[^ ]*/pkgrel=1/" "./$BUILDSCRIPT" + source "$BUILDSCRIPT" fi fi fi @@ -1496,7 +1496,7 @@ if [ "$CLEANCACHE" = "1" ]; then fi fi -if [ -z $BUILDSCRIPT ]; then +if [ -z "$BUILDSCRIPT" ]; then error "$(gettext "BUILDSCRIPT is undefined! Ensure you have updated %s.")" "$MAKEPKG_CONF" exit 1 fi @@ -1556,7 +1556,7 @@ if [ ! -f "$BUILDSCRIPT" ]; then BUILDSCRIPT=/dev/stdin fi else - crlftest=$(file $BUILDSCRIPT | grep -F 'CRLF' || true) + crlftest=$(file "$BUILDSCRIPT" | grep -F 'CRLF' || true) if [ "$crlftest" != "" ]; then error "$(gettext "%s contains CRLF characters and cannot be sourced.")" "$BUILDSCRIPT" exit 1 -- 1.6.0.6

2 1

[pacman-dev] makepkg developmental version
by Allan McRae 25 Dec '08

25 Dec '08

Hi all, I have just packaged a version of makepkg from my git working branch for people to test out. This installs alongside the makepkg from pacman so won't affect your regular package building. i686: http://dev.archlinux.org/~allan/makepkg-git-20081223-1-i686.pkg.tar.gz x86_64: http://dev.archlinux.org/~allan/makepkg-git-20081223-1-x86_64.pkg.tar.gz New features of interest: Package splitting support! - multiple packages are specified in the pkgname variable - each package gets its own packaging function where package variables can be overridden - see PKGBUILD-split.proto (in /usr/share/pacman) for "documentation" Added optional package() function in PKGBUILDs - limits fakeroot usage to packaging stages - see "man PKGBUILD-git.conf" for more info Treat info pages like man pages: - automatically compress info pages - info pages are not removed with the !docs options Added purge option: - automatically removes common conflicting or removed files - specify files in PURGE_TARGETS in makepkg.conf - default PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod) Replace getopt usage with internal function: - allows makepkg to run on systems with old getopt version (e.g. Mac OS X) Other smaller changes: - ability to pass PKGBUILD from pipe - package compression type auto-detection - add configuration option to specify man/info page directories - specify alternative configuration file using the --config option - all integrity sums provided are checked - some fixes from the pacman 3.2 maintenance branch: - detect CRLF line endings - when automatically updating an SCM package pkgver, pkgrel is reset to 1 - speed up mercurial package revision checking - improved compatibility with older bash versions Known issues: - The -p flag to specify an alternative name for a PKGBUILD is broken - Repackaging with split-packages definitely does not work - Repackaging when using package() function probably does not work... - Some minor output changes are needed for package splitting Bug reports either to the pacman-dev list or the pacman/makepkg section of the bug tracker. Enjoy, Allan

2 7

[pacman-dev] [PATCH] Remove libdownload support and fix libfetch one.
by Xavier Chantry 25 Dec '08

25 Dec '08

Aaron said to consider libdownload a dead project so libdownload support was removed to more easily fix libfetch one (otherwise many ifdef needed). There was no direct replacement for ferror to detect an error while downloading. So instead, I added a check at the end to see if the file was fully downloaded, which is just a small chunk of code taken from here: http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/net/libfetch/files/fetch.c?only_… Signed-off-by: Xavier Chantry <shiningxc(a)gmail.com> --- configure.ac | 13 +++++-------- lib/libalpm/dload.c | 37 ++++++++++++++++--------------------- lib/libalpm/error.c | 5 +---- 3 files changed, 22 insertions(+), 33 deletions(-) diff --git a/configure.ac b/configure.ac index c7841b8..1b3d702 100644 --- a/configure.ac +++ b/configure.ac @@ -88,9 +88,9 @@ AC_ARG_WITH(db-ext, AS_HELP_STRING([--with-db-ext=ext], [set the file extension used by the database]), [DBEXT=$withval], [DBEXT=.db.tar.gz]) -# Help line for libdownload/libfetch +# Help line for libfetch AC_ARG_ENABLE(internal-download, - AS_HELP_STRING([--disable-internal-download], [do not build with libdownload/libfetch support]), + AS_HELP_STRING([--disable-internal-download], [do not build with libfetch support]), [internaldownload=$enableval], [internaldownload=yes]) # Help line for documentation @@ -131,17 +131,14 @@ AM_GNU_GETTEXT_VERSION(0.13.1) AC_CHECK_LIB([archive], [archive_read_data], , AC_MSG_ERROR([libarchive is needed to compile pacman!])) -# Enable or disable usage of libdownload/libfetch -# - this is a nested check- first see if we need a library, if we do then -# check for libdownload first, then fallback to libfetch, then die +# Enable or disable usage of libfetch AC_MSG_CHECKING(whether to link with download library) if test "x$internaldownload" = "xyes" ; then AC_MSG_RESULT(yes) AC_DEFINE([INTERNAL_DOWNLOAD], , [Use internal download library]) # Check for a download library if it was actually requested - AC_CHECK_LIB([download], [downloadParseURL], , - AC_CHECK_LIB([fetch], [fetchParseURL], , - AC_MSG_ERROR([libdownload or libfetch are needed to compile with internal download support])) ) + AC_CHECK_LIB([fetch], [fetchParseURL], , + AC_MSG_ERROR([libfetch is needed to compile with internal download support]) ) else AC_MSG_RESULT(no) fi diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 9934f30..c0ee28d 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -34,15 +34,7 @@ #include <sys/param.h> /* MAXHOSTNAMELEN */ #endif -#if defined(HAVE_LIBDOWNLOAD) -#include <download.h> -#define fetchFreeURL downloadFreeURL -#define fetchLastErrCode downloadLastErrCode -#define fetchLastErrString downloadLastErrString -#define fetchParseURL downloadParseURL -#define fetchTimeout downloadTimeout -#define fetchXGet downloadXGet -#elif defined(HAVE_LIBFETCH) +#if defined(INTERNAL_DOWNLOAD) #include <fetch.h> #endif @@ -109,7 +101,8 @@ static struct url *url_for_string(const char *url) static int download_internal(const char *url, const char *localpath, time_t mtimeold, time_t *mtimenew) { - FILE *dlf, *localf = NULL; + fetchIO *dlf = NULL; + FILE *localf = NULL; struct url_stat ust; struct stat st; int chk_resume = 0, ret = 0; @@ -208,16 +201,8 @@ static int download_internal(const char *url, const char *localpath, handle->dlcb(filename, 0, ust.size); } - while((nread = fread(buffer, 1, PM_DLBUF_LEN, dlf)) > 0) { + while((nread = fetchIO_read(dlf, buffer, PM_DLBUF_LEN)) > 0) { size_t nwritten = 0; - if(ferror(dlf)) { - pm_errno = PM_ERR_LIBFETCH; - _alpm_log(PM_LOG_ERROR, _("error downloading '%s': %s\n"), - filename, fetchLastErrString); - ret = -1; - goto cleanup; - } - while(nwritten < nread) { nwritten += fwrite(buffer, 1, (nread - nwritten), localf); if(ferror(localf)) { @@ -233,12 +218,22 @@ static int download_internal(const char *url, const char *localpath, handle->dlcb(filename, dl_thisfile, ust.size); } } + + /* did the transfer complete normally? */ + if (ust.size != -1 && dl_thisfile < ust.size) { + pm_errno = PM_ERR_LIBFETCH; + _alpm_log(PM_LOG_ERROR, _("%s appears to be truncated: %jd/%jd bytes\n"), + filename, (intmax_t)dl_thisfile, (intmax_t)ust.size); + ret = -1; + goto cleanup; + } + /* probably safer to close the file descriptors now before renaming the file, * for example to make sure the buffers are flushed. */ fclose(localf); localf = NULL; - fclose(dlf); + fetchIO_close(dlf); dlf = NULL; rename(tempfile, destfile); @@ -254,7 +249,7 @@ cleanup: fclose(localf); } if(dlf != NULL) { - fclose(dlf); + fetchIO_close(dlf); } fetchFreeURL(fileurl); return(ret); diff --git a/lib/libalpm/error.c b/lib/libalpm/error.c index 80a4fb1..df93cb7 100644 --- a/lib/libalpm/error.c +++ b/lib/libalpm/error.c @@ -30,10 +30,7 @@ #include <sys/param.h> /* MAXHOSTNAMELEN */ #endif -#if defined(HAVE_LIBDOWNLOAD) -#include <download.h> /* downloadLastErrString */ -#define fetchLastErrString downloadLastErrString -#elif defined(HAVE_LIBFETCH) +#if defined(INTERNAL_DOWNLOAD) #include <fetch.h> /* fetchLastErrString */ #endif -- 1.6.0.6

1 0