January 2014 - Pacman-dev - lists.archlinux.org

[pacman-dev] Please make the colourised output exactly like that of "pacman-color"
by Xavion 29 Apr '16

29 Apr '16

Hi All, It's great that "pacman" finally has native colour support in v4.1.0. It's just a shame that it's not as good as that of "pacman-color". Without meaning any offence, it falls way short of the mark IMHO. I'm hoping you can just replicate the colourisation of "pacman-color". For your convenience, I've attached the relevant files you'd need. I'm only requesting this because "pacman-color" no longer exists. -- Regards, Xavion.

10 60

[pacman-dev] [PATCH 1/4] makepkg: checkout a revision specified in SVN fragment in download_svn.
by Lukáš Jirkovský 17 Nov '14

17 Nov '14

Previously the sources were dowloaded in HEAD revision in the download_svn(). If a specific revision was requested in fragment, the code was updated to that revision in extract_svn(). However, because SVN is a centralized system, this means that the changed sources has to be downloaded again. By moving the fragment handling to download_svn(), we get the correct revision without having to download it later in extract_svn(). Signed-off-by: Lukáš Jirkovský <l.jirkovsky(a)gmail.com> --- scripts/makepkg.sh.in | 43 +++++++++++++++---------------------------- 1 file changed, 15 insertions(+), 28 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 28e8e7a..aeb231a 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -706,10 +706,23 @@ download_svn() { fi url=${url%%#*} + local ref=HEAD + if [[ -n $fragment ]]; then + case ${fragment%%=*} in + revision) + ref="${fragment##*=}" + ;; + *) + error "$(gettext "Unrecognized reference: %s")" "${fragment}" + plain "$(gettext "Aborting...")" + exit 1 + esac + fi + if [[ ! -d "$dir" ]] || dir_is_empty "$dir" ; then msg2 "$(gettext "Cloning %s %s repo...")" "${repo}" "svn" mkdir -p "$dir/.makepkg" - if ! svn checkout --config-dir "$dir/.makepkg" "$url" "$dir"; then + if ! svn checkout -r ${ref} --config-dir "$dir/.makepkg" "$url" "$dir"; then error "$(gettext "Failure while downloading %s %s repo")" "${repo}" "svn" plain "$(gettext "Aborting...")" exit 1 @@ -717,7 +730,7 @@ download_svn() { elif (( ! HOLDVER )); then msg2 "$(gettext "Updating %s %s repo...")" "${repo}" "svn" cd_safe "$dir" - if ! svn update; then + if ! svn update -r ${ref}; then # only warn on failure to allow offline builds warning "$(gettext "Failure while updating %s %s repo")" "${repo}" "svn" fi @@ -727,11 +740,6 @@ download_svn() { extract_svn() { local netfile=$1 - local fragment=${netfile#*#} - if [[ $fragment = "$netfile" ]]; then - unset fragment - fi - local dir=$(get_filepath "$netfile") [[ -z "$dir" ]] && dir="$SRCDEST/$(get_filename "$netfile")" @@ -742,29 +750,8 @@ extract_svn() { pushd "$srcdir" &>/dev/null rm -rf "${dir##*/}" - local ref - if [[ -n $fragment ]]; then - case ${fragment%%=*} in - revision) - ref="${fragment##*=}" - ;; - *) - error "$(gettext "Unrecognized reference: %s")" "${fragment}" - plain "$(gettext "Aborting...")" - exit 1 - esac - fi - cp -a "$dir" . - if [[ -n ${ref} ]]; then - cd_safe "$(get_filename "$netfile")" - if ! svn update -r ${ref}; then - error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "svn" - plain "$(gettext "Aborting...")" - fi - fi - popd &>/dev/null } -- 1.8.5.1

7 28

[pacman-dev] [PATCH v2] Extend database upgrade script to handle alpm db version 9
by Allan McRae 20 Aug '14

20 Aug '14

Signed-off-by: Allan McRae <allan(a)archlinux.org> --- v2: Use awk to do symlink to actual directory replacement As an example of what changes get made: diff -Naur local.tmp/glibc-2.18-4/files local/glibc-2.18-4/files --- local.tmp/glibc-2.18-4/files 2013-09-18 18:45:35.747068765 +1000 +++ local/glibc-2.18-4/files 2013-09-18 18:51:29.633643945 +1000 @@ -4,8 +4,8 @@ etc/locale.gen etc/nscd.conf etc/rpc -lib/ -lib/libc.a +usr/lib/ +usr/lib/libc.a usr/ usr/bin/ usr/bin/catchsegv Note that now usr/lib/ is present twice in the file (may not always be the case) and these entries are out of alphabetical order. Both these are cosmetic to pacman. scripts/pacman-db-upgrade.sh.in | 59 ++++++++++++++++++++++++++++++++++------- 1 file changed, 49 insertions(+), 10 deletions(-) diff --git a/scripts/pacman-db-upgrade.sh.in b/scripts/pacman-db-upgrade.sh.in index a1630c5..d6f4169 100644 --- a/scripts/pacman-db-upgrade.sh.in +++ b/scripts/pacman-db-upgrade.sh.in @@ -109,18 +109,57 @@ fi # do not let pacman run while we do this touch "$lockfile" -# pacman-3.4 to 3.5 upgrade - merge depends into desc -if [[ $(find "$dbroot"/local -name depends) ]]; then - msg "$(gettext "Pre-3.5 database format detected - upgrading...")" - for i in "$dbroot"/local/*; do - if [[ -f "$i"/depends ]]; then - cat "$i"/depends >> "$i"/desc - rm "$i"/depends - fi - done - msg "$(gettext "Done.")" +if [[ -f "${dbroot}"/local/.alpm_db_version ]]; then + db_version=$(cat "${dbroot}"/local/.alpm_db_version) fi +if [[ -z "$db_version" ]]; then + # pacman-3.4 to 3.5 upgrade - merge depends into desc + if [[ $(find "$dbroot"/local -name depends) ]]; then + msg "$(gettext "Pre-3.5 database format detected - upgrading...")" + for i in "$dbroot"/local/*; do + if [[ -f "$i"/depends ]]; then + cat "$i"/depends >> "$i"/desc + rm "$i"/depends + fi + done + msg "$(gettext "Done.")" + fi + + # pacman 4.1 to 4.2 upgrade - remove directory symlink support + dirlist=() + + unset GREP_OPTIONS + while IFS= read -r dir; do + dirlist+=("/${dir%/}") + done < <(grep -h '/$' "$dbroot"/local/*/files | sort -u) + + mapfile -t dirlist < <(find "${dirlist[@]}" -maxdepth 0 -type l) + + if [[ ${#dirlist[@]} != 0 ]]; then + msg "$(gettext "Pre-4.2 database format detected - upgrading...")" + for dir in "${dirlist[@]}"; do + realdir="$(cd $dir; pwd -P)" + for f in "$dbroot"/local/*/files; do + awk -v "dir=${dir#/}/" -v "realdir=${realdir#/}/" ' + BEGIN { + i = length(dir) + 1 + } + { + if (index($0, dir) == 1) { + printf("%s%s\n", realdir, substr($0, i)) + } else { + print + } + }' $f > $f.tmp + mv $f.tmp $f + done + done + fi +fi + +echo "9" > "$dbroot"/local/.alpm_db_version + # remove the lock file rm -f "$lockfile" -- 1.8.4

2 9

[pacman-dev] [PATCH] handle time changes during progress display
by Andrew Gregory 28 Jun '14

28 Jun '14

Previously, if the system time was adjusted backwards during a progress display, get_update_timediff return negative values prevent the progress bar from updating. Instead, on negative values, the saved time is reset. Additionally, the download callback ignores the amount downloaded to avoid skewing the rate. Fixes FS#36983 Signed-off-by: Andrew Gregory <andrew.gregory.8(a)gmail.com> --- src/pacman/callback.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/src/pacman/callback.c b/src/pacman/callback.c index a181fa5..0ffb2ef 100644 --- a/src/pacman/callback.c +++ b/src/pacman/callback.c @@ -73,8 +73,8 @@ static long get_update_timediff(int first_call) retval = (diff_sec * 1000) + (diff_usec / 1000); - /* do not update last_time if interval was too short */ - if(retval >= UPDATE_SPEED_MS) { + /* update last_time if it is in the future or interval was long enough */ + if(retval < 0 || retval >= UPDATE_SPEED_MS) { last_time = this_time; } } @@ -554,7 +554,7 @@ void cb_dl_progress(const char *filename, off_t file_xfered, off_t file_total) { static double rate_last; static off_t xfered_last; - static struct timeval initial_time; + static long elapsed_time = 0; int infolen; int filenamelen; char *fname, *p; @@ -616,25 +616,23 @@ void cb_dl_progress(const char *filename, off_t file_xfered, off_t file_total) /* set default starting values, ensure we only call this once * if TotalDownload is enabled */ if(!totaldownload || (totaldownload && list_xfered == 0)) { - gettimeofday(&initial_time, NULL); + elapsed_time = 0; xfered_last = (off_t)0; rate_last = 0.0; get_update_timediff(1); } } else if(file_xfered == file_total) { /* compute final values */ - struct timeval current_time; - time_t diff_sec; - suseconds_t diff_usec; + timediff = get_update_timediff(0); - gettimeofday(&current_time, NULL); - diff_sec = current_time.tv_sec - initial_time.tv_sec; - diff_usec = current_time.tv_usec - initial_time.tv_usec; - timediff = (diff_sec * 1000) + (diff_usec / 1000); if(timediff > 0) { - rate = (double)xfered / (timediff / 1000.0); + elapsed_time += timediff; + } + + if(elapsed_time > 0) { + rate = (double)xfered / (elapsed_time / 1000.0); /* round elapsed time (in ms) to the nearest second */ - eta_s = (unsigned int)(timediff + 500) / 1000; + eta_s = (unsigned int)(elapsed_time + 500) / 1000; } else { eta_s = 0; } @@ -642,10 +640,17 @@ void cb_dl_progress(const char *filename, off_t file_xfered, off_t file_total) /* compute current average values */ timediff = get_update_timediff(0); - if(timediff < UPDATE_SPEED_MS) { + if(timediff < 0) { + /* we lost a chunk of time due to a clock change, including the amount + * xfered during that period in the next update would skew the rate */ + xfered_last = xfered; + return; + } else if(timediff < UPDATE_SPEED_MS) { /* return if the calling interval was too short */ return; } + + elapsed_time += timediff; rate = (double)(xfered - xfered_last) / (timediff / 1000.0); /* average rate to reduce jumpiness */ rate = (rate + 2 * rate_last) / 3; -- 1.8.4.2

4 6

[pacman-dev] [PATCH 0/8] Events
by Olivier Brunel 16 Feb '14

16 Feb '14

The first three patches are just small changes, then goes the update to the different structs instead of the two void* arguments, and making the log stuff into an event as well. Then adding events about creation of the pac* files as originally discussed; And I've also included a few new events, as mentioned before, that could be useful to frontends, even though pacman just ignores them (for now). A couple of other things : - Andrew mentioned FS#37711, which could be addressed now that those (creation of pacnew/pacorig files) are fully handled by pacman, but for now I've simply let those as warnings, much like they were before. Since the color issue has been fixed (i.e. such messages didn't have colors applied when they were delayed due to a progress operation) I'm not sure if anything else is really needed anyways. - I don't think addressing FS#36504 was right. This was about moving the log messages about a package being installed/upgraded/etc before the scriptlet messages, and not after as they were. I think this might have been a mistake: I guess it all depends on when you consider the operation to be completed, once pacman's extraction reached 100%, or after the scriptlet have run. I feel the later is right, since they are part of the process of installing/upgrading/etc a package; Of course, you may disagree. Either way, the current situation is wrong, because right now ALPM does : 1. add log message about the operation being done 2. run scriptlet 3. signal frontend (via event) that operation is done IOW if a frontend was to actually use the event to present its output to the user, it wouldn't match the order in the log, which is wrong. This was done because pacman ignores that event, only showing the progress of the extraction, and thus people feel that it goes 1. install done; then 2. scriptlet. In fact, pacman doesn't really ignore the event, and uses it to show (new) optdep. Should this be done before or after the scriptlet? One could also ask, when using --noprogressbar pacman doesn't say anything at the end of the operation (save for optdep), imagine it would: should it be before or after the scriptlet? As I said, I think the logging should be moved after the scriptlet have run, as this is when the operation is really done. If you were to disagree though, then the event should be moved before the scriptlet run, to keep consistency between the frontend and the log (which is broken ATM), but this just feels wrong to me, since then a frontend would consider a package upgrade/etc done, when in fact the scriptlet have yet to run (and produce new output relating to the operation supposedly done). Olivier Brunel (8): pacman: Make sure delayed warnings/errors are shown the same Rename ALPM_EVENT_OPTDEP_REQUIRED to _OPTDEP_REMOVED alpm: Rename a variable for clarity Update the event callback Remove log_cb, add ALPM_EVENT_LOG instead Add events on pacnew/pacsave/pacorig file creation Add events ALPM_EVENT_RETRIEVE_{DONE,FAILED} Add events _PKGDOWNLOAD_{START,DONE,FAILED} lib/libalpm/add.c | 86 +++++++++++------ lib/libalpm/alpm.h | 248 ++++++++++++++++++++++++++++++++---------------- lib/libalpm/be_sync.c | 6 +- lib/libalpm/handle.c | 13 --- lib/libalpm/handle.h | 5 +- lib/libalpm/log.c | 14 ++- lib/libalpm/remove.c | 32 +++++-- lib/libalpm/sync.c | 94 +++++++++++++----- lib/libalpm/util.c | 6 +- src/pacman/callback.c | 181 ++++++++++++++++++++++++----------- src/pacman/callback.h | 6 +- src/pacman/conf.c | 1 - src/pacman/util.c | 19 +++- src/pacman/util.h | 1 + src/util/cleanupdelta.c | 14 +-- src/util/testdb.c | 14 +-- src/util/testpkg.c | 14 +-- 17 files changed, 510 insertions(+), 244 deletions(-) -- 1.8.4.2

4 35

[pacman-dev] [PATCH 1/5] Check if a file is in the package's file list before extracting
by Florian Pritz 14 Feb '14

14 Feb '14

Signed-off-by: Florian Pritz <bluewind(a)xinu.at> --- lib/libalpm/add.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/libalpm/add.c b/lib/libalpm/add.c index 5f1a953..7930117 100644 --- a/lib/libalpm/add.c +++ b/lib/libalpm/add.c @@ -178,6 +178,11 @@ static int extract_single_file(alpm_handle_t *handle, struct archive *archive, archive_read_data_skip(archive); return 0; } else { + if (!alpm_filelist_contains(&newpkg->files, entryname)) { + _alpm_log(handle, ALPM_LOG_WARNING, _("file not found in file list for package %s. skipping extraction of %s\n"), + newpkg->name, entryname); + return 0; + } /* build the new entryname relative to handle->root */ snprintf(filename, PATH_MAX, "%s%s", handle->root, entryname); } -- 1.8.5.3

5 36

[pacman-dev] [PATCH 1/8] avoid double freeing unresolvable packages
by Andrew Gregory 07 Feb '14

07 Feb '14

If an unresolvable package was loaded from a file it would be free'd again when freeing trans->add. Signed-off-by: Andrew Gregory <andrew.gregory.8(a)gmail.com> --- lib/libalpm/trans.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/lib/libalpm/trans.c b/lib/libalpm/trans.c index ada8100..4930e94 100644 --- a/lib/libalpm/trans.c +++ b/lib/libalpm/trans.c @@ -246,12 +246,19 @@ int SYMEXPORT alpm_trans_release(alpm_handle_t *handle) void _alpm_trans_free(alpm_trans_t *trans) { + alpm_list_t *i; + if(trans == NULL) { return; } - alpm_list_free_inner(trans->unresolvable, - (alpm_list_fn_free)_alpm_pkg_free_trans); + /* only free packages not in trans->add to avoid + * double free'ing packages loaded from files. */ + for(i = trans->unresolvable; i; i = i->next) { + if(!alpm_list_find_ptr(trans->add, i->data)) { + _alpm_pkg_free_trans(i->data); + } + } alpm_list_free(trans->unresolvable); alpm_list_free_inner(trans->add, (alpm_list_fn_free)_alpm_pkg_free_trans); alpm_list_free(trans->add); -- 1.8.5.2

2 13

[pacman-dev] [PATCH] Put explicitly installed packages in bold during update/install/removal
by Guillaume Bouchard 05 Feb '14

05 Feb '14

Hello. A really small change in pacman which, I hope, improves the feedback about explicit packages on install/removal/update. Have a nice day. Output of git format-patch origin: >From bd5c3aa7826c3c073cca691f0e30b5363487cd6f Mon Sep 17 00:00:00 2001 From: Guillaume Bouchard <guillaum.bouchard(a)gmail.com> Date: Fri, 24 Jan 2014 12:20:16 +0100 Subject: [PATCH] Put explicitly installed packages in bold during update/install/removal In extended table view, packages which are explicitly installed are displayed as CELL_TITLE (i.e. in bold). This helps understanding why packages are updated. This helps the package managment workflow. During update, we can quickly have a look at why packages are updated and easilly track and remove the explicit packages which are not longer required. During remove, it shows all the explicit packages which are also removed by a cascade removal. During install, it provides a feedback on how your action will affect the database. This only affects "VerbosePkgLists" output. Patching the un-verbose output appears more difficult and I think this information is only interesting during verbose output. Signed-off-by: Guillaume Bouchard <guillaum.bouchard(a)gmail.com> --- src/pacman/util.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/pacman/util.c b/src/pacman/util.c index 58b0cec..a3ee5ee 100644 --- a/src/pacman/util.c +++ b/src/pacman/util.c @@ -808,7 +808,12 @@ static alpm_list_t *create_verbose_row(pm_target_t *target) } else { pm_asprintf(&str, "%s", alpm_pkg_get_name(target->remove)); } - add_table_cell(&ret, str, CELL_NORMAL); + if(alpm_pkg_get_reason(target->remove ? target->remove : target->install) == ALPM_PKG_REASON_EXPLICIT) { + /* put emphasis on the package name if it is an explicitly installed package */ + add_table_cell(&ret, str, CELL_TITLE); + } else { + add_table_cell(&ret, str, CELL_NORMAL); + } /* old and new versions */ pm_asprintf(&str, "%s", -- 1.8.5.2 -- Guillaume

7 19

[pacman-dev] [PATCH 1/5] valgrind.supp: add fakeroot/fakechroot errors
by Andrew Gregory 04 Feb '14

04 Feb '14

Signed-off-by: Andrew Gregory <andrew.gregory.8(a)gmail.com> --- Patch submitted upstream for the __lxstat error. valgrind.supp | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/valgrind.supp b/valgrind.supp index 55effa9..ac54b74 100644 --- a/valgrind.supp +++ b/valgrind.supp @@ -109,3 +109,43 @@ ... fun:regexec } +{ + fakeroot-msgsnd + Memcheck:Param + msgsnd(msgp->mtext) + fun:__msgsnd_nocancel +} +{ + fakechroot-add-to-environ + Memcheck:Leak + fun:malloc + fun:__add_to_environ +} +{ + fakechroot-static-fakechroot-init-malloc + Memcheck:Leak + fun:malloc + ... + fun:fakechroot_init +} +{ + fakechroot-static-fakechroot-init-calloc + Memcheck:Leak + fun:calloc + ... + fun:fakechroot_init +} +{ + fakechroot-static-bindtextdomain + Memcheck:Leak + match-leak-kinds: reachable + fun:malloc + obj:/usr/lib/libfakeroot/fakechroot/libfakechroot.so + obj:/usr/lib/libfakeroot/fakechroot/libfakechroot.so + fun:bindtextdomain +} +{ + fakechroot-uninitialized-var-lxstat + Memcheck:Cond + fun:__lxstat +} -- 1.8.5.2

3 10

[pacman-dev] [PATCH] checkupdates: Add package download mode with "-d"
by Chris Down 02 Feb '14

02 Feb '14

This new mode, which must be run as root, allows for the precaching of packages which can then subsequently be updated by calling "pacman -Syu" as normal. This can then be integrated with a cron script (for example, checkupdates-cron[0]) for automated, safe downloading of packages alongside getting the list of packages to update, or run manually on the command line to download new packages, but avoid updating the main package database. As a side effect of adding the -d option, argument handling (which was previously not present) is now done using "getopts". This seems like a reasonable way to do it instead of just hacking in a check for "-d" as $1 now, but I'm not particularly averse to either. "getopts" would just allow for better future flexibility in this area. It is possible to do this as another script, say, downloadupdates, but we would encounter pretty heavy code duplication with "checkupdates", so it seems more sensible to do it here. Due to the use of --noconfirm, this has some limitations when we encounter more complex upgrade scenarios, for example when a new package conflict occurs, an upgrade path cannot be correctly/automatically determined, a package was replaced, etc. This is not a major problem -- in the worst case, we just don't get the packages that we wanted downloaded and an error message is displayed to the user. The choice to hide both STDOUT and STDERR of "pacman -Syuw" is potentially up for debate. It makes it more difficult to determine where the problem was if the command fails -- we still display a message saying that that was the case, and set the exit code accordingly, but the actual cause needs to be determined by the user elsewhere. For this reason, "-p" exists to force the package download to output as normal. It seems desirable that "-p" also affects the silence of the database update, but this is also up for discussion if people feel it should not be within the remit of the (( silent )) checks. [0]: https://github.com/cdown/checkupdates-cron --- contrib/checkupdates.sh.in | 41 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 38 insertions(+), 3 deletions(-) diff --git a/contrib/checkupdates.sh.in b/contrib/checkupdates.sh.in index 413b78a..02acb67 100644 --- a/contrib/checkupdates.sh.in +++ b/contrib/checkupdates.sh.in @@ -21,16 +21,43 @@ declare -r myname='checkupdates' declare -r myver='@PACKAGE_VERSION@' -if (( $# > 0 )); then +download=0 +silent=1 + +usage() { echo "${myname} (pacman) v${myver}" echo echo "Safely print a list of pending updates" echo echo "Usage: ${myname}" echo + echo "Options:" + echo " -d Download upgradeable packages to pacman package cache." + echo " -p Print pacman output from the database upgrade and package download." + echo echo 'Note: Export the "CHECKUPDATES_DB" variable to change the path of the temporary database.' exit 0 -fi +} + +run_silent() ( + (( silent )) && exec &>/dev/null + "$@" +) + +while getopts ':dp' opt; do + case $opt in + d) + if (( UID )); then + echo "Download mode can only be run as root." >&2 + exit 2 + fi + + download=1 + ;; + p) silent=0 ;; + \?) usage ;; + esac +done if [[ -z $CHECKUPDATES_DB ]]; then CHECKUPDATES_DB="${TMPDIR:-/tmp}/checkup-db-${USER}/" @@ -43,9 +70,17 @@ eval $(awk -F' *= *' '$1 ~ /DBPath/ { print $1 "=" $2 }' @sysconfdir@/pacman.con mkdir -p "$CHECKUPDATES_DB" ln -s "${DBPath}/local" "$CHECKUPDATES_DB" &> /dev/null -fakeroot pacman -Sy --dbpath "$CHECKUPDATES_DB" --logfile /dev/null &> /dev/null +run_silent fakeroot pacman -Sy --dbpath "$CHECKUPDATES_DB" --logfile /dev/null pacman -Qqu --dbpath "$CHECKUPDATES_DB" 2> /dev/null +if (( download )); then + if ! run_silent pacman -Suw --noconfirm --dbpath "$CHECKUPDATES_DB"; then + ret=$? + (( silent )) && echo "Failed to download packages." >&2 + exit "$ret" + fi +fi + exit 0 # vim: set ts=2 sw=2 noet: -- 1.8.5.3

2 1