- Pacman-dev - lists.archlinux.org

[pacman-dev] Pacman output issues
by Dan McGee 08 Oct '07

08 Oct '07

A few questions to pose to the list before I make changes that need to be made since I busted some things up with the last few updates of pacman output trying to make it simpler. 1. What information should go to stdout, and what information should go to stderr? I really should look at how it is done in the current 3.0 branch so I can get it right going forward in 3.1, but I want to make sure that what was done in 3.0 is right according to the list. I would think something like this: * non-essential info (progress meters and the like): stdout * list of packages to be upgraded, Y/n questions: this probably should go to stdout, but has gone to stderr in the past so that redirection of stdout works and users still see questions, and pacman doesn't seem to hang without reason. Thoughts? * "package is newer than...": stdout * errors/warnings: stderr 2. I introduced the pm_printf family of functions a while back but didn't switch much of the pacman output over to it. It mainly deals with not having to duplicate the use of "warning:" and "error:", etc in the code. Any reason not to use this as often as possible in the pacman code? I plan on doing this; I just want to first make sure I'm not making a stupid mistake. Obviously not everything will get switched (usage and help come to mind as being plain printf candidates), but I think just about everything else should go. My only thought holding me back is the fact that the error level codes are in libalpm and not the frontend, which may be a non-issue. 3. I broke the newline stuff. If a progress bar is going and some output needs to be placed on the screen, it doesn't end up on the next line. I'd really like a clean way to fix this- the old global variable way seemed very weird to me, or at least it was too complex. I have one idea in my head, but haven't sat down and tried it yet. -Dan

2 1

[pacman-dev] [PATCH] Add deltas to repository database.
by Nathan Jones 08 Oct '07

08 Oct '07

This patch adds delta information to the repository database along with the other information about a package. This would be useful if two or more package versions have been built since a user upgrades. I don't think it would be too difficult to create an algorithm to decide which delta files, if any, would be downloaded and applied. Any comments? Sample output: $ ls -l *.pkg.tar.gz *.delta -rw-r--r-- 1 nathanj users 1045940 2007-05-12 21:03 pan-devel-0.129-1-i686.pkg.tar.gz -rw-r--r-- 1 nathanj users 583339 2007-10-05 18:38 pan-devel-0.129-1_to_0.130-1-i686.delta -rw-r--r-- 1 nathanj users 1044271 2007-05-19 13:17 pan-devel-0.130-1-i686.pkg.tar.gz -rw-r--r-- 1 nathanj users 425727 2007-10-05 18:38 pan-devel-0.130-1_to_0.131-1-i686.delta -rw-r--r-- 1 nathanj users 1055063 2007-05-26 18:40 pan-devel-0.131-1-i686.pkg.tar.gz -rw-r--r-- 1 nathanj users 680644 2007-10-05 18:38 pan-devel-0.131-1_to_0.132-1-i686.delta -rw-r--r-- 1 nathanj users 1025859 2007-08-15 22:03 pan-devel-0.132-1-i686.pkg.tar.gz -rw-r--r-- 1 nathanj users 1792 2007-10-05 18:38 pan-devel-0.132-1_to_0.132-2-i686.delta -rw-r--r-- 1 nathanj users 1025826 2007-08-01 17:04 pan-devel-0.132-2-i686.pkg.tar.gz $ ~/code/pacman/scripts/repo-add repo.db.tar.gz pan-devel-0.132-2-i686.pkg.tar.gz *.delta ==> Extracting database to a temporary location... ==> Adding package 'pan-devel-0.132-2-i686.pkg.tar.gz' -> Removing existing package 'pan-devel-0.132-2'... -> Creating 'desc' db entry... -> Computing md5 checksums... -> Creating 'depends' db entry... ==> Adding delta 'pan-devel-0.129-1_to_0.130-1-i686.delta' ==> Adding delta 'pan-devel-0.130-1_to_0.131-1-i686.delta' ==> Adding delta 'pan-devel-0.131-1_to_0.132-1-i686.delta' ==> Adding delta 'pan-devel-0.132-1_to_0.132-2-i686.delta' ==> Creating updated database file /home/nathanj/pkgs/pan-devel/repo.db.tar.gz $ tar zxOf repo.db.tar.gz pan-devel-0.132-2/deltas %DELTAS% 0.129-1 0.130-1 583339 0.130-1 0.131-1 425727 0.131-1 0.132-1 680644 0.132-1 0.132-2 1792 Signed-off-by: Nathan Jones <nathanj(a)insightbb.com> --- scripts/repo-add.sh.in | 64 +++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 63 insertions(+), 1 deletions(-) diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index cb741d7..4a7949e 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -97,6 +97,61 @@ write_list_entry() { fi } +# write a delta entry to the pacman database +# arg1 - path to delta +db_write_delta() +{ + # blank out all variables and set deltafile + local deltafile=$(readlink -f "$1") + local pkgname fromver tover arch csize + + pkgname="$(echo "$1" | sed -e 's/$.*$-$.*-.*$_to_$.*-.*$-$.*$.delta/\1/')" + fromver="$(echo "$1" | sed -e 's/$.*$-$.*-.*$_to_$.*-.*$-$.*$.delta/\2/')" + tover="$(echo "$1" | sed -e 's/$.*$-$.*-.*$_to_$.*-.*$-$.*$.delta/\3/')" + arch="$(echo "$1" | sed -e 's/$.*$-$.*-.*$_to_$.*-.*$-$.*$.delta/\4/')" + + # get size of delta + csize=$(du -b -L "$deltafile" | cut -f 1) + + pushd "$gstmpdir" 2>&1 >/dev/null + + # ensure $pkgname and $pkgver variables were found + if [ -z "$pkgname" -o -z "$fromver" ]; then + error "$(gettext "Invalid delta file '%s'.")" "$deltafile" + popd 2>&1 >/dev/null + return 1 + fi + + # cd into the package's directory + local existing + local gotpackage=f + for existing in *; do + if [ "${existing%-*-*}" = "$pkgname" ]; then + cd "$existing" + gotpackage=t + break + fi + done + + # exit out if the package's directory does not exist + if [ "$gotpackage" == "f" ]; then + error "$(gettext "Cannot find directory for '%s'.")" "$pkgname" + popd 2>&1 >/dev/null + return 1 + fi + + # create deltas file if it doesn't exist + if [ ! -f "deltas" ]; then + echo -e "%DELTAS%" >>deltas + fi + + # add the entry for this delta file + echo -e "$fromver $tover $csize" >>deltas + + popd 2>&1 >/dev/null +} # end db_write_delta + + # write an entry to the pacman database # arg1 - path to package db_write_entry() @@ -242,7 +297,14 @@ for arg in "$@"; do fi else if [ -f "$arg" ]; then - if ! bsdtar -tf "$arg" .PKGINFO 2>&1 >/dev/null; then + # see if extension is .delta ... there's gotta be a better way + if [ "${arg%.delta}" != "$arg" ]; then + msg "$(gettext "Adding delta '%s'")" "$arg" + + if db_write_delta "$arg"; then + success=1 + fi + elif ! bsdtar -tf "$arg" .PKGINFO 2>&1 >/dev/null; then error "$(gettext "'%s' is not a package file, skipping")" "$arg" else msg "$(gettext "Adding package '%s'")" "$arg" -- 1.5.3.4

3 3

Re: [pacman-dev] [arch-dev-public] GIT Daemon up on projects.archlinux.org
by Xavier 05 Oct '07

05 Oct '07

On Wed, Oct 03, 2007 at 10:22:57PM -0500, Dan McGee wrote: > For all out there that may not be aware, we now have a subdomain at > http://projects.archlinux.org where we host some of our internal Arch > projects. We have been moving a lot of our projects from other version > control systems to GIT as of late, and currently have quite a bunch of > these projects hosted, as you can see here: > http://projects.archlinux.org/git/ > > For those interested in helping out with these projects or just > following along with the code, we now have a git daemon up and > running. All of these projects can still be fetched over http if need > be, but using the GIT native protocol is much more efficient. > > The following two examples should get you started on cloning one of > our projects: > (Over HTTP) git clone http://projects.archlinux.org/git/namcap.git namcap > (Over GIT) git clone git://projects.archlinux.org/pacman.git pacman > > Note the slight difference in URLs- for cloning over GIT, no /git > directory is used. Happy git-cloning, and we'd really like to see some > community participation with these projects if you have something to > contribute! Patches always welcome on the mailing lists. > > -Dan > Is this why I can't get the last update on pacman git repo (using http) ? I remember you warned me several times to enable the post-update hook when using http. "# An example hook script to prepare a packed repository for use over dumb transports." Could it be what is missing there? I switched to git protocol, and it worked fine. And I have no reason for using http over git, so that's fine, but I was just curious :)

2 1

[pacman-dev] -Qu vs -Su
by Xavier 05 Oct '07

05 Oct '07

Since the discussion on http://bugs.archlinux.org/task/7884 is confusing me, I thought I would bring it here. The issue reported by Dan is the following : The -Qu output is different than -Su one, because -Su resolves dependencies, while -Qu does not. My first question : is that really an issue? The description of -Qu says : -u, --upgrades list all packages that can be upgraded IMO, a new dependency does not belong to the list of packages that can be upgraded. So it isn't necessarily a bug if it shows up in -Su, but not in -Qu. Also, note that -Qu can't match exactly -Su, because -Su is interactive : Replace %s with %s/%s? [Y/n] %s conflicts with %s. Remove %s? [Y/n] But if you answer no to the conflicts question, pacman just fails and stops there. So not very interesting. And the replace question is generally answered by yes. So answering yes to both questions should give a good picture of what will happen.

6 20

[pacman-dev] [GIT] pacman branch, master now at v3.0.0-434-g48bf088
by Aaron Griffin 05 Oct '07

05 Oct '07

Hello, This is an automated email from the git hooks/update script, it was generated because a ref change was pushed to the repository. Updating branch, master, via 48bf088e65403049dc430fadf360355a8d5dc07d (commit) via b86479cf9b584c62dbda967902cee2a81678c9ef (commit) via 4942d216323cbe2f099c43893cc95368a3cd1197 (commit) via 50bb16e0151e31479cb2602c9749ffe18f7a3592 (commit) via cb3c3a920d976942fbc72b2d9dbff5320ca9415c (commit) via f2f4ada1d6cfee83674684b948e5698049479f95 (commit) via 47622eef4dd8fd86a0aa0e3ebdb7b33f7c9d6804 (commit) from 219808714f94788a66a430786c552f60e95b1a01 (commit) - Log ----------------------------------------------------------------- commit 48bf088e65403049dc430fadf360355a8d5dc07d Author: Aaron Griffin <aaronmgriffin(a)gmail.com> Date: Thu Sep 27 23:38:51 2007 -0500 Use ANSI 'foreground' color instead of hardcoding 'white' Signed-off-by: Aaron Griffin <aaronmgriffin(a)gmail.com> commit b86479cf9b584c62dbda967902cee2a81678c9ef Author: Aaron Griffin <aaronmgriffin(a)gmail.com> Date: Thu Sep 27 23:04:00 2007 -0500 Allow --refresh specified for most -S options Included options are -g, -i, -s, and -l All of these will sync the DB if -y is specified and permissions are appropriate. Signed-off-by: Aaron Griffin <aaronmgriffin(a)gmail.com> commit 4942d216323cbe2f099c43893cc95368a3cd1197 Author: Aaron Griffin <aaronmgriffin(a)gmail.com> Date: Thu Sep 27 23:03:35 2007 -0500 Break out transaction test to a separate function Added needs_transaction, putting out "hey do we need root?" tests in one place. Signed-off-by: Aaron Griffin <aaronmgriffin(a)gmail.com> commit 50bb16e0151e31479cb2602c9749ffe18f7a3592 Author: Aaron Griffin <aaronmgriffin(a)gmail.com> Date: Thu Sep 27 21:38:47 2007 -0500 OMG a space! Yeah, I added a space. It deserves its own commit. Signed-off-by: Aaron Griffin <aaronmgriffin(a)gmail.com> commit cb3c3a920d976942fbc72b2d9dbff5320ca9415c Author: Aaron Griffin <aaronmgriffin(a)gmail.com> Date: Thu Sep 27 21:18:06 2007 -0500 Minor "cannot remove" message cleanup The strings differed in quotes only, this way we have only one i18n string for translation Signed-off-by: Aaron Griffin <aaronmgriffin(a)gmail.com> commit f2f4ada1d6cfee83674684b948e5698049479f95 Author: Aaron Griffin <aaronmgriffin(a)gmail.com> Date: Wed Sep 19 00:25:38 2007 -0500 Remove buildtype package accessor prototype This was removed a while ago, but the prototype was still hanging out. Signed-off-by: Aaron Griffin <aaronmgriffin(a)gmail.com> commit 47622eef4dd8fd86a0aa0e3ebdb7b33f7c9d6804 Author: Aaron Griffin <aaronmgriffin(a)gmail.com> Date: Wed Sep 19 00:21:56 2007 -0500 Support for localized times in metadata Packages and DBs now support using the UNIX epoch (seconds since Jan 1, 1970) for use in builddate and installdate. This will only affect newly built packages. Old existing packages with the text format are still supported, but this is deprecated. In the case of removal of text time support, this code will fail gracefully, returning the start of the epoch for broken packages. Signed-off-by: Aaron Griffin <aaronmgriffin(a)gmail.com> ----------------------------------------------------------------------- Diffstat: lib/libalpm/add.c | 5 +--- lib/libalpm/alpm.h | 5 +-- lib/libalpm/be_files.c | 38 ++++++++++++++++++++++++++++------- lib/libalpm/package.c | 16 ++++++++------ lib/libalpm/package.h | 4 +- lib/libalpm/remove.c | 3 +- scripts/makepkg.sh.in | 12 +++++----- src/pacman/package.c | 7 +++-- src/pacman/pacman.c | 20 +++--------------- src/pacman/sync.c | 51 +++++++++++++++++++++++++---------------------- src/pacman/util.c | 19 +++++++++++++++++ src/pacman/util.h | 1 + 12 files changed, 107 insertions(+), 74 deletions(-) hooks/update --- Git Source Code Management System hooks/update refs/heads/master \ 219808714f94788a66a430786c552f60e95b1a01 \ 48bf088e65403049dc430fadf360355a8d5dc07d

3 4

[pacman-dev] Fix -Sy and -Sp operations.
by Xavier 04 Oct '07

04 Oct '07

When the -Sy operation failed, the lock wasn't removed because alpm_trans_release wasn't called. This is fixed now. Also, after my last change in commit 52e7e6d74733f7a431376a9e528c4fe4d3732068 , Sp didn't do anything anymore. That's because needs_transaction returned false for -Sp, so the sync transaction wasn't run. However, the current implementation of -Sp requires a sync transaction. Also, since a transaction creates the lock file at the beginning, and releases it at the end, this mean that -Sp requires root access anyway.. I think I understand now why Aaron found that the current -Sp implementation is hackish :) Signed-off-by: Chantry Xavier <shiningxc(a)gmail.com> --- src/pacman/sync.c | 3 ++- src/pacman/util.c | 3 +-- 2 files changed, 3 insertions(+), 3 deletions(-)

2 1

[pacman-dev] [PATCH] makepkg: Fix typo to make xdelta creation work.
by Nathan Jones 04 Oct '07

04 Oct '07

Signed-off-by: Nathan Jones <nathanj(a)insightbb.com> --- scripts/makepkg.sh.in | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index e4b96f0..4a68c84 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -877,7 +877,7 @@ create_xdelta() { local old_file old_version for old_file in $(ls {"$cache_dir","$PKGDEST"}/${pkgname}-*-*{,-$CARCH}$PKGEXT 2>/dev/null); do - bsdtar -xOf "$oldfile" .PKGINFO > "$pkginfo" || continue + bsdtar -xOf "$old_file" .PKGINFO > "$pkginfo" || continue if [ "$(cat "$pkginfo" | grep '^pkgname = ')" != "pkgname = $pkgname" ]; then continue # Package name does not match. elif [ "$(cat "$pkginfo" | grep '^arch = ')" != "arch = $CARCH" ] ; then -- 1.5.3.4

2 1

[pacman-dev] [GIT] The official pacman repository branch, master now at v3.0.0-441-g52e7e6d
by Aaron Griffin 04 Oct '07

04 Oct '07

Hello, This is an automated email from the git hooks/update script, it was generated because a ref change was pushed to the repository. Updating branch, master, via 52e7e6d74733f7a431376a9e528c4fe4d3732068 (commit) from 4453ce155c2f2c5d48e45269250ac0bab52ac63f (commit) - Log ----------------------------------------------------------------- commit 52e7e6d74733f7a431376a9e528c4fe4d3732068 Author: Chantry Xavier <shiningxc(a)gmail.com> Date: Wed Oct 3 21:02:36 2007 -0500 needs_transaction adjustments I just moved the root path check out of needs_transaction, and put it directly in pacman.c . I think this part is alright. For the other problems, I thought about doing the transaction first, in a new sync trans function, which will init and release a transaction. And then doing the commands like -Ss / -Sl / -Sg / -Si. The problem is that for commands like -Sys / -Syl / etc, only the refresh part of the transaction should be done. So I had to introduce an ugly sync_only hack. Signed-off-by: Aaron Griffin <aaronmgriffin(a)gmail.com> ----------------------------------------------------------------------- Diffstat: src/pacman/pacman.c | 4 +- src/pacman/sync.c | 115 +++++++++++++++++++++++++++++---------------------- src/pacman/util.c | 5 +-- 3 files changed, 69 insertions(+), 55 deletions(-) hooks/update --- Git Source Code Management System hooks/update refs/heads/master \ 4453ce155c2f2c5d48e45269250ac0bab52ac63f \ 52e7e6d74733f7a431376a9e528c4fe4d3732068

1 0

[pacman-dev] new failing pactest file: sync1004.py
by Nagy Gabor 02 Oct '07

02 Oct '07

Hi! I post this pactest as a motivation to review my patches ;-) This is the rewrite of sync1003.py to a more common case. Bye, ngaba ==sync1004.py== self.description = "Induced removal would break dependency (2)" sp1 = pmpkg("pkg1") sp1.conflicts = [ "depend" ] self.addpkg2db("sync", sp1) sp2 = pmpkg("pkg2") sp2.depends = ["depend"] self.addpkg2db("sync", sp2) lp = pmpkg("depend") self.addpkg2db("local", lp) self.args = "-S pkg1 pkg2" self.addrule("PACMAN_RETCODE=1") self.addrule("PKG_EXIST=depend") ---------------------------------------------------- SZTE Egyetemi Könyvtár - http://www.bibl.u-szeged.hu This mail sent through IMP: http://horde.org/imp/

3 2

[pacman-dev] [GIT] pacman branch, master now at v3.0.0-440-g4453ce1
by Dan McGee 01 Oct '07

01 Oct '07

Hello, This is an automated email from the git hooks/update script, it was generated because a ref change was pushed to the repository. Updating branch, master, via 4453ce155c2f2c5d48e45269250ac0bab52ac63f (commit) via 01e92e9ded7dc5e9a26a65bace71477a51f08bdb (commit) via 7d5dca4043b5f4ff94cd6033fccd4eadb3610a66 (commit) via d5fa2da171c9f78e3b31053fe70b5ea24962a9c8 (commit) via 5977a6ec9e50ed7777db2ebb9f41aae1dbebf3d6 (commit) from 69d68ef0b5804435bdf19d15e139f644639caa7b (commit) - Log ----------------------------------------------------------------- commit 4453ce155c2f2c5d48e45269250ac0bab52ac63f Author: Dan McGee <dan(a)archlinux.org> Date: Fri Sep 28 23:25:10 2007 -0500 libalpm: Optimize/inline the md5 functionality The md5 routines are one of the chokepoints of libalpm (main chokepoint being archive extraction). Although IO delay causes a lot of it, we can at least inline some of the md5 stuff as we aren't that concerned about space and eliminate quite a few function calls. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 01e92e9ded7dc5e9a26a65bace71477a51f08bdb Author: Dan McGee <dan(a)archlinux.org> Date: Fri Sep 28 23:22:32 2007 -0500 libalpm/deps.c: fix memleak found by valgrind Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 7d5dca4043b5f4ff94cd6033fccd4eadb3610a66 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Sep 28 23:21:56 2007 -0500 libalpm/be_files: Fix outdated code setting path for lastupdate file The code for setting the lastupdate file path still referenced both the root and dbpath, when we should only be using dbpath. Fix it. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit d5fa2da171c9f78e3b31053fe70b5ea24962a9c8 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Sep 28 12:45:14 2007 -0500 makepkg: Fix output color codes for real Make sure we reset both the bold attribute and the color attributes when it is necessary. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 5977a6ec9e50ed7777db2ebb9f41aae1dbebf3d6 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Sep 28 06:46:27 2007 -0500 Remove outdated mirror Signed-off-by: Dan McGee <dan(a)archlinux.org> ----------------------------------------------------------------------- Diffstat: etc/pacman.d/mirrorlist.in | 1 - lib/libalpm/be_files.c | 4 ++-- lib/libalpm/deps.c | 1 + lib/libalpm/md5.c | 10 +++++----- lib/libalpm/md5.h | 24 ------------------------ scripts/makepkg.sh.in | 10 +++++----- 6 files changed, 13 insertions(+), 37 deletions(-) hooks/update --- Git Source Code Management System hooks/update refs/heads/master \ 69d68ef0b5804435bdf19d15e139f644639caa7b \ 4453ce155c2f2c5d48e45269250ac0bab52ac63f

1 0