[pacman-dev] List packages from a given repo
Hi, I attach a script tentatively called "repopkg" that lists all packages installed from a given repo. I find this useful for monitoring what I have installed from [testing]. It is not entirely fool proof as it assumes you have installed packages using "pacman -S pkg" and not "pacman -S repo/pkg". A couple of queries. Can someone come up with a better name which is not too long? And any comments before I submit the for inclusion in the contrib directory (it is best to come as a git patch right?). Cheers, Allan #!/bin/bash # repopkg - List all packages installed from a given repo # # Copyright (C) 2008 Allan McRae <mcrae_allan@hotmail.com> # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. # TODO: Could match version numbers to catch packages # install with pacman -S repo/pkg readonly progname="repopkg" readonly version="1.0" if [ -z "$1" -o "$1" = "--help" -o "$1" = "-h" ]; then echo "Usage: $progname <repo>" echo "Ex: $progname testing" exit 0 fi if [ "$1" = "--version" -o "$1" = "-v" ]; then echo "$progname version $version" echo "Copyright (C) 2008 Allan McRae" exit 0 fi pkglist=$(mktemp) pacman -Sl > $pkglist for pkg in $(pacman -Qq); do match=$(grep -m1 " $pkg " $pkglist | grep "^$1") if [ -n "${match}" ]; then echo $pkg fi done rm $pkglist
On Thu, May 8, 2008 at 9:36 AM, Allan McRae <allan.mcrae@qimr.edu.au> wrote:
Hi,
I attach a script tentatively called "repopkg" that lists all packages installed from a given repo. I find this useful for monitoring what I have installed from [testing]. It is not entirely fool proof as it assumes you have installed packages using "pacman -S pkg" and not "pacman -S repo/pkg". Handy little script.
A couple of queries. Can someone come up with a better name which is not too long? And any comments before I submit the for inclusion in the contrib directory (it is best to come as a git patch right?). Yeah, that would be great, so you can add a bit of a description in the commit message.
Cheers, Allan
#!/bin/bash # repopkg - List all packages installed from a given repo # # Copyright (C) 2008 Allan McRae <mcrae_allan@hotmail.com> # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>.
# TODO: Could match version numbers to catch packages # install with pacman -S repo/pkg Did you take a look at my pacsearch script? That does some sort of intelligent matching of package versions.
readonly progname="repopkg" readonly version="1.0"
if [ -z "$1" -o "$1" = "--help" -o "$1" = "-h" ]; then echo "Usage: $progname <repo>" echo "Ex: $progname testing"
Just spell out "Example"? We can afford a few more bytes. :)
exit 0 fi
if [ "$1" = "--version" -o "$1" = "-v" ]; then echo "$progname version $version" echo "Copyright (C) 2008 Allan McRae" exit 0 fi
pkglist=$(mktemp) pacman -Sl > $pkglist
for pkg in $(pacman -Qq); do match=$(grep -m1 " $pkg " $pkglist | grep "^$1")
Not exactly cheap as you have to do two grep calls for every package you list. Perhaps cmp or diff could be used? And now for the real off-the-wall suggestion- doing this in perl would give you the power of regular expressions and probably make this all real easy. That is if you can figure out perl.
if [ -n "${match}" ]; then echo $pkg fi done
rm $pkglist
Dan McGee wrote:
[snip]
# TODO: Could match version numbers to catch packages # install with pacman -S repo/pkg
Did you take a look at my pacsearch script? That does some sort of intelligent matching of package versions.
I like the description "some sort of intelligent matching"... Sounds convincing!
readonly progname="repopkg" readonly version="1.0"
if [ -z "$1" -o "$1" = "--help" -o "$1" = "-h" ]; then echo "Usage: $progname <repo>" echo "Ex: $progname testing"
Just spell out "Example"? We can afford a few more bytes. :)
I was copying your pacsearch script!
[snip]
for pkg in $(pacman -Qq); do match=$(grep -m1 " $pkg " $pkglist | grep "^$1")
Not exactly cheap as you have to do two grep calls for every package you list. Perhaps cmp or diff could be used?
And now for the real off-the-wall suggestion- doing this in perl would give you the power of regular expressions and probably make this all real easy. That is if you can figure out perl.
The saying that "perl is like an explosion in an ascii factory" pretty much describes how I see perl... If/when I add the pkgver matching then this will disappear. Also, I think I will rename the script "paclist" as that is perhaps slightly better that "repopkg". It also has the "pac" prefix which matches pacsearch and pacdiff. Allan
Allan McRae wrote:
Hi,
I attach a script tentatively called "repopkg" that lists all packages installed from a given repo. I find this useful for monitoring what I have installed from [testing]. It is not entirely fool proof as it assumes you have installed packages using "pacman -S pkg" and not "pacman -S repo/pkg".
A couple of queries. Can someone come up with a better name which is not too long? And any comments before I submit the for inclusion in the contrib directory (it is best to come as a git patch right?). Cheers, Allan
[snip] Well, I made the following changes to speed it up by 40%: 39c39 < pacman -Sl > $pkglist ---
pacman -Sl | grep "^$1" > $pkglist 42c42 < match=$(grep -m1 " $pkg " $pkglist | grep "^$1")
match=$(grep -m1 " $pkg " $pkglist)
R. Dale Thomas wrote:
[snip]
Well, I made the following changes to speed it up by 40%:
39c39 < pacman -Sl > $pkglist ---
pacman -Sl | grep "^$1" > $pkglist 42c42 < match=$(grep -m1 " $pkg " $pkglist | grep "^$1")
match=$(grep -m1 " $pkg " $pkglist)
The problem with those changes is if a package is in [testing] and [core] you only want to match it to the first repo in your pacman.conf (i.e. [testing] in this case). Your change would match it in both repos. Allan
On Thu, May 8, 2008 at 6:47 PM, Allan McRae <mcrae_allan@hotmail.com> wrote:
R. Dale Thomas wrote:
[snip]
Well, I made the following changes to speed it up by 40%:
39c39 < pacman -Sl > $pkglist ---
pacman -Sl | grep "^$1" > $pkglist 42c42 < match=$(grep -m1 " $pkg " $pkglist | grep "^$1")
match=$(grep -m1 " $pkg " $pkglist)
The problem with those changes is if a package is in [testing] and [core] you only want to match it to the first repo in your pacman.conf (i.e. [testing] in this case). Your change would match it in both repos.
Allan
I used wc to shorten the output; the package list is the same in both cases, except my version also shows and matches version numbers. dmcgee@dublin /tmp $ time ./repopkg.orig testing | wc -l 28 real 0m13.195s user 0m9.166s sys 0m3.343s dmcgee@dublin /tmp $ time ./repopkg testing | wc -l 28 real 0m6.445s user 0m4.523s sys 0m1.203s The magic: pkglist=$(mktemp) pacman -Sl $1 > $pkglist pacman -Q | while read pkg; do ret=0 grep -q "$pkg" $pkglist || ret=$? if [ $ret -eq 0 ]; then echo $pkg fi done rm $pkglist And sample output: $ ./repopkg testing bzip2 1.0.5-2 exiv2 0.16-2 inputproto 1.4.3-1 kernel26 2.6.25-1 kismet 2007_10_R1-1 kqemu 1.3.0pre11-19 libldap 2.3.40-1 libtool 2.2.2-1 libx11 1.1.4-1 libxkbfile 1.0.5-1 libxv 1.0.4-1 licenses 2.4-1 links 2.1pre35-1 mesa 7.0.3rc3-1 nasm 2.01-1 ntfs-3g 1.2412-1 nvidia-96xx 96.43.05-3 openssh 5.0p1-1 patch 2.5.9-1 pinentry 0.7.3-2 postgresql 8.3.0-1 postgresql-libs 8.3.0-1 printproto 1.0.4-1 qt 4.3.4-1 sudo 1.6.9p15-1 xf86-video-nv 2.1.8-1 xorg-server 1.4.0.90-10 xproto 7.0.12-1 -Dan
Dan McGee wrote:
The magic:
pkglist=$(mktemp) pacman -Sl $1 > $pkglist
pacman -Q | while read pkg; do ret=0 grep -q "$pkg" $pkglist || ret=$? if [ $ret -eq 0 ]; then echo $pkg fi done
rm $pkglist
Nice! I will submit the patch which adds this script to contrib later today. Cheers, Allan
Dan McGee wrote:
dmcgee@dublin /tmp $ time ./repopkg.orig testing | wc -l 28
real 0m13.195s user 0m9.166s sys 0m3.343s
dmcgee@dublin /tmp $ time ./repopkg testing | wc -l 28
real 0m6.445s user 0m4.523s sys 0m1.203s
I still wasn't happy with the performance, so I tried to do it in perl as you suggested. This is my first perl script so it wouldn't surprise me if it was poorly written. Also, I don't have testing repo enabled so I am not perfectly sure the script works correctly, but it works fine with normal repos and the performance difference is huge. $ time ./repolist.sh extra | wc -l 372 real 0m4.739s user 0m4.260s sys 0m0.440s $ time ./repolist.pl extra | wc -l 372 real 0m0.100s user 0m0.070s sys 0m0.020s #!/usr/bin/perl %count = (); $output = `pacman -Sl $ARGV[0]`; @sync = split(/\n/, $output); foreach $pkg (@sync) { @info = split(/ /, $pkg); $pkg = @info[1] . " " . @info[2]; $count{$pkg}++; } $output = `pacman -Q`; @local = split(/\n/, $output); foreach $pkg (@local) { $count{$pkg}++; } foreach $element (keys %count) { if ($count{$element} == 2) { push @{ \@intersection }, $element; } } @intersection = sort @intersection; foreach $pkg (@intersection) { print $pkg . "\n"; } Maybe someone can rewrite this in two lines? :D
Xavier wrote:
I still wasn't happy with the performance, so I tried to do it in perl as you suggested. This is my first perl script so it wouldn't surprise me if it was poorly written.
Thanks to Dan for cleaning various things and commenting the code, it should be better perl code and much easier to read now. Then I also stole the args parsing from Allan's script to complete it.
Xavier wrote:
and bask in the glory and speed of perl
Oooo, I am, I am! Now I preface this by saying I know next to nothing about perl.... but from what I can tell, this script looks good. Just one query, when adding an item to a hash, does perl not return an indicator about whether a new item was created or not? If so, we can probably combine the adding of the "pacman -Q" list and the creation of the intersection. Otherwise, I'm happy to bung the copyright notice on the top and recreate the patch. Allan
On Fri, May 9, 2008 at 8:29 PM, Allan McRae <mcrae_allan@hotmail.com> wrote:
Xavier wrote:
and bask in the glory and speed of perl
Oooo, I am, I am! And while we are on the perl bus: http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=555e2091725...
Now I preface this by saying I know next to nothing about perl.... but from what I can tell, this script looks good. Just one query, when adding an item to a hash, does perl not return an indicator about whether a new item was created or not? If so, we can probably combine You might be right, but I think it is a bit clearer as is? I don't
Comments/questions/feedback would be much appreciated there. Perl isn't always obvious at first sight, so be sure to tell me if there are places that need more commenting. think we are going to get much of a performance gain over 0.1 seconds anyway. :) Note that the hash table holds (key, value) pairs of (pkginfo, # of times found). I didn't get this at first but it makes sense.
Otherwise, I'm happy to bung the copyright notice on the top and recreate the patch. Did we decide on a name for this yet? Xavier called it repolist, you wanted to call it paclist and/or repopkg.
-Dan
Dan McGee wrote:
On Fri, May 9, 2008 at 8:29 PM, Allan McRae <mcrae_allan@hotmail.com> wrote:
Xavier wrote:
and bask in the glory and speed of perl
Oooo, I am, I am!
And while we are on the perl bus: http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=555e2091725...
Comments/questions/feedback would be much appreciated there. Perl isn't always obvious at first sight, so be sure to tell me if there are places that need more commenting.
Here we go: <code> +if ($#ARGV lt 0 || $ARGV[0] eq "--help" || $ARGV[0] eq "-h") { + print "$progname - List all packages installed from a given repo\n"; + print "Usage: $progname <pattern>\n"; + print "Example: $progname ^gnome\n"; </code> You will want to fix the description there... Otherwise, I can follow what you are doing there and without understanding the specific details I don't see any obvious problems.
Now I preface this by saying I know next to nothing about perl.... but from what I can tell, this script looks good. Just one query, when adding an item to a hash, does perl not return an indicator about whether a new item was created or not? If so, we can probably combine
You might be right, but I think it is a bit clearer as is? I don't think we are going to get much of a performance gain over 0.1 seconds anyway. :)
Good point. Leave it.
Otherwise, I'm happy to bung the copyright notice on the top and recreate the patch.
Did we decide on a name for this yet? Xavier called it repolist, you wanted to call it paclist and/or repopkg.
I vote for paclist because it lists packages, not repos and it fits with pacsearch and pacnew.
Dan McGee wrote:
Comments/questions/feedback would be much appreciated there. Perl isn't always obvious at first sight, so be sure to tell me if there are places that need more commenting.
Looks fine to me. But doing a step back over pacsearch / paclist, it seems like the task of adding an install marker (***) would be much easier to do in pacman itself. Then paclist would be a one liner : pacman -Sl community | grep "^*" And pacsearch should be much simpler as well, it would just have to worry about colors. And keeping all that color stuff in an external script seems reasonable to me. So the main advantage of doing it directly in pacman would be an overall code reduction / simplification. But the downside is an useless loss of performance for people who don't need that feature. -Sl before the patch is 0.085ms and after it is 0.225ms. I believe that pacman lookups are not very efficient since they do a basic linear search. But I don't know how much we could gain here, for example by using a binary search over a sorted list, or using a hash table for the pkgcache. Otherwise, this feature of marking installed packages could be disabled by default and enabled with a flag.
Otherwise, I'm happy to bung the copyright notice on the top and recreate the patch. Did we decide on a name for this yet? Xavier called it repolist, you wanted to call it paclist and/or repopkg.
I am fine with paclist, so I just changed the name and added the copyright notice, I can then put it on my git repo . diff --git a/src/pacman/sync.c b/src/pacman/sync.c index 3c6dfd5..f6da9c4 100644 --- a/src/pacman/sync.c +++ b/src/pacman/sync.c @@ -296,12 +296,17 @@ static int sync_search(alpm_list_t *syncs, alpm_list_t *targets) for(j = ret; j; j = alpm_list_next(j)) { alpm_list_t *grp; pmpkg_t *pkg = alpm_list_getdata(j); - + const char *pkgname = alpm_pkg_get_name(pkg); + const char *pkgver = alpm_pkg_get_version(pkg); + pmpkg_t *lpkg = alpm_db_get_pkg(db_local, pkgname); + const char *s = ""; + if(lpkg && strcmp(pkgver, alpm_pkg_get_version(lpkg)) == 0) { + s = "***"; + } if (!config->quiet) { - printf("%s/%s %s", alpm_db_get_name(db), alpm_pkg_get_name(pkg), - alpm_pkg_get_version(pkg)); + printf("%s%s/%s %s", s, alpm_db_get_name(db), pkgname, pkgver); } else { - printf("%s", alpm_pkg_get_name(pkg)); + printf("%s", pkgname); } /* print the package size with the output if ShowSize option set */ @@ -506,11 +511,17 @@ static int sync_list(alpm_list_t *syncs, alpm_list_t *targets) for(j = alpm_db_getpkgcache(db); j; j = alpm_list_next(j)) { pmpkg_t *pkg = alpm_list_getdata(j); + const char *pkgname = alpm_pkg_get_name(pkg); + const char *pkgver = alpm_pkg_get_version(pkg); + pmpkg_t *lpkg = alpm_db_get_pkg(db_local, pkgname); + const char *s = ""; + if(lpkg && strcmp(pkgver, alpm_pkg_get_version(lpkg)) == 0) { + s = "***"; + } if (!config->quiet) { - printf("%s %s %s\n", alpm_db_get_name(db), alpm_pkg_get_name(pkg), - alpm_pkg_get_version(pkg)); + printf("%s%s %s %s\n", s, alpm_db_get_name(db), pkgname, pkgver); } else { - printf("%s\n", alpm_pkg_get_name(pkg)); + printf("%s\n", pkgname); } } }
participants (5)
-
Allan McRae
-
Allan McRae
-
Dan McGee
-
R. Dale Thomas
-
Xavier