On Thu, May 8, 2008 at 6:47 PM, Allan McRae wrote:

R. Dale Thomas wrote:

...

[snip]

Well, I made the following changes to speed it up by 40%:

39c39 < pacman -Sl > $pkglist ---

...

pacman -Sl | grep "^$1" > $pkglist 42c42 < match=$(grep -m1 " $pkg " $pkglist | grep "^$1")

---

...

match=$(grep -m1 " $pkg " $pkglist)

The problem with those changes is if a package is in [testing] and [core] you only want to match it to the first repo in your pacman.conf (i.e. [testing] in this case). Your change would match it in both repos.

Allan

I used wc to shorten the output; the package list is the same in both cases, except my version also shows and matches version numbers. dmcgee@dublin /tmp $ time ./repopkg.orig testing | wc -l 28 real 0m13.195s user 0m9.166s sys 0m3.343s dmcgee@dublin /tmp $ time ./repopkg testing | wc -l 28 real 0m6.445s user 0m4.523s sys 0m1.203s The magic: pkglist=$(mktemp) pacman -Sl $1 > $pkglist pacman -Q | while read pkg; do ret=0 grep -q "$pkg" $pkglist || ret=$? if [ $ret -eq 0 ]; then echo $pkg fi done rm $pkglist And sample output: $ ./repopkg testing bzip2 1.0.5-2 exiv2 0.16-2 inputproto 1.4.3-1 kernel26 2.6.25-1 kismet 2007_10_R1-1 kqemu 1.3.0pre11-19 libldap 2.3.40-1 libtool 2.2.2-1 libx11 1.1.4-1 libxkbfile 1.0.5-1 libxv 1.0.4-1 licenses 2.4-1 links 2.1pre35-1 mesa 7.0.3rc3-1 nasm 2.01-1 ntfs-3g 1.2412-1 nvidia-96xx 96.43.05-3 openssh 5.0p1-1 patch 2.5.9-1 pinentry 0.7.3-2 postgresql 8.3.0-1 postgresql-libs 8.3.0-1 printproto 1.0.4-1 qt 4.3.4-1 sudo 1.6.9p15-1 xf86-video-nv 2.1.8-1 xorg-server 1.4.0.90-10 xproto 7.0.12-1 -Dan

Dan McGee wrote:

dmcgee@dublin /tmp $ time ./repopkg.orig testing | wc -l 28

real 0m13.195s user 0m9.166s sys 0m3.343s

dmcgee@dublin /tmp $ time ./repopkg testing | wc -l 28

real 0m6.445s user 0m4.523s sys 0m1.203s

I still wasn't happy with the performance, so I tried to do it in perl as you suggested. This is my first perl script so it wouldn't surprise me if it was poorly written. Also, I don't have testing repo enabled so I am not perfectly sure the script works correctly, but it works fine with normal repos and the performance difference is huge. $ time ./repolist.sh extra | wc -l 372 real 0m4.739s user 0m4.260s sys 0m0.440s $ time ./repolist.pl extra | wc -l 372 real 0m0.100s user 0m0.070s sys 0m0.020s #!/usr/bin/perl %count = (); $output = `pacman -Sl $ARGV[0]`; @sync = split(/\n/, $output); foreach $pkg (@sync) { @info = split(/ /, $pkg); $pkg = @info[1] . " " . @info[2]; $count{$pkg}++; } $output = `pacman -Q`; @local = split(/\n/, $output); foreach $pkg (@local) { $count{$pkg}++; } foreach $element (keys %count) { if ($count{$element} == 2) { push @{ \@intersection }, $element; } } @intersection = sort @intersection; foreach $pkg (@intersection) { print $pkg . "\n"; } Maybe someone can rewrite this in two lines? :D

Xavier wrote:

and bask in the glory and speed of perl

Oooo, I am, I am! Now I preface this by saying I know next to nothing about perl.... but from what I can tell, this script looks good. Just one query, when adding an item to a hash, does perl not return an indicator about whether a new item was created or not? If so, we can probably combine the adding of the "pacman -Q" list and the creation of the intersection. Otherwise, I'm happy to bung the copyright notice on the top and recreate the patch. Allan

Dan McGee wrote:

On Fri, May 9, 2008 at 8:29 PM, Allan McRae wrote:

...

Xavier wrote:

...

and bask in the glory and speed of perl

Oooo, I am, I am!

And while we are on the perl bus: http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=555e2091725...

Comments/questions/feedback would be much appreciated there. Perl isn't always obvious at first sight, so be sure to tell me if there are places that need more commenting.

Here we go: <code> +if ($#ARGV lt 0 || $ARGV[0] eq "--help" || $ARGV[0] eq "-h") { + print "$progname - List all packages installed from a given repo\n"; + print "Usage: $progname <pattern>\n"; + print "Example: $progname ^gnome\n"; </code> You will want to fix the description there... Otherwise, I can follow what you are doing there and without understanding the specific details I don't see any obvious problems.

...

Now I preface this by saying I know next to nothing about perl.... but from what I can tell, this script looks good. Just one query, when adding an item to a hash, does perl not return an indicator about whether a new item was created or not? If so, we can probably combine

You might be right, but I think it is a bit clearer as is? I don't think we are going to get much of a performance gain over 0.1 seconds anyway. :)

Good point. Leave it.

...

Otherwise, I'm happy to bung the copyright notice on the top and recreate the patch.

Did we decide on a name for this yet? Xavier called it repolist, you wanted to call it paclist and/or repopkg.

I vote for paclist because it lists packages, not repos and it fits with pacsearch and pacnew.