[pacman-dev] pacman -Qs first-run performance

newer
[pacman-dev] [PATCH 00/15] Full...

Philipp

1 Sep 2011 1 Sep '11

5:33 p.m.

Hi there, pacman works very well for me, with one exception: searching for installed packages. For me a -Ss takes about 5 seconds, a -Qs 25 seconds. I also noticed that this seems to be true for the first search run only, the following ones are less than a second, no matter whether it's a repo or local search. But if you want to just quickly find out a thing, 25 seconds or so is a long time. Anyway, I had a very brief look at the code and am far from understanding it, but I think libalpm/db.c handles the search, the package names, descriptions, etc. are stored in a linked list of structs, the whole thing is cached in memory only and regex wizardry is used for the search. If that's true, the bottleneck I experience is the caching. 5 seconds for -Ss is acceptable for me, but I wonder whether there's a reasonably easy way to improve the 25 seconds of -Qs. My knowledge of pacman internals is non-existant and my C skills are minimal, so I don't think I can be a lot of help with this one, but maybe there's something else I can be of help with instead. Best regards, Philipp

Show replies by date

Karol Blazewicz

1 Sep 1 Sep

5:38 p.m.

On Thu, Sep 1, 2011 at 7:33 PM, Philipp wrote:

...

Hi there, pacman works very well for me, with one exception: searching for installed packages. For me a -Ss takes about 5 seconds, a -Qs 25 seconds. I also noticed that this seems to be true for the first search run only, the following ones are less than a second, no matter whether it's a repo or local search. But if you want to just quickly find out a thing, 25 seconds or so is a long time.

Anyway, I had a very brief look at the code and am far from understanding it, but I think libalpm/db.c handles the search, the package names, descriptions, etc. are stored in a linked list of structs, the whole thing is cached in memory only and regex wizardry is used for the search. If that's true, the bottleneck I experience is the caching.

5 seconds for -Ss is acceptable for me, but I wonder whether there's a reasonably easy way to improve the 25 seconds of -Qs.

My knowledge of pacman internals is non-existant and my C skills are minimal, so I don't think I can be a lot of help with this one, but maybe there's something else I can be of help with instead.

Best regards, Philipp

I'm getting 10s -Qs first-time searches, what harddrive are you using??

Philipp Überbacher

5:57 p.m.

Excerpts from Karol Blazewicz's message of 2011-09-01 19:38:32 +0200:

...

On Thu, Sep 1, 2011 at 7:33 PM, Philipp wrote:

...
Hi there, pacman works very well for me, with one exception: searching for installed packages. For me a -Ss takes about 5 seconds, a -Qs 25 seconds. I also noticed that this seems to be true for the first search run only, the following ones are less than a second, no matter whether it's a repo or local search. But if you want to just quickly find out a thing, 25 seconds or so is a long time.

Anyway, I had a very brief look at the code and am far from understanding it, but I think libalpm/db.c handles the search, the package names, descriptions, etc. are stored in a linked list of structs, the whole thing is cached in memory only and regex wizardry is used for the search. If that's true, the bottleneck I experience is the caching.

5 seconds for -Ss is acceptable for me, but I wonder whether there's a reasonably easy way to improve the 25 seconds of -Qs.

My knowledge of pacman internals is non-existant and my C skills are minimal, so I don't think I can be a lot of help with this one, but maybe there's something else I can be of help with instead.

Best regards, Philipp

I'm getting 10s -Qs first-time searches, what harddrive are you using??

It's a normal laptop HD, Hitachi HTS54321, 160GB, 5400 rpm. It's really kind of weird that the web search takes longer than the local search, intuition dictates it should be the other way around.

Denis A. Altoé Falqueto

6:14 p.m.

On Thu, Sep 1, 2011 at 2:57 PM, Philipp Überbacher wrote:

...

Excerpts from Karol Blazewicz's message of 2011-09-01 19:38:32 +0200:

...
On Thu, Sep 1, 2011 at 7:33 PM, Philipp wrote:

...
Hi there, pacman works very well for me, with one exception: searching for installed packages. For me a -Ss takes about 5 seconds, a -Qs 25 seconds. I also noticed that this seems to be true for the first search run only, the following ones are less than a second, no matter whether it's a repo or local search. But if you want to just quickly find out a thing, 25 seconds or so is a long time.

Anyway, I had a very brief look at the code and am far from understanding it, but I think libalpm/db.c handles the search, the package names, descriptions, etc. are stored in a linked list of structs, the whole thing is cached in memory only and regex wizardry is used for the search. If that's true, the bottleneck I experience is the caching.

5 seconds for -Ss is acceptable for me, but I wonder whether there's a reasonably easy way to improve the 25 seconds of -Qs.

My knowledge of pacman internals is non-existant and my C skills are minimal, so I don't think I can be a lot of help with this one, but maybe there's something else I can be of help with instead.

Best regards, Philipp

I'm getting 10s -Qs first-time searches, what harddrive are you using??

It's a normal laptop HD, Hitachi HTS54321, 160GB, 5400 rpm. It's really kind of weird that the web search takes longer than the local search, intuition dictates it should be the other way around.

It is not a web search, though. -Ss searches in /var/lib/pacman/sync and -Qs searches in /var/lib/pacman/local. Both are limited by the same I/O bus. Theoretically, you'll have less packages installed than not installed, so the size of /var/lib/pacman/local should be smaller than /var/lib/pacman/sync. Could you compare the sizes of those two directories? -- A: Because it obfuscates the reading. Q: Why is top posting so bad? ------------------------------------------- Denis A. Altoe Falqueto Linux user #524555 -------------------------------------------

Dan McGee

6:17 p.m.

On Thu, Sep 1, 2011 at 1:14 PM, Denis A. Altoé Falqueto wrote:

...

On Thu, Sep 1, 2011 at 2:57 PM, Philipp Überbacher wrote:

...
It's really kind of weird that the web search takes longer than the local search, intuition dictates it should be the other way around.

It is not a web search, though. -Ss searches in /var/lib/pacman/sync and -Qs searches in /var/lib/pacman/local. Yes, this is definitely not a web search, I'm not sure where that idea came from? Is our documentation unclear?

...

Both are limited by the same I/O bus. Theoretically, you'll have less packages installed than not installed, so the size of /var/lib/pacman/local should be smaller than /var/lib/pacman/sync. Could you compare the sizes of those two directories? This won't really be indicative of anything- the sync databases are both single-file and compressed, meaning there is no 4K filesystem block rounding per file and text compresses well, among other things. More useful numbers would be a lot of the commands I dropped in my last email.

-Dan

Philipp Überbacher

6:37 p.m.

Excerpts from Dan McGee's message of 2011-09-01 20:17:47 +0200:

...

On Thu, Sep 1, 2011 at 1:14 PM, Denis A. Altoé Falqueto wrote:

...
On Thu, Sep 1, 2011 at 2:57 PM, Philipp Überbacher wrote:

...
It's really kind of weird that the web search takes longer than the local search, intuition dictates it should be the other way around.

It is not a web search, though. -Ss searches in /var/lib/pacman/sync and -Qs searches in /var/lib/pacman/local. Yes, this is definitely not a web search, I'm not sure where that idea came from? Is our documentation unclear?

It simply arose because it searches for packages that aren't necessarily installed. If it only searches everything it knows from the last 'y' that's fine too.

...

...
Both are limited by the same I/O bus. Theoretically, you'll have less packages installed than not installed, so the size of /var/lib/pacman/local should be smaller than /var/lib/pacman/sync. Could you compare the sizes of those two directories? This won't really be indicative of anything- the sync databases are both single-file and compressed, meaning there is no 4K filesystem block rounding per file and text compresses well, among other things. More useful numbers would be a lot of the commands I dropped in my last email.

-Dan

I ran ncdu anyway, local has 42MiB, sync 1.2MiB. The more interesting numbers are: Items: 4606. I don't know whether Items includes directories. sync accounts for 6 files, so from the file structure I assume I have either about 1500 or 2300 packages installed, but there's probably a better way to find out. Regards, Philipp

Karol Blazewicz

6:40 p.m.

On Thu, Sep 1, 2011 at 8:37 PM, Philipp Überbacher wrote:

...

Excerpts from Dan McGee's message of 2011-09-01 20:17:47 +0200:

...
On Thu, Sep 1, 2011 at 1:14 PM, Denis A. Altoé Falqueto wrote:

...
On Thu, Sep 1, 2011 at 2:57 PM, Philipp Überbacher wrote:

...
It's really kind of weird that the web search takes longer than the local search, intuition dictates it should be the other way around.

It is not a web search, though. -Ss searches in /var/lib/pacman/sync and -Qs searches in /var/lib/pacman/local. Yes, this is definitely not a web search, I'm not sure where that idea came from? Is our documentation unclear?

It simply arose because it searches for packages that aren't necessarily installed. If it only searches everything it knows from the last 'y' that's fine too.

...
...
Both are limited by the same I/O bus. Theoretically, you'll have less packages installed than not installed, so the size of /var/lib/pacman/local should be smaller than /var/lib/pacman/sync. Could you compare the sizes of those two directories? This won't really be indicative of anything- the sync databases are both single-file and compressed, meaning there is no 4K filesystem block rounding per file and text compresses well, among other things. More useful numbers would be a lot of the commands I dropped in my last email.

-Dan

I ran ncdu anyway, local has 42MiB, sync 1.2MiB. The more interesting numbers are: Items: 4606. I don't know whether Items includes directories. sync accounts for 6 files, so from the file structure I assume I have either about 1500 or 2300 packages installed, but there's probably a better way to find out.

Regards, Philipp

'pacman -Qq | wc -l' will tell you how many packages have you installed. My sync is 1.3 MB, local 8.1 MB, 1375 items.

Philipp Überbacher

6:50 p.m.

Excerpts from Karol Blazewicz's message of 2011-09-01 20:40:52 +0200:

...

On Thu, Sep 1, 2011 at 8:37 PM, Philipp Überbacher wrote:

...
Excerpts from Dan McGee's message of 2011-09-01 20:17:47 +0200:

...
On Thu, Sep 1, 2011 at 1:14 PM, Denis A. Altoé Falqueto wrote:

...
On Thu, Sep 1, 2011 at 2:57 PM, Philipp Überbacher wrote:

...
It's really kind of weird that the web search takes longer than the local search, intuition dictates it should be the other way around.

It is not a web search, though. -Ss searches in /var/lib/pacman/sync and -Qs searches in /var/lib/pacman/local. Yes, this is definitely not a web search, I'm not sure where that idea came from? Is our documentation unclear?

It simply arose because it searches for packages that aren't necessarily installed. If it only searches everything it knows from the last 'y' that's fine too.

...
...
Both are limited by the same I/O bus. Theoretically, you'll have less packages installed than not installed, so the size of /var/lib/pacman/local should be smaller than /var/lib/pacman/sync. Could you compare the sizes of those two directories? This won't really be indicative of anything- the sync databases are both single-file and compressed, meaning there is no 4K filesystem block rounding per file and text compresses well, among other things. More useful numbers would be a lot of the commands I dropped in my last email.

-Dan

I ran ncdu anyway, local has 42MiB, sync 1.2MiB. The more interesting numbers are: Items: 4606. I don't know whether Items includes directories. sync accounts for 6 files, so from the file structure I assume I have either about 1500 or 2300 packages installed, but there's probably a better way to find out.

Regards, Philipp

'pacman -Qq | wc -l' will tell you how many packages have you installed. My sync is 1.3 MB, local 8.1 MB, 1375 items.

$ pacman -Qq | wc -l 1429 Lots installed, few programs running.

Denis A. Altoé Falqueto

6:19 p.m.

On Thu, Sep 1, 2011 at 3:14 PM, Denis A. Altoé Falqueto wrote:

...

On Thu, Sep 1, 2011 at 2:57 PM, Philipp Überbacher wrote:

...
Excerpts from Karol Blazewicz's message of 2011-09-01 19:38:32 +0200:

...
On Thu, Sep 1, 2011 at 7:33 PM, Philipp wrote:

...
Hi there, pacman works very well for me, with one exception: searching for installed packages. For me a -Ss takes about 5 seconds, a -Qs 25 seconds. I also noticed that this seems to be true for the first search run only, the following ones are less than a second, no matter whether it's a repo or local search. But if you want to just quickly find out a thing, 25 seconds or so is a long time.

Anyway, I had a very brief look at the code and am far from understanding it, but I think libalpm/db.c handles the search, the package names, descriptions, etc. are stored in a linked list of structs, the whole thing is cached in memory only and regex wizardry is used for the search. If that's true, the bottleneck I experience is the caching.

5 seconds for -Ss is acceptable for me, but I wonder whether there's a reasonably easy way to improve the 25 seconds of -Qs.

My knowledge of pacman internals is non-existant and my C skills are minimal, so I don't think I can be a lot of help with this one, but maybe there's something else I can be of help with instead.

Best regards, Philipp

I'm getting 10s -Qs first-time searches, what harddrive are you using??

It's a normal laptop HD, Hitachi HTS54321, 160GB, 5400 rpm. It's really kind of weird that the web search takes longer than the local search, intuition dictates it should be the other way around.

It is not a web search, though. -Ss searches in /var/lib/pacman/sync and -Qs searches in /var/lib/pacman/local. Both are limited by the same I/O bus. Theoretically, you'll have less packages installed than not installed, so the size of /var/lib/pacman/local should be smaller than /var/lib/pacman/sync. Could you compare the sizes of those two directories?

Replying to myself.... /var/lib/pacman/sync is using tar.gz's to store the database repositories, so what I said is not true anymore (it was for old versions os pacman). /var/lib/pacman/sync is smaller than /var/lib/pacman/local, because local is made from lots of small files. Should have tested it before sending the email :) -- A: Because it obfuscates the reading. Q: Why is top posting so bad? ------------------------------------------- Denis A. Altoe Falqueto Linux user #524555 -------------------------------------------

Dan McGee

5:54 p.m.

On Thu, Sep 1, 2011 at 12:33 PM, Philipp wrote:

...

Hi there, pacman works very well for me, with one exception: searching for installed packages. For me a -Ss takes about 5 seconds, a -Qs 25 seconds. I also noticed that this seems to be true for the first search run only, the following ones are less than a second, no matter whether it's a repo or local search. But if you want to just quickly find out a thing, 25 seconds or so is a long time.

Of course subsequent runs are faster- the page cache comes into play. 25 seconds is beyond slow. I have some results from my slowest machine (HDDs are running using PIO4 right now, not even DMA), and I can't top 5 seconds on -Qs or 10 seconds on -Ss *with* caches dropped [1]. The first timing in each command is uncached; the second is then the cached time.

...

Anyway, I had a very brief look at the code and am far from understanding it, but I think libalpm/db.c handles the search, the package names, descriptions, etc. are stored in a linked list of structs, the whole thing is cached in memory only and regex wizardry is used for the search. If that's true, the bottleneck I experience is the caching.

5 seconds for -Ss is acceptable for me, but I wonder whether there's a reasonably easy way to improve the 25 seconds of -Qs. No, the bottleneck is your slow hard drive, your lack of RAM, and/or your slow CPU. Notice that on a cached search, my times don't exceed 0.3 seconds for any operation.

...

My knowledge of pacman internals is non-existant and my C skills are minimal, so I don't think I can be a lot of help with this one, but maybe there's something else I can be of help with instead. If you could tell us more about your setup, that would be very helpful. CPU info from /proc/cpuinfo, memory info from /proc/meminfo and `free -m` output, `hdparm -Tt` numbers, etc.

We've massively sped up pacman database reading for both hot and cold cases over what it was a year or so ago, so most of us consider this a "solved" problem by now. None of us see anywhere near the times you are seeing for operations, so anything you can provide us would be helpful. The output from these two commands might also be useful: echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Ss foobarbaz; /usr/bin/time pacman -Ss foobarbaz -Dan [1] [root@dublin ~]# echo 3 > /proc/sys/vm/drop_caches; time pacman -Ss foobarbaz; time pacman -Ss foobarbaz real 0m5.533s user 0m0.360s sys 0m0.553s real 0m0.349s user 0m0.343s sys 0m0.007s [root@dublin ~]# echo 3 > /proc/sys/vm/drop_caches; time pacman -Qs foobarbaz; time pacman -Qs foobarbaz real 0m10.030s user 0m0.050s sys 0m0.290s real 0m0.063s user 0m0.040s sys 0m0.020s

Philipp Überbacher

6:22 p.m.

Excerpts from Dan McGee's message of 2011-09-01 19:54:34 +0200:

...

On Thu, Sep 1, 2011 at 12:33 PM, Philipp wrote:

...
Hi there, pacman works very well for me, with one exception: searching for installed packages. For me a -Ss takes about 5 seconds, a -Qs 25 seconds. I also noticed that this seems to be true for the first search run only, the following ones are less than a second, no matter whether it's a repo or local search. But if you want to just quickly find out a thing, 25 seconds or so is a long time.

Of course subsequent runs are faster- the page cache comes into play.

I didn't expect them to help this much.

...

25 seconds is beyond slow. I have some results from my slowest machine (HDDs are running using PIO4 right now, not even DMA), and I can't top 5 seconds on -Qs or 10 seconds on -Ss *with* caches dropped [1]. The first timing in each command is uncached; the second is then the cached time.

...
Anyway, I had a very brief look at the code and am far from understanding it, but I think libalpm/db.c handles the search, the package names, descriptions, etc. are stored in a linked list of structs, the whole thing is cached in memory only and regex wizardry is used for the search. If that's true, the bottleneck I experience is the caching.

5 seconds for -Ss is acceptable for me, but I wonder whether there's a reasonably easy way to improve the 25 seconds of -Qs. No, the bottleneck is your slow hard drive, your lack of RAM, and/or your slow CPU. Notice that on a cached search, my times don't exceed 0.3 seconds for any operation.

...
My knowledge of pacman internals is non-existant and my C skills are minimal, so I don't think I can be a lot of help with this one, but maybe there's something else I can be of help with instead. If you could tell us more about your setup, that would be very helpful. CPU info from /proc/cpuinfo, memory info from /proc/meminfo and `free -m` output, `hdparm -Tt` numbers, etc.

cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 22 model name : Intel(R) Celeron(R) CPU 560 @ 2.13GHz stepping : 1 cpu MHz : 2128.233 cache size : 1024 KB fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss tm pbe syscall nx lm constant_tsc up arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl tm2 ssse3 cx16 xtpr pdcm lahf_lm dts bogomips : 4258.81 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: cat /proc/meminfo MemTotal: 2050332 kB MemFree: 964024 kB Buffers: 17888 kB Cached: 287340 kB SwapCached: 10868 kB Active: 487996 kB Inactive: 276964 kB Active(anon): 448104 kB Inactive(anon): 176348 kB Active(file): 39892 kB Inactive(file): 100616 kB Unevictable: 224952 kB Mlocked: 130928 kB SwapTotal: 2104508 kB SwapFree: 2003180 kB Dirty: 1432 kB Writeback: 0 kB AnonPages: 675560 kB Mapped: 134016 kB Shmem: 73536 kB Slab: 46608 kB SReclaimable: 14012 kB SUnreclaim: 32596 kB KernelStack: 1472 kB PageTables: 8620 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 3129672 kB Committed_AS: 1367008 kB VmallocTotal: 34359738367 kB VmallocUsed: 542528 kB VmallocChunk: 34359167180 kB HardwareCorrupted: 0 kB AnonHugePages: 241664 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 13120 kB DirectMap2M: 2074624 kB free -m total used free shared buffers cached Mem: 2002 1080 922 0 17 299 -/+ buffers/cache: 763 1239 Swap: 2055 98 1956 hdparm -Tt /dev/sda /dev/sda: Timing cached reads: 1426 MB in 2.00 seconds = 712.73 MB/sec Timing buffered disk reads: 166 MB in 3.01 seconds = 55.09 MB/sec

...

We've massively sped up pacman database reading for both hot and cold cases over what it was a year or so ago, so most of us consider this a "solved" problem by now. None of us see anywhere near the times you are seeing for operations, so anything you can provide us would be helpful.

The output from these two commands might also be useful:

echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Ss foobarbaz; /usr/bin/time pacman -Ss foobarbaz

-Dan

echo 3 > /proc/sys/vm/drop_caches; time pacman -Qs foobarbaz; time pacman -Qs foobarbaz real 1m45.235s user 0m0.127s sys 0m0.590s real 0m0.097s user 0m0.043s sys 0m0.023s echo 3 > /proc/sys/vm/drop_caches; time pacman -Ss foobarbaz; time pacman -Ss foobarbaz real 0m1.299s user 0m0.200s sys 0m0.023s real 0m0.252s user 0m0.200s sys 0m0.010s My 25 seconds weren't measured, it seems like it's even worse in reality. Best regards, Philipp

...

[1] [root@dublin ~]# echo 3 > /proc/sys/vm/drop_caches; time pacman -Ss foobarbaz; time pacman -Ss foobarbaz

real 0m5.533s user 0m0.360s sys 0m0.553s

real 0m0.349s user 0m0.343s sys 0m0.007s

[root@dublin ~]# echo 3 > /proc/sys/vm/drop_caches; time pacman -Qs foobarbaz; time pacman -Qs foobarbaz

real 0m10.030s user 0m0.050s sys 0m0.290s

real 0m0.063s user 0m0.040s sys 0m0.020s

Dan McGee

6:41 p.m.

...

Excerpts from Dan McGee's message of 2011-09-01 19:54:34 +0200: free -m total used free shared buffers cached Mem: 2002 1080 922 0 17 299 -/+ buffers/cache: 763 1239 Swap: 2055 98 1956 Wow- what are you running on a laptop that is keeping 763 MB of RAM

On Thu, Sep 1, 2011 at 1:22 PM, Philipp Überbacher wrote: pegged? Although there appears to be something else in play here.

...

hdparm -Tt /dev/sda

/dev/sda: Timing cached reads: 1426 MB in 2.00 seconds = 712.73 MB/sec Timing buffered disk reads: 166 MB in 3.01 seconds = 55.09 MB/sec This is a bit slow, but nothing that should impact your performance that much.

...

echo 3 > /proc/sys/vm/drop_caches; time pacman -Qs foobarbaz; time pacman -Qs foobarbaz

real 1m45.235s user 0m0.127s sys 0m0.590s

real 0m0.097s user 0m0.043s sys 0m0.023s Can you repeat this and use /usr/bin/time instead of just plain "time"? You may have to install the "time" package. echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz

Once you do that (but please get the numbers first!), try running `pacman-optimize` then rerunning the above test. Finally, what is the output of df -h for whatever drive /var/lib is on? Is it near capacity? And what filesystem with what mount options?

...

echo 3 > /proc/sys/vm/drop_caches; time pacman -Ss foobarbaz; time pacman -Ss foobarbaz

real 0m1.299s user 0m0.200s sys 0m0.023s

real 0m0.252s user 0m0.200s sys 0m0.010s

At least this one is OK. -Dan

Philipp Überbacher

7:03 p.m.

Excerpts from Dan McGee's message of 2011-09-01 20:41:27 +0200:

...

...
Excerpts from Dan McGee's message of 2011-09-01 19:54:34 +0200: free -m total used free shared buffers cached Mem: 2002 1080 922 0 17 299 -/+ buffers/cache: 763 1239 Swap: 2055 98 1956 Wow- what are you running on a laptop that is keeping 763 MB of RAM

On Thu, Sep 1, 2011 at 1:22 PM, Philipp Überbacher wrote: pegged? Although there appears to be something else in play here.

At least half of that is due to firefox, a music player with a lot of songs in the playlist (aqualung), smplayer and the mail client (sup) should account for most of the rest.

...

...
hdparm -Tt /dev/sda

/dev/sda: Timing cached reads: 1426 MB in 2.00 seconds = 712.73 MB/sec Timing buffered disk reads: 166 MB in 3.01 seconds = 55.09 MB/sec This is a bit slow, but nothing that should impact your performance that much.

...
echo 3 > /proc/sys/vm/drop_caches; time pacman -Qs foobarbaz; time pacman -Qs foobarbaz

real 1m45.235s user 0m0.127s sys 0m0.590s

real 0m0.097s user 0m0.043s sys 0m0.023s Can you repeat this and use /usr/bin/time instead of just plain "time"? You may have to install the "time" package.

Ah, yes, that was necessary for /usr/bin/time to work.

...

echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz

echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz Command exited with non-zero status 1 0.08user 0.62system 0:48.48elapsed 1%CPU (0avgtext+0avgdata 3428maxresident)k 173016inputs+0outputs (16major+2353minor)pagefaults 0swaps Command exited with non-zero status 1 0.02user 0.03system 0:00.08elapsed 78%CPU (0avgtext+0avgdata 3428maxresident)k 0inputs+0outputs (0major+2369minor)pagefaults 0swaps echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Ss foobarbaz; /usr/bin/time pacman -Ss foobarbaz Command exited with non-zero status 1 0.19user 0.03system 0:01.51elapsed 14%CPU (0avgtext+0avgdata 8440maxresident)k 9864inputs+0outputs (16major+2177minor)pagefaults 0swaps Command exited with non-zero status 1 0.19user 0.01system 0:00.27elapsed 76%CPU (0avgtext+0avgdata 8444maxresident)k 0inputs+0outputs (0major+2194minor)pagefaults 0swaps

...

Once you do that (but please get the numbers first!), try running `pacman-optimize` then rerunning the above test.

echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz Command exited with non-zero status 1 0.07user 0.46system 0:30.68elapsed 1%CPU (0avgtext+0avgdata 3428maxresident)k 31752inputs+0outputs (16major+2353minor)pagefaults 0swaps Command exited with non-zero status 1 0.02user 0.04system 0:00.09elapsed 74%CPU (0avgtext+0avgdata 3428maxresident)k 0inputs+0outputs (0major+2369minor)pagefaults 0swaps echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Ss foobarbaz; /usr/bin/time pacman -Ss foobarbaz Command exited with non-zero status 1 0.19user 0.02system 0:01.57elapsed 14%CPU (0avgtext+0avgdata 8440maxresident)k 8832inputs+0outputs (16major+2178minor)pagefaults 0swaps Command exited with non-zero status 1 0.20user 0.00system 0:00.26elapsed 80%CPU (0avgtext+0avgdata 8440maxresident)k 0inputs+0outputs (0major+2193minor)pagefaults 0swaps

...

Finally, what is the output of df -h for whatever drive /var/lib is on? Is it near capacity? And what filesystem with what mount options?

from df -h /dev/disk/by-uuid/456aa338-0417-43ca-85b2-a03d1b36ec1e ext4 21G 13G 6.2G 68% / from mtab /dev/disk/by-uuid/456aa338-0417-43ca-85b2-a03d1b36ec1e / ext4 rw,noatime,user_xattr,acl,barrier=1,nodelalloc,data=ordered 0 0

...

...
echo 3 > /proc/sys/vm/drop_caches; time pacman -Ss foobarbaz; time pacman -Ss foobarbaz

real 0m1.299s user 0m0.200s sys 0m0.023s

real 0m0.252s user 0m0.200s sys 0m0.010s

At least this one is OK.

-Dan

Ok, thanks a lot Dan and everyone else. Regards, Philipp

Philipp Überbacher

11 Sep 11 Sep

9:22 a.m.

Excerpts from Philipp Überbacher's message of 2011-09-01 21:03:47 +0200:

...

Excerpts from Dan McGee's message of 2011-09-01 20:41:27 +0200:

...
...
Excerpts from Dan McGee's message of 2011-09-01 19:54:34 +0200: free -m total used free shared buffers cached Mem: 2002 1080 922 0 17 299 -/+ buffers/cache: 763 1239 Swap: 2055 98 1956 Wow- what are you running on a laptop that is keeping 763 MB of RAM

On Thu, Sep 1, 2011 at 1:22 PM, Philipp Überbacher wrote: pegged? Although there appears to be something else in play here.

At least half of that is due to firefox, a music player with a lot of songs in the playlist (aqualung), smplayer and the mail client (sup) should account for most of the rest.

...
...
hdparm -Tt /dev/sda

/dev/sda: Timing cached reads: 1426 MB in 2.00 seconds = 712.73 MB/sec Timing buffered disk reads: 166 MB in 3.01 seconds = 55.09 MB/sec This is a bit slow, but nothing that should impact your performance that much.

...
echo 3 > /proc/sys/vm/drop_caches; time pacman -Qs foobarbaz; time pacman -Qs foobarbaz

real 1m45.235s user 0m0.127s sys 0m0.590s

real 0m0.097s user 0m0.043s sys 0m0.023s Can you repeat this and use /usr/bin/time instead of just plain "time"? You may have to install the "time" package.

Ah, yes, that was necessary for /usr/bin/time to work.

...
echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz

echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz Command exited with non-zero status 1 0.08user 0.62system 0:48.48elapsed 1%CPU (0avgtext+0avgdata 3428maxresident)k 173016inputs+0outputs (16major+2353minor)pagefaults 0swaps Command exited with non-zero status 1 0.02user 0.03system 0:00.08elapsed 78%CPU (0avgtext+0avgdata 3428maxresident)k 0inputs+0outputs (0major+2369minor)pagefaults 0swaps

echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Ss foobarbaz; /usr/bin/time pacman -Ss foobarbaz Command exited with non-zero status 1 0.19user 0.03system 0:01.51elapsed 14%CPU (0avgtext+0avgdata 8440maxresident)k 9864inputs+0outputs (16major+2177minor)pagefaults 0swaps Command exited with non-zero status 1 0.19user 0.01system 0:00.27elapsed 76%CPU (0avgtext+0avgdata 8444maxresident)k 0inputs+0outputs (0major+2194minor)pagefaults 0swaps

...
Once you do that (but please get the numbers first!), try running `pacman-optimize` then rerunning the above test.

echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Qs foobarbaz; /usr/bin/time pacman -Qs foobarbaz Command exited with non-zero status 1 0.07user 0.46system 0:30.68elapsed 1%CPU (0avgtext+0avgdata 3428maxresident)k 31752inputs+0outputs (16major+2353minor)pagefaults 0swaps Command exited with non-zero status 1 0.02user 0.04system 0:00.09elapsed 74%CPU (0avgtext+0avgdata 3428maxresident)k 0inputs+0outputs (0major+2369minor)pagefaults 0swaps

echo 3 > /proc/sys/vm/drop_caches; /usr/bin/time pacman -Ss foobarbaz; /usr/bin/time pacman -Ss foobarbaz Command exited with non-zero status 1 0.19user 0.02system 0:01.57elapsed 14%CPU (0avgtext+0avgdata 8440maxresident)k 8832inputs+0outputs (16major+2178minor)pagefaults 0swaps Command exited with non-zero status 1 0.20user 0.00system 0:00.26elapsed 80%CPU (0avgtext+0avgdata 8440maxresident)k 0inputs+0outputs (0major+2193minor)pagefaults 0swaps

...
Finally, what is the output of df -h for whatever drive /var/lib is on? Is it near capacity? And what filesystem with what mount options?

from df -h /dev/disk/by-uuid/456aa338-0417-43ca-85b2-a03d1b36ec1e ext4 21G 13G 6.2G 68% /

from mtab /dev/disk/by-uuid/456aa338-0417-43ca-85b2-a03d1b36ec1e / ext4 rw,noatime,user_xattr,acl,barrier=1,nodelalloc,data=ordered 0 0

...
...
echo 3 > /proc/sys/vm/drop_caches; time pacman -Ss foobarbaz; time pacman -Ss foobarbaz

real 0m1.299s user 0m0.200s sys 0m0.023s

real 0m0.252s user 0m0.200s sys 0m0.010s

At least this one is OK.

-Dan

Ok, thanks a lot Dan and everyone else.

Regards, Philipp

So what do you guys think, is it simply the combination of slow HD and many packages or is there something wrong with my system that could be fixed by changing some config? Regards, Philipp

4621

Age (days ago)

4631

Last active (days ago)

List overview

Download

13 comments

5 participants

participants (5)

Dan McGee
Denis A. Altoé Falqueto
Karol Blazewicz
Philipp
Philipp Überbacher