[arch-general] Way too much kworkers

[Anatol Pomozov]

Hi

On Tue, Oct 1, 2013 at 2:30 PM, Dimitris Zervas <dzervas at dzervas.gr> wrote:
> Um, I have a very powerful pc, so I get no slow downs. i7-3820 with 16GB of
> ram.

So if your system works fine then I am not sure what you are complaining about.

On a bit more serious note. The threading in Linux kernel is quite
effective. A thread data takes just a few kibibytes of non-swappable
kernel memory. And 200 threads is nothing for modern server/desktop
machines. I saw servers with 30K threads that were doing absolutely
fine. And by the way - a of the kworkers are per-CPU so more (virtual)
CPU cores you have more kworkers threads you will see.

The problem with threading is not the number of all threads (when most
of the threads are sleeping and doing nothing), but *active* jumping
from one thread to another. Such behavior happens in networking
services where a thread starts, does some small amount of job then get
blocked because it doing a disk or database access, switches to
another task that does also a little of work and switches back. Such
behavior causes cache thrashing, TLB cache flushes and thus poor
performance. Some people try to fight with this problem by avoiding
thread switches using some kind of callback frameworks (a-la node.js)
or user-space threads (a-la goroutines in Go).

As I said if kernel threads are not active then they almost free for
your system. And there is a good reason why kernel started using
kworkers more actively. Imagine you write a kernel code that need a
delayed execution of two independent functions A() and B(). You can
start one thread and call A() and B() serially, but in this case if
A() is blocked then B() will not run until A() is done. If you start
two threads, then when A() will be blocked, CPU can switch execution
to B(). Thus your tasks will finish faster and the system becomes more
responsive. And most server/desktop users choose overall system
responsiveness even if it takes a little bit more memory.


> htop reports no CPU usage, but free -m reports some interesting memory
> consumption.
> free -m
>                     total       used        free     shared    buffers
> cached
> Mem:         16030      15868        161             0        150      13831
> -/+ buffers/cache:       1886      14143
> Swap:            0          0          0
>
> I know that "used" is for caching/etc., but 15GB is a bit too much.
> Downgrading the kernel is not an option because the kernel is patched and
> downgrading would take just too much.
> I know have 234 kworkers, without suspending.

This cache usage has noting to do with number of kworkers. This amount
is used by buffer cache. Buffer cache contains data read from slow
devices like disk to avoid reading it again (this is what cache for).
You can drop the cache by

sync; echo 3 | sudo tee /proc/sys/vm/drop_caches