[arch-general] Way too much kworkers
Hello, I've googled a bit and saw that other people have this problem too, but without solution. They had maybe 8 or 10 kworker processes. After 2 suspends, I have 172. ps -ef | grep kworker | wc -l 173 (one of the line is grep's process). Is there any solution?
On Tue, Oct 1, 2013 at 9:16 PM, Dimitris Zervas <dzervas@dzervas.gr> wrote:
Hello,
I've googled a bit and saw that other people have this problem too, but without solution.
What do these processes do? Slow down you computer, eat up RAM ...?
They had maybe 8 or 10 kworker processes. After 2 suspends, I have 172.
ps -ef | grep kworker | wc -l 173
$ pgrep -c kworker 7
(one of the line is grep's process). Is there any solution?
-----Original Message----- From: arch-general [mailto:arch-general-bounces@archlinux.org] On Behalf Of Karol Blazewicz
On Tue, Oct 1, 2013 at 9:16 PM, Dimitris Zervas <dzervas@dzervas.gr> wrote:
Hello,
I've googled a bit and saw that other people have this problem too, but without solution.
What do these processes do? Slow down you computer, eat up RAM ...?
If I'm not mistaken, kworker is a kernel task used to defer work from interrupt handlers into a task context. There used to be 1 kworker per cpu but workqueue design has changed. You can learn by reading Documentation/workqueue.txt in the kernel documentation package. Usually, kworkers are used by device drivers, you could maybe check if downgrading your kernel helps and narrow down the issue to a particular release. ________________________________ CONFIDENTIALITY : This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium.
On Tue, 1 Oct 2013 22:16:53 +0300 Dimitris Zervas <dzervas@dzervas.gr> wrote:
Hello,
I've googled a bit and saw that other people have this problem too, but without solution. They had maybe 8 or 10 kworker processes. After 2 suspends, I have 172.
ps -ef | grep kworker | wc -l 173
(one of the line is grep's process). Is there any solution?
These [kworker] processes were probably created during the suspend/resume phase and have not been killed yet. If they don't consume CPU time, you have nothing to be concerned about. From Documentation/workqueue.txt: "Keeping idle workers around doesn't cost other than the memory space for kthreads, so cmwq holds onto idle ones for a while before killing them." However, I don't know how long "a while" is... Cheers, -- Leonid Isaev GnuPG key: 0x164B5A6D Fingerprint: C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
Um, I have a very powerful pc, so I get no slow downs. i7-3820 with 16GB of ram. htop reports no CPU usage, but free -m reports some interesting memory consumption. free -m total used free shared buffers cached Mem: 16030 15868 161 0 150 13831 -/+ buffers/cache: 1886 14143 Swap: 0 0 0 I know that "used" is for caching/etc., but 15GB is a bit too much. Downgrading the kernel is not an option because the kernel is patched and downgrading would take just too much. I know have 234 kworkers, without suspending. Thank you for your answers
2013/10/1 Dimitris Zervas <dzervas@dzervas.gr>:
Um, I have a very powerful pc, so I get no slow downs. i7-3820 with 16GB of ram. htop reports no CPU usage, but free -m reports some interesting memory consumption. free -m total used free shared buffers cached Mem: 16030 15868 161 0 150 13831 -/+ buffers/cache: 1886 14143 Swap: 0 0 0
I know that "used" is for caching/etc., but 15GB is a bit too much. Downgrading the kernel is not an option because the kernel is patched and downgrading would take just too much. I know have 234 kworkers, without suspending.
Thank you for your answers
This is perfectly normal, and expected, if you haven't rebooted for a while. Unused RAM is wasted RAM. You should care about the "-/+ buffers/cache:" figures instead. -- Pedro Emílio Machado de Brito Engenharia de Computação 2012 - Unicamp Coordenador Tecnológico - Centro Acadêmico da Computação (CACo) (19) 99873-0963 - Facebook/Skype: pedroembrito - Github: pemb 18hxPDGcNDpsYYKJ2aghSh3mm9n3c4C7pf "Repair what you can — but when you must fail, fail noisily and as soon as possible." · Eric S. Raymond
Hi On Tue, Oct 1, 2013 at 2:30 PM, Dimitris Zervas <dzervas@dzervas.gr> wrote:
Um, I have a very powerful pc, so I get no slow downs. i7-3820 with 16GB of ram.
So if your system works fine then I am not sure what you are complaining about. On a bit more serious note. The threading in Linux kernel is quite effective. A thread data takes just a few kibibytes of non-swappable kernel memory. And 200 threads is nothing for modern server/desktop machines. I saw servers with 30K threads that were doing absolutely fine. And by the way - a of the kworkers are per-CPU so more (virtual) CPU cores you have more kworkers threads you will see. The problem with threading is not the number of all threads (when most of the threads are sleeping and doing nothing), but *active* jumping from one thread to another. Such behavior happens in networking services where a thread starts, does some small amount of job then get blocked because it doing a disk or database access, switches to another task that does also a little of work and switches back. Such behavior causes cache thrashing, TLB cache flushes and thus poor performance. Some people try to fight with this problem by avoiding thread switches using some kind of callback frameworks (a-la node.js) or user-space threads (a-la goroutines in Go). As I said if kernel threads are not active then they almost free for your system. And there is a good reason why kernel started using kworkers more actively. Imagine you write a kernel code that need a delayed execution of two independent functions A() and B(). You can start one thread and call A() and B() serially, but in this case if A() is blocked then B() will not run until A() is done. If you start two threads, then when A() will be blocked, CPU can switch execution to B(). Thus your tasks will finish faster and the system becomes more responsive. And most server/desktop users choose overall system responsiveness even if it takes a little bit more memory.
htop reports no CPU usage, but free -m reports some interesting memory consumption. free -m total used free shared buffers cached Mem: 16030 15868 161 0 150 13831 -/+ buffers/cache: 1886 14143 Swap: 0 0 0
I know that "used" is for caching/etc., but 15GB is a bit too much. Downgrading the kernel is not an option because the kernel is patched and downgrading would take just too much. I know have 234 kworkers, without suspending.
This cache usage has noting to do with number of kworkers. This amount is used by buffer cache. Buffer cache contains data read from slow devices like disk to avoid reading it again (this is what cache for). You can drop the cache by sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
Ok, thank you for your answers! It seems that I should dive into the linux kernel and learn more about it. Arch community proven very helpful once again, thank you very much.
participants (6)
-
Anatol Pomozov
-
Dimitris Zervas
-
Karol Blazewicz
-
LANGLOIS Olivier PIS -EXT
-
Leonid Isaev
-
Pedro Emílio Machado de Brito