[arch-general] KDE plasma, baloo and UI/video playback freezes
Hi, My problem is: when I'm watching a video, running Konversation in the background with logging on disk enabled, the playback of video slows down, freezes for a while. Running iotop and ksysguard I found that baloo_file_extractor is writing on disk a big amount of data, filling the write queue; balooct monitor shows baloo is indexing Konversation's logs file changes. Peaks of 60-90 MB/s of data on mechanical hdd are generated and the playback of videos (or audio, or other GUI-related tasks) slow down or freeze. Very very annoying. I'm running updated versions of: KDE plasma (5.12.5), kernel (4.16.7), baloo (5.45); CFQ I/O scheduler on sda. On the same machine, the same version of softwares, the same configurations (of programs, mounts, IO scheduler) BUT on a Debian sid install (home is shared between installs) is NOT showing the same issue. On Debian, playback is running smootly even if baloo_file_extractor is heavly loading IO writes queue. On each installation I added this tweaks, to improve Vm management by the kernel: [frapox@tungsteno ~]$ cat /etc/sysctl.d/local.conf vm.dirty_background_ratio = 2 vm.dirty_ratio = 5 vm.vfs_cache_pressure = 60 I also tried to recompile the Arch kernel with CONFIG_PREEMPT_VOLUNTARY=y, to reflect the same default compile setting of Debian, but the issue wasn't solved. So i'm wondering which other config files or settings/tweaks I can look into to overcome this issue on Arch linux (I'd really love to keep on use Arch as my main OS). Thanks in advance. -- fp
On 5/8/18, Francesco Porro via arch-general <arch-general@archlinux.org> wrote:
Hi,
My problem is: when I'm watching a video, running Konversation in the background with logging on disk enabled, the playback of video slows down, freezes for a while.
Running iotop and ksysguard I found that baloo_file_extractor is writing on disk a big amount of data, filling the write queue; balooct monitor shows baloo is indexing Konversation's logs file changes. Peaks of 60-90 MB/s of data on mechanical hdd are generated and the playback of videos (or audio, or other GUI-related tasks) slow down or freeze. Very very annoying.
I'm running updated versions of: KDE plasma (5.12.5), kernel (4.16.7), baloo (5.45); CFQ I/O scheduler on sda.
On the same machine, the same version of softwares, the same configurations (of programs, mounts, IO scheduler) BUT on a Debian sid install (home is shared between installs) is NOT showing the same issue. On Debian, playback is running smootly even if baloo_file_extractor is heavly loading IO writes queue.
On each installation I added this tweaks, to improve Vm management by the kernel:
[frapox@tungsteno ~]$ cat /etc/sysctl.d/local.conf vm.dirty_background_ratio = 2 vm.dirty_ratio = 5 vm.vfs_cache_pressure = 60
I also tried to recompile the Arch kernel with CONFIG_PREEMPT_VOLUNTARY=y, to reflect the same default compile setting of Debian, but the issue wasn't solved.
So i'm wondering which other config files or settings/tweaks I can look into to overcome this issue on Arch linux (I'd really love to keep on use Arch as my main OS).
Linux block layer's writeback system was supposed to fix this, but I've also noticed that the mechanism isn't perfect and you can still have a "hanging" application when doing the infamous USB-to-USB transfer that kills the VM subsystem. Another way I can reproduce it is when there SSD-to-thumb-drive and you decide to some disk activity, too. https://lwn.net/Articles/682582/ The problem is that VM gets pressured a lot and the whole construct fails in a way, while working as designed, manifesting as hanging programs. First, to confirm, if you manage to run the indexer like you would `ionice -c idle <the-indexer>`, and it shows less hangs, you know the issue is unfair I/O queuing. You can compare the block layer kernel configuration of Debian vs Arch. You can try deadline or bfq schedulers. One is dead simple and the other optimizes for desktop responsiveness.
In data martedì 8 maggio 2018 21:08:35 CEST, Carsten Mattner ha scritto:
Linux block layer's writeback system was supposed to fix this, but I've also noticed that the mechanism isn't perfect and you can still have a "hanging" application when doing the infamous USB-to-USB transfer that kills the VM subsystem.
Another way I can reproduce it is when there SSD-to-thumb-drive and you decide to some disk activity, too.
https://lwn.net/Articles/682582/
The problem is that VM gets pressured a lot and the whole construct fails in a way, while working as designed, manifesting as hanging programs.
Ok, but I'm talking about HDD on Sata bus. No Usb-to-usb transfers involved. And, as said before, I've altready tweaked the vm.dirty{writeback,background_ratio} to partialy word around this, either on Debian or Arch linux.
First, to confirm, if you manage to run the indexer like you would `ionice -c idle <the-indexer>`, and it shows less hangs, you know the issue is unfair I/O queuing.
The indexer's process autostarts and rapidly kills himself in a bunch of seconds... its impossible to renice the process, however I saw (by Ksysguard) it's always 19 as nice value... meaning high fairness both for the CPU, and I think for the IO too.
You can compare the block layer kernel configuration of Debian vs Arch.
Sorry, how to? Which are the keyword for searching?
You can try deadline or bfq schedulers. One is dead simple and the other optimizes for desktop responsiveness.
As a last chance, I'll look into these alternatives schedulers. But now, either Debian or Arch are using the same one: Cfq. So I don't think this is the cause of the problem. -- fp
On 5/8/18, Francesco Porro <frapox@gmail.com> wrote:
In data martedì 8 maggio 2018 21:08:35 CEST, Carsten Mattner ha scritto:
Linux block layer's writeback system was supposed to fix this, but I've also noticed that the mechanism isn't perfect and you can still have a "hanging" application when doing the infamous USB-to-USB transfer that kills the VM subsystem.
Another way I can reproduce it is when there SSD-to-thumb-drive and you decide to some disk activity, too.
https://lwn.net/Articles/682582/
The problem is that VM gets pressured a lot and the whole construct fails in a way, while working as designed, manifesting as hanging programs.
Ok, but I'm talking about HDD on Sata bus. No Usb-to-usb transfers involved. And, as said before, I've altready tweaked the vm.dirty{writeback,background_ratio} to partialy word around this, either on Debian or Arch linux.
It's easier to trigger with slow devices but ultimately the same issue of block layer and vm subsystem getting overwhelmed.
First, to confirm, if you manage to run the indexer like you would `ionice -c idle <the-indexer>`, and it shows less hangs, you know the issue is unfair I/O queuing.
The indexer's process autostarts and rapidly kills himself in a bunch of seconds... its impossible to renice the process, however I saw (by Ksysguard) it's always 19 as nice value... meaning high fairness both for the CPU, and I think for the IO too.
You can compare the block layer kernel configuration of Debian vs Arch.
Sorry, how to? Which are the keyword for searching?
On both systems there should be /proc/config.gz of the running kernel. If you have a suspicion which kernel option is responsible, you can check if it's different between Arch and Debian.
You can try deadline or bfq schedulers. One is dead simple and the other optimizes for desktop responsiveness.
As a last chance, I'll look into these alternatives schedulers. But now, either Debian or Arch are using the same one: Cfq. So I don't think this is the cause of the problem.
Sounds true. Is it only that one application or others as well when they cause much I/O?
So, I finally tried the BFQ scheduler as Carsten suggested. I tested it eg. using youtube-dl to download a big video from Vimeo (it creates/remove quickly a lot of temp files that baloo indexes every time causing a lot of IO), while Qbittorrent was downloading several big files... So 100% disk activity going on (monitored with iotop). Overloading the disk queue this way, with BFQ enabled, the result was that the video playback of a 1080p movie by VLC was smooth and didn't hang/freeze at all. I can assume BFQ was able to handle better all the stuff going on. So thanks Carsten for the suggestion :) This fixed (or at least worked around) the problem. It remains to understand why on Debian and derivs. this issue doesn't exist. Maybe the CONFIG_PREEMPT_VOLUNTARY=y set by default on kernel's config or some other block device related configs I don't have the skill (and the time) to inspect now. Cheers -- fp
participants (2)
-
Carsten Mattner
-
Francesco Porro