[arch-general] Limiting pacman's IO rate
Dear archers, I am running Arch Linux inside virtualbox, on a Ubuntu 12.04 host on which I have very limited power (the only command I can run as root is "apt-get install"). For this reason my virtualbox version is very outdated (4.1.12, the last version being 4.3.28!). My main problem is than when I ask to run "pacman -S ghc", the installation begins and then the machine suddenly halts at approximately 25% (best case 50%), with the following error log message: 00:02:06.072 !!Assertion Failed!! 00:02:06.072 Expression: pSgBuf->cbSegLeft <= 5 * _1M && (uintptr_t)pSgBuf->pvSegCur >= (uintptr_t)pSgBuf->paSegs[pSgBuf->idxSeg].pvSeg && (uintptr_t)pSgBuf->pvSegCur + pSgBuf->cbSegLeft <= (uintptr_t)pSgBuf->paSegs[pSgBuf->idxSeg].pvSeg + pSgBuf->paSegs[pSgBuf->idxSeg].cbSeg 00:02:06.072 Location : /build/buildd/virtualbox-4.1.12-dfsg/src/VBox/Runtime/common/misc/sg.cpp(54) void* sgBufGet(PRTSGBUF, size_t*) 00:02:06.072 pSgBuf->idxSeg=0 pSgBuf->cSegs=1 pSgBuf->pvSegCur=00007fbd109e0000 pSgBuf->cbSegLeft=10321920 pSgBuf->paSegs[0].pvSeg=00007fbd109e0000 pSgBuf->paSegs[0].cbSeg=10321920 Following this ticket <https://www.virtualbox.org/ticket/10434> and ghc being a huge package, I am assuming that the problem is the same: this virtualbox bug is triggered because pacman performs too heavy IO when installing ghc (speculative, but looks reasonable). So, I tried to limit pacman's IO rate in the following way (after having installed libcgroup from the AUR and cgmanager): # cgcreate -g blkio:wlimit # echo "8:0 1024"
/sys/fs/cgroup/blkio/wlimit/blkio.throttle.write_bps_device # cgexec -g blkio:wlimit pacman -S --force ghc
(8:0 is /dev/sda, which is the only disk, and 1024 the byte rate I want to enforce) However the machine still halts in the same way and doesn't seem to go beyond 50% either. Even if I replace 1024 by, say, 1, pacman doesn't seem to struggle more to write to the disk -- when I would expect it to take forever to do 1% of the installation. Is there something wrong in the commands I have written above? Is there a way to check that the limitation is indeed enforced? (maybe the source of the bug is elsewhere) Alternatively, can you think of another solution to this problem? Many thanks -- Yannis
On Thu, May 21, 2015 at 3:58 PM, Yannis Juglaret <yjuglaret@gmail.com> wrote:
Dear archers,
Is there something wrong in the commands I have written above? Is there a way to check that the limitation is indeed enforced? (maybe the source of the bug is elsewhere) Alternatively, can you think of another solution to this problem?
No idea about your commands, but I have a few alternative solutions... let's see if any one works. Solution #1. 1. Run `pacman -S ghc` normally. 2. When it advances about 10%, hopefully without crashing, press Ctrl+Z to sleep the process. 3. Wait a bit until the cache memory stabilizes, then run `sync` to force the remaining cached data to disk. 4. Type `fg` to continue the instalation, 5. Go to step 2 and repeat until finished. Solution #2. 1. Run `pacman -S -p ghc`. That will dump the URL of the package but it will do nothing else. 2. In the host OS (Ubuntu) get the package with `wget <url>` or whatever download command you prefer. 3. Copy the just downloaded file to the guest OS (Arch). You can use rsync, in case the copy gets interrupted. 4. Install the program with `pacman -U <file>`. HTH
Hello, Thanks for the hints! Solution #1 looks great. As for solution #2, I don't have any problem downloading the ghc package (which is "only" 71.7 MB) but only installing it (which requires 738.2 MB), so my guess is that pacman -U would trigger the same problem, but mixing #1 and #2 could be useful for bigger packages (if any). However, I also have great news on my side. If I enable the "Use host IO caching" option, VirtualBox stops trying to manage the disk IO cache itself, and everything works just fine. So that's probably where the bug was located. Problem solved! For the record, I have a guess regarding why my commands didn't seem to affect pacman: blkio will only limit direct IO rate, not cached IO, so unless the program you execute performs direct IO you can't notice the change. Thanks, -- Yannis Le 22/05/2015 09:38, Rodrigo Rivas a écrit :
On Thu, May 21, 2015 at 3:58 PM, Yannis Juglaret <yjuglaret@gmail.com> wrote:
Dear archers,
Is there something wrong in the commands I have written above? Is there a way to check that the limitation is indeed enforced? (maybe the source of the bug is elsewhere) Alternatively, can you think of another solution to this problem?
No idea about your commands, but I have a few alternative solutions... let's see if any one works.
Solution #1.
1. Run `pacman -S ghc` normally. 2. When it advances about 10%, hopefully without crashing, press Ctrl+Z to sleep the process. 3. Wait a bit until the cache memory stabilizes, then run `sync` to force the remaining cached data to disk. 4. Type `fg` to continue the instalation, 5. Go to step 2 and repeat until finished.
Solution #2.
1. Run `pacman -S -p ghc`. That will dump the URL of the package but it will do nothing else. 2. In the host OS (Ubuntu) get the package with `wget <url>` or whatever download command you prefer. 3. Copy the just downloaded file to the guest OS (Arch). You can use rsync, in case the copy gets interrupted. 4. Install the program with `pacman -U <file>`.
HTH
participants (2)
-
Rodrigo Rivas
-
Yannis Juglaret