[arch-general] virtual memory exhausted
I'm not sure this is an Arch issue, but I can't figure out what else can be. I'm trying to compile non-Arch stuff in an arch host. I tried Binutils (of LFS) and Buildroot (of uclibc). Both failed with the "virtual memory exhausted: Cannot allocate memory" error. The computer is a one-user workstation, which wasn't doing anything else worth mentioning... I tried both as root and as common user, same problem... # pacman -Q gcc gcc 4.6.1-2 # cat /proc/meminfo MemTotal: 2056484 kB MemFree: 1356260 kB Buffers: 74324 kB Cached: 551824 kB SwapCached: 0 kB Active: 148888 kB Inactive: 506268 kB Active(anon): 10624 kB Inactive(anon): 5716 kB Active(file): 138264 kB Inactive(file): 500552 kB Unevictable: 0 kB Mlocked: 0 kB HighTotal: 1178888 kB HighFree: 593112 kB LowTotal: 877596 kB LowFree: 763148 kB SwapTotal: 996024 kB SwapFree: 996024 kBDirty: 84 kB Writeback: 0 kB AnonPages: 10592 kB Mapped: 5868 kB Shmem: 5748 kB Slab: 31360 kB SReclaimable: 25156 kB SUnreclaim: 6204 kB KernelStack: 1112 kB PageTables: 1476 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 2024264 kB Committed_AS: 61648 kB VmallocTotal: 122880 kB VmallocUsed: 14672 kB VmallocChunk: 80164 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 4096 kB DirectMap4k: 20472 kB DirectMap4M: 888832 kB # ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) 58593 scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 16046 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 58593 cpu time (seconds, -t) unlimited max user processes (-u) 16046 virtual memory (kbytes, -v) 58593 file locks (-x) unlimited Maybe there is some arch-specific configuring to do? I have no idea whether these limit parameters require tweaking... TIA Jorge Almeida
On Fri, Jul 29, 2011 at 11:28 AM, Jorge Almeida <jjalmeida@gmail.com> wrote:
I'm not sure this is an Arch issue, but I can't figure out what else can be.
I'm trying to compile non-Arch stuff in an arch host. I tried Binutils (of LFS) and Buildroot (of uclibc). Both failed with the "virtual memory exhausted: Cannot allocate memory" error. The computer is a one-user workstation, which wasn't doing anything else worth mentioning... I tried both as root and as common user, same problem...
# pacman -Q gcc gcc 4.6.1-2
# cat /proc/meminfo MemTotal: 2056484 kB MemFree: 1356260 kB Buffers: 74324 kB Cached: 551824 kB SwapCached: 0 kB Active: 148888 kB Inactive: 506268 kB Active(anon): 10624 kB Inactive(anon): 5716 kB Active(file): 138264 kB Inactive(file): 500552 kB Unevictable: 0 kB Mlocked: 0 kB HighTotal: 1178888 kB HighFree: 593112 kB LowTotal: 877596 kB LowFree: 763148 kB SwapTotal: 996024 kB SwapFree: 996024 kBDirty: 84 kB Writeback: 0 kB AnonPages: 10592 kB Mapped: 5868 kB Shmem: 5748 kB Slab: 31360 kB SReclaimable: 25156 kB SUnreclaim: 6204 kB KernelStack: 1112 kB PageTables: 1476 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 2024264 kB Committed_AS: 61648 kB VmallocTotal: 122880 kB VmallocUsed: 14672 kB VmallocChunk: 80164 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 4096 kB DirectMap4k: 20472 kB DirectMap4M: 888832 kB
# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) 58593 scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 16046 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 58593 cpu time (seconds, -t) unlimited max user processes (-u) 16046 virtual memory (kbytes, -v) 58593 file locks (-x) unlimited
Maybe there is some arch-specific configuring to do? I have no idea whether these limit parameters require tweaking...
TIA
Jorge Almeida
what exactly are you trying to compile.. maybe that's your problem -- "Breath Deeply and Dream"
On Fri, Jul 29, 2011 at 5:30 PM, Jeffrey Lynn Parke Jr. <jeffrey.parke@gmail.com> wrote:
what exactly are you trying to compile.. maybe that's your problem
As I said, Binutils (for building a Linux From Scratch system). But I also tried to compile Buildroot, which is totally unrelated with LFS. That's why I think the problem is not with the compiling stuff... J.A.
On Fri, Jul 29, 2011 at 1:28 PM, Jorge Almeida <jjalmeida@gmail.com> wrote:
I'm not sure this is an Arch issue, but I can't figure out what else can be.
I'm trying to compile non-Arch stuff in an arch host. I tried Binutils (of LFS) and Buildroot (of uclibc). Both failed with the "virtual memory exhausted: Cannot allocate memory" error. The computer is a one-user workstation, which wasn't doing anything else worth mentioning... I tried both as root and as common user, same problem...
# pacman -Q gcc gcc 4.6.1-2
# cat /proc/meminfo MemTotal: 2056484 kB MemFree: 1356260 kB Buffers: 74324 kB Cached: 551824 kB SwapCached: 0 kB Active: 148888 kB Inactive: 506268 kB Active(anon): 10624 kB Inactive(anon): 5716 kB Active(file): 138264 kB Inactive(file): 500552 kB Unevictable: 0 kB Mlocked: 0 kB HighTotal: 1178888 kB HighFree: 593112 kB LowTotal: 877596 kB LowFree: 763148 kB SwapTotal: 996024 kB SwapFree: 996024 kBDirty: 84 kB Writeback: 0 kB AnonPages: 10592 kB Mapped: 5868 kB Shmem: 5748 kB Slab: 31360 kB SReclaimable: 25156 kB SUnreclaim: 6204 kB KernelStack: 1112 kB PageTables: 1476 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 2024264 kB Committed_AS: 61648 kB VmallocTotal: 122880 kB VmallocUsed: 14672 kB VmallocChunk: 80164 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 4096 kB DirectMap4k: 20472 kB DirectMap4M: 888832 kB
# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) 58593 scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 16046 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 58593 cpu time (seconds, -t) unlimited max user processes (-u) 16046 virtual memory (kbytes, -v) 58593 file locks (-x) unlimited
Maybe there is some arch-specific configuring to do? I have no idea whether these limit parameters require tweaking...
TIA
Jorge Almeida
Add a few gigabytes of swap, if you can. If that doesn't work, try running x86_64, if you don't.
On Fri, Jul 29, 2011 at 10:21 PM, Jan Steffens <jan.steffens@gmail.com> wrote:
On Fri, Jul 29, 2011 at 1:28 PM, Jorge Almeida <jjalmeida@gmail.com> wrote:
I'm not sure this is an Arch issue, but I can't figure out what else can be.
I'm trying to compile non-Arch stuff in an arch host. I tried Binutils (of LFS) and Buildroot (of uclibc). Both failed with the "virtual memory exhausted: Cannot allocate memory" error. The computer is a one-user workstation, which wasn't doing anything else worth mentioning... I tried both as root and as common user, same problem...
Add a few gigabytes of swap, if you can. If that doesn't work, try running x86_64, if you don't.
I already found a solution. The problem is that the limits are too restrictive. There's probably a reason to have those values as default, but then nobody seems to understand this stuff that much, judging by Google... The solution is to issue, as root ulimit -u unlimited lfs ulimit -v unlimited lfs and then do the compiling as user lfs. By the way: this syntax (with the user as argument) seems to be completely undocumented! How hard could it be to add a few characters to the bash man page? (And finding that ulimit is a bash build-in is another matter, I don't remember how I found it.) I still don't know how to change the default... Cheers JA
On 07/29/11 at 05:28pm, Jorge Almeida wrote:
I'm not sure this is an Arch issue, but I can't figure out what else can be.
I'm trying to compile non-Arch stuff in an arch host. I tried Binutils (of LFS) and Buildroot (of uclibc). Both failed with the "virtual memory exhausted: Cannot allocate memory" error. The computer is a one-user workstation, which wasn't doing anything else worth mentioning... I tried both as root and as common user, same problem...
# ulimit -a virtual memory (kbytes, -v) 58593
You might need more than ~57MB of address space to run the compilation. -- Byron Clark
On Fri, Jul 29, 2011 at 10:46 PM, Byron Clark <byron@theclarkfamily.name> wrote:
On 07/29/11 at 05:28pm, Jorge Almeida wrote:
# ulimit -a virtual memory (kbytes, -v) 58593
You might need more than ~57MB of address space to run the compilation.
Yes, although I have no idea of what values would be right. Anyway, "unlimited" does it. Thanks JA
On 29-07-2011 22:50, Jorge Almeida wrote:
On Fri, Jul 29, 2011 at 10:46 PM, Byron Clark <byron@theclarkfamily.name> wrote:
On 07/29/11 at 05:28pm, Jorge Almeida wrote:
# ulimit -a virtual memory (kbytes, -v) 58593
You might need more than ~57MB of address space to run the compilation.
Yes, although I have no idea of what values would be right. Anyway, "unlimited" does it. Thanks
JA
Are you sure you haven't personalized that? 'ulimit -v' says unlimited here and I haven't changed that config from arch's default. 'ulimit -u unlimited' might be a bad idea, if anything goes haywire and starts spawning threads it can crash your machine very quickly (think fork bomb), even the default value seems to be too much for normal usage. -- Mauro Santos
On Fri, Jul 29, 2011 at 11:26 PM, Mauro Santos <registo.mailling@gmail.com> wrote:
Are you sure you haven't personalized that?
Yes. For a good reason: what little I knew about ulimit (from Gentoo) was forgotten, and it only came to mind now that I was bitten. 'ulimit -v' says unlimited
here and I haven't changed that config from arch's default.
OK, I have one system like you and another one with the limits I posted. I suppose it has to do with the time the systems were installed.
'ulimit -u unlimited' might be a bad idea, if anything goes haywire and starts spawning threads it can crash your machine very quickly (think fork bomb), even the default value seems to be too much for normal usage.
Agreed. But how is one supposed to know how to choose sane values? JA
On 29-07-2011 23:55, Jorge Almeida wrote:
Agreed. But how is one supposed to know how to choose sane values?
For the "max user processes" what I've seen is that you want to set it a little over the maximum number of tasks you expect to run (the number that top reports) so there is some protection from any program going rogue and starting to create new threads. I've seen people saying 256 to 512 is a good number (depends on the amount of programs and cruft you intend to have running), for a single user system, which most personal systems are, it should be enough. However on some distros this limit is sometimes even higher than it is on arch by default, on other distros it is somewhat smaller (but still way too big to really avoid even simple fork bombs I'd say) so either I got the wrong idea or no one looks into this, which is a shame because it seems some sensible ulimit defaults might help avoid some crashes. The only trouble with this stuff is that one size doesn't fit all and what may be sensible to some users might not be enough (or too much) for others. -- Mauro Santos
participants (5)
-
Byron Clark
-
Jan Steffens
-
Jeffrey Lynn Parke Jr.
-
Jorge Almeida
-
Mauro Santos