[arch-general] Almost system crash, users getting logged out
Hi list, I had an "almost crash" last night. When I got up in the morning, the session on the first six TTYs were closed, but the sessions on TTY 8-10 were still open. Having taken a look at the journal, I found a lot of activity around 4AM. That may have been preceeded by something around midnight, when my music player suddenly quit, the system informing me, that there was not enough memory. A restart of the music player was successful though. I don't have any remote login services active and this sort of thing hasn't happened before, not on Arch since my installation a few weeks ago nor on the previous system. My kernel version is 4.8.2, not having rebooted since the last upgrade. The journal excerpt with the potential problem is here: https://freeshell.de/~silvain/journal.log I'd be grateful for any help and of course eagre to supply any further info to solve this issue. Thank you and best wishes Jeanette -------- When you need someone, you just turn around and I will be there <3
On Sat, Nov 5, 2016, 10:45 J. C. <julien@mail.upb.de> wrote:
Hi list, I had an "almost crash" last night. When I got up in the morning, the session on the first six TTYs were closed, but the sessions on TTY 8-10 were still open. Having taken a look at the journal, I found a lot of activity around 4AM. That may have been preceeded by something around midnight, when my music player suddenly quit, the system informing me, that there was not enough memory. A restart of the music player was successful though.
I don't have any remote login services active and this sort of thing hasn't happened before, not on Arch since my installation a few weeks ago nor on the previous system.
My kernel version is 4.8.2, not having rebooted since the last upgrade.
The journal excerpt with the potential problem is here: https://freeshell.de/~silvain/journal.log
I'd be grateful for any help and of course eagre to supply any further info to solve this issue.
Thank you and
best wishes
Jeanette
-------- When you need someone, you just turn around and I will be there <3
Do you have any tmpfs mounted that's larger than half your RAM? Your system repeatedly ran out of memory, but I can't spot any directly responsible process, so my next thought is that the memory was consumed by files on one or more tmpfs mounts.
Nov 5 2016, Jan Alexander Steffens via arch-general has written: ...
Do you have any tmpfs mounted that's larger than half your RAM? Your system repeatedly ran out of memory, but I can't spot any directly responsible process, so my next thought is that the memory was consumed by files on one or more tmpfs mounts. Hi Jan, it seems that my /tmp folder is mounted as tmpfs with 1.7G and there's another 1.7G mount on /dev/shm and 1.7G on /sys/fs/cgroup and a smaller 330M tmpfs on /run/user/1000 and one more for user 0.
That does seem a lot. None of that is part of my /etc/fstab, so the next question would be: how to change these mounts? Besides, I can't understand which job could have used up so much memory, I was asleep at the time and the regular cron job was finished at 4:01AM. Would you have an idea for further investigations on the particulars? Best wishes, Jeanette -------- When you need someone, you just turn around and I will be there <3
On Sat, 5 Nov 2016 12:09:35 +0100 (CET), J. C. wrote:
1.7G [snip] That does seem a lot.
Hi Jeanette, no, it isn't, you're most likely fooled by the confusing output of df. Assuming you're still using fstab, then as long as there isn't an explicit uncommented entry in fstab, tmpfs takes around half of the available memory. In your case just one time 1,7G for tmpfs as a whole. I suspect you have 4 GiB of RAM and a [rocketmouse@archlinux ~]$ df -h|head -1;df -h|grep tmpfs Filesystem Size Used Avail Use% Mounted on tmpfs 1.9G 4.0K 1.9G 1% /dev/shm tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup tmpfs 1.9G 8.0K 1.9G 1% /tmp tmpfs 371M 32K 371M 1% /run/user/1000 [rocketmouse@archlinux ~]$ grep tmpfs /etc/fstab #tmpfs /tmp tmpfs nodev,nosuid,size=3G 0 0 [rocketmouse@archlinux ~]$ hwinfo --memory | grep Size Memory Size: 3 GB + 512 MB [rocketmouse@archlinux ~]$ uname -m x86_64 An issue I experience since a very long time is, that I'm missing 256 MiB, since the frame buffer size should be 256 MiB, due to BIOS settings, so I expect 3 GiB + 768 MiB. I guess the missing 256 MiB aren't reserved for the kernel, since the output of hwinfo years ago didn't made such arbitrary calculations.
None of that is part of my /etc/fstab, so the next question would be:
Btw. it's possible to directly use systemd to mount, without keeping an fstab. Regards, Ralf
On Sat, 5 Nov 2016 13:21:14 +0100, Ralf Mardorf wrote:
I suspect you have 4 GiB of RAM and a
Oops ... "and a" much to large frame buffer for an on-board graphics, given that you don't use a display at all.
[rocketmouse@archlinux ~]$ df -h|head -1;df -h|grep tmpfs Filesystem Size Used Avail Use% Mounted on tmpfs 1.9G 4.0K 1.9G 1% /dev/shm tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup tmpfs 1.9G 8.0K 1.9G 1% /tmp tmpfs 371M 32K 371M 1% /run/user/1000 This looks similar with two extra messages: df: /run/user/1000/gvfs: Transport endpoint is not connected
Nov 5 2016, Ralf Mardorf has written: ... printed once above the table heading and once below it. ...
[rocketmouse@archlinux ~]$ hwinfo --memory | grep Size Memory Size: 3 GB + 512 MB 3 GB + 256 MB Probably the graphic memory is much to large. Perchance any idea on how to change it? :) I've got an nvidia card, using the Nouveau driver.
[rocketmouse@archlinux ~]$ uname -m x86_64 i686 on mine, which is to be expected. :) ... Btw. it's possible to directly use systemd to mount, without keeping an fstab. Is it recommended? ...
-------- When you need someone, you just turn around and I will be there <3
On Sat, 5 Nov 2016 13:33:01 +0100 (CET), J. C. wrote:
df: /run/user/1000/gvfs: Transport endpoint is not connected printed once above the table heading and once below it.
I don't google for you :D. This seems to be worth to google.
Probably the graphic memory is much to large. Perchance any idea on how to change it? :) I've got an nvidia card, using the Nouveau driver.
The BIOS settings provide the option for graphics that don't have their own memory. It might be tricky for you, if Braille shouldn't work in the BIOS settings dialogue.
Btw. it's possible to directly use systemd to mount, without keeping an fstab. Is it recommended?
I only read the first 5 posts: https://bbs.archlinux.org/viewtopic.php?id=153131 This thread is from 2012, so sane people should stay with fstab. I only read freedesktop pages, if it's unavoidable, so I didn't read those pages: https://www.freedesktop.org/software/systemd/man/systemd.mount.html https://www.freedesktop.org/software/systemd/man/systemd-fstab-generator.htm... Regards, Ralf
On Sat, 5 Nov 2016 13:33:01 +0100 (CET), J. C. wrote:
gvfs
OT: I'm gvfs allergic. [root@archlinux moonstudio]# pacman -Qi gvfs | grep Description Description : Dummy package [root@archlinux moonstudio]# systemd-nspawn -q dpkg -l gvfs | grep ii ii gvfs 2016:07-13-moonstudio all Dummy package One reason for this allergy is the fact that gvfs wakes up green drives for no reason, so they spin down and up again and again and ... as soon as gvfs is removed and no other culprit, such as smartd is running, the green drives stay asleep. Most, if not all apps with a hard dependency to gfvs don't require gvfs, IOW they could make it an optional dependency.
participants (3)
-
J. C.
-
Jan Alexander Steffens
-
Ralf Mardorf