[arch-general] most efficient way to get linux kernel statistics
hello archers, thought i ask here first, before i try some kernel mailing list. i'm writing (for fun) a tiny daemon that sends linux usage (cpu, mem, net, etc.) statistics as efficiently as possible to another machine, repeatedly, to get "live" data. at the moment i'm simply sending /proc files, but they sometimes have too much data. so was wondering if there is a more efficient way to get (only parts of) the data thats available in /proc? for example, /proc/meminfo has all this info: MemTotal: 8051660 kB MemFree: 6046020 kB MemAvailable: 6843080 kB Buffers: 109116 kB Cached: 708336 kB SwapCached: 0 kB Active: 1332416 kB Inactive: 455428 kB Active(anon): 971040 kB Inactive(anon): 22340 kB Active(file): 361376 kB Inactive(file): 433088 kB Unevictable: 16 kB Mlocked: 16 kB SwapTotal: 1048572 kB SwapFree: 1048572 kB Dirty: 16 kB Writeback: 0 kB AnonPages: 970412 kB Mapped: 323384 kB Shmem: 23008 kB Slab: 71760 kB SReclaimable: 44860 kB SUnreclaim: 26900 kB KernelStack: 6544 kB PageTables: 22924 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 5074400 kB Committed_AS: 3487604 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 413696 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 168600 kB DirectMap2M: 8091648 kB but i only need: MemTotal: 8051660 kB MemAvailable: 6843080 kB Cached: 708336 kB event better would be if kernel would only tell me: 8051660 6843080 708336 if /proc is the only way to get this info, i wonder if creating a kernel module for this would be more efficient, or even possible? Cheers Andre Schmidt
On Sun, Mar 06, 2016 at 03:23:33PM +0100, Andre Schmidt wrote:
hello archers,
thought i ask here first, before i try some kernel mailing list.
i'm writing (for fun) a tiny daemon that sends linux usage (cpu, mem, net, etc.) statistics as efficiently as possible to another machine, repeatedly, to get "live" data.
at the moment i'm simply sending /proc files, but they sometimes have too much data. so was wondering if there is a more efficient way to get (only parts of) the data thats available in /proc?
for example, /proc/meminfo has all this info:
MemTotal: 8051660 kB MemFree: 6046020 kB MemAvailable: 6843080 kB Buffers: 109116 kB Cached: 708336 kB SwapCached: 0 kB Active: 1332416 kB Inactive: 455428 kB Active(anon): 971040 kB Inactive(anon): 22340 kB Active(file): 361376 kB Inactive(file): 433088 kB Unevictable: 16 kB Mlocked: 16 kB SwapTotal: 1048572 kB SwapFree: 1048572 kB Dirty: 16 kB Writeback: 0 kB AnonPages: 970412 kB Mapped: 323384 kB Shmem: 23008 kB Slab: 71760 kB SReclaimable: 44860 kB SUnreclaim: 26900 kB KernelStack: 6544 kB PageTables: 22924 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 5074400 kB Committed_AS: 3487604 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 413696 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 168600 kB DirectMap2M: 8091648 kB
but i only need:
MemTotal: 8051660 kB MemAvailable: 6843080 kB Cached: 708336 kB
event better would be if kernel would only tell me:
8051660 6843080 708336
if /proc is the only way to get this info, i wonder if creating a kernel module for this would be more efficient, or even possible?
Hello Andre, According to what you need, you may want to use the 'free' command instead, as its output is much simpler than the output of the /proc/meminfo file E.g.
$ free total used free shared buff/cache available Mem: 5951844 486324 4969788 75812 495732 5339492 Swap: 0 0 0
or, if you prefer it be displayed in mebibytes,
$ free -m total used free shared buff/cache available Mem: 5812 496 4832 73 484 5193 Swap: 0 0 0
you can also check the man page for more display options Hope this helps, Jonathan
On 03/06/2016 04:47 PM, Jonathan Horacio Villatoro Córdoba wrote:
On Sun, Mar 06, 2016 at 03:23:33PM +0100, Andre Schmidt wrote:
hello archers,
thought i ask here first, before i try some kernel mailing list.
i'm writing (for fun) a tiny daemon that sends linux usage (cpu, mem, net, etc.) statistics as efficiently as possible to another machine, repeatedly, to get "live" data.
at the moment i'm simply sending /proc files, but they sometimes have too much data. so was wondering if there is a more efficient way to get (only parts of) the data thats available in /proc?
for example, /proc/meminfo has all this info:
[…]
if /proc is the only way to get this info, i wonder if creating a kernel module for this would be more efficient, or even possible?
Hello Andre,
According to what you need, you may want to use the 'free' command instead, as its output is much simpler than the output of the /proc/meminfo file
[…]
Hello, The free command gets its information from /proc/meminfo. Performance-wise, it doesn't really matter if a few additional lines need to be parsed.
On Sun, Mar 06, 2016 at 05:13:37PM +0100, Florian Pelz wrote:
On 03/06/2016 04:47 PM, Jonathan Horacio Villatoro Córdoba wrote:
On Sun, Mar 06, 2016 at 03:23:33PM +0100, Andre Schmidt wrote:
hello archers,
thought i ask here first, before i try some kernel mailing list.
i'm writing (for fun) a tiny daemon that sends linux usage (cpu, mem, net, etc.) statistics as efficiently as possible to another machine, repeatedly, to get "live" data.
at the moment i'm simply sending /proc files, but they sometimes have too much data. so was wondering if there is a more efficient way to get (only parts of) the data thats available in /proc?
for example, /proc/meminfo has all this info:
[…]
if /proc is the only way to get this info, i wonder if creating a kernel module for this would be more efficient, or even possible?
Hello Andre,
According to what you need, you may want to use the 'free' command instead, as its output is much simpler than the output of the /proc/meminfo file
[…]
Hello,
The free command gets its information from /proc/meminfo. Performance-wise, it doesn't really matter if a few additional lines need to be parsed.
Hello, Thank you Florian. Actually, I already knew it's the same, I just thought that he could use the free command instead of parsing it himself. You're right. Performance-wise, it's pretty much the same.
On 03/07/2016 04:36 PM, Jonathan Horacio Villatoro Córdoba wrote:
On Sun, Mar 06, 2016 at 05:13:37PM +0100, Florian Pelz wrote:
On 03/06/2016 04:47 PM, Jonathan Horacio Villatoro Córdoba wrote:
On Sun, Mar 06, 2016 at 03:23:33PM +0100, Andre Schmidt wrote:
hello archers,
thought i ask here first, before i try some kernel mailing list.
i'm writing (for fun) a tiny daemon that sends linux usage (cpu, mem, net, etc.) statistics as efficiently as possible to another machine, repeatedly, to get "live" data.
at the moment i'm simply sending /proc files, but they sometimes have too much data. so was wondering if there is a more efficient way to get (only parts of) the data thats available in /proc?
for example, /proc/meminfo has all this info:
[…]
if /proc is the only way to get this info, i wonder if creating a kernel module for this would be more efficient, or even possible?
Hello Andre,
According to what you need, you may want to use the 'free' command instead, as its output is much simpler than the output of the /proc/meminfo file
[…]
Hello,
The free command gets its information from /proc/meminfo. Performance-wise, it doesn't really matter if a few additional lines need to be parsed.
Hello, Thank you Florian. Actually, I already knew it's the same, I just thought that he could use the free command instead of parsing it himself. You're right. Performance-wise, it's pretty much the same.
Sorry, I didn't express myself properly. I didn't mean to criticize free. What I meant to say was that the time it takes to parse /proc/meminfo or free is negligible. It doesn't need to be any more efficient. free parses /proc/meminfo. libgtop parses /proc/meminfo. Querying the information probably takes much more time than parsing it. Your suggestion to use free may indeed be useful if the original poster wants parsing to be slightly simpler and does not mind the additional dependency. My impression was that they considered parsing to be too inefficient.
The free command gets its information from /proc/meminfo. Performance-wise, it doesn't really matter if a few additional lines need to be parsed.
Hello, Thank you Florian. Actually, I already knew it's the same, I just thought that he could use the free command instead of parsing it himself. You're right. Performance-wise, it's pretty much the same.
Sorry, I didn't express myself properly. I didn't mean to criticize free. What I meant to say was that the time it takes to parse /proc/meminfo or free is negligible. It doesn't need to be any more efficient. free parses /proc/meminfo. libgtop parses /proc/meminfo. Querying the information probably takes much more time than parsing it.
Your suggestion to use free may indeed be useful if the original poster wants parsing to be slightly simpler and does not mind the additional dependency. My impression was that they considered parsing to be too inefficient.
there's also `sysinfo(2)` -- damjan
On 03/07/2016 09:20 PM, Damjan Georgievski wrote:
there's also `sysinfo(2)`
I didn't know there's a syscall for that. That seems like the easiest way to get memory data. I'm not sure if you can get the other information the original poster wanted without parsing /proc though. On 02/19/2016 04:01 AM, Jonathan Horacio Villatoro Córdoba wrote:
Don't worry. Actually, I understood your point perfectly. I suggested free only because I originally supposed the OP wanted his program to contain as little code as possible in his program, and didn't want to parse the file himself.
OK :) .
On Mon, 7 Mar 2016 21:20:18 +0100
Damjan Georgievski
The free command gets its information from /proc/meminfo. Performance-wise, it doesn't really matter if a few additional lines need to be parsed.
Hello, Thank you Florian. Actually, I already knew it's the same, I just thought that he could use the free command instead of parsing it himself. You're right. Performance-wise, it's pretty much the same.
Sorry, I didn't express myself properly. I didn't mean to criticize free. What I meant to say was that the time it takes to parse /proc/meminfo or free is negligible. It doesn't need to be any more efficient. free parses /proc/meminfo. libgtop parses /proc/meminfo. Querying the information probably takes much more time than parsing it.
Your suggestion to use free may indeed be useful if the original poster wants parsing to be slightly simpler and does not mind the additional dependency. My impression was that they considered parsing to be too inefficient.
there's also `sysinfo(2)`
ah, thank you! and now that i found (or rather, thought about it) the source for /proc/meminfo [0] i can see that it uses sysinfo too. but sadly the cached size is not in sysinfo. then i tried to do what /proc/stat [1] does and after googling learned some facts: - you can't use "kernel-space" functions in "user-space" apps - so i cant use all the same functions as /proc/stat uses - you can't create new "syscall" functions in a kernel module - so i can't "export/map" those "kernel-space" functions to be used in my "user-space" app so it seems the only way to get deeper than /proc/stat would be to add new "syscalls" to the linux kernel? (for functions like for_each_possible_cpu and kcpustat_cpu) i'm just a webmonkey, so ill stick to /proc for now :D (i cant notice my daemon in cpu usage yet, so not really a "problem") Cheers Andre Schmidt [0] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/fs... [1] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/fs...
On Mon, Mar 07, 2016 at 09:15:06PM +0100, Florian Pelz wrote:
Sorry, I didn't express myself properly. I didn't mean to criticize free. What I meant to say was that the time it takes to parse /proc/meminfo or free is negligible. It doesn't need to be any more efficient. free parses /proc/meminfo. libgtop parses /proc/meminfo. Querying the information probably takes much more time than parsing it.
Your suggestion to use free may indeed be useful if the original poster wants parsing to be slightly simpler and does not mind the additional dependency. My impression was that they considered parsing to be too inefficient.
Don't worry. Actually, I understood your point perfectly. I suggested free only because I originally supposed the OP wanted his program to contain as little code as possible in his program, and didn't want to parse the file himself.
participants (4)
-
Andre Schmidt
-
Damjan Georgievski
-
Florian Pelz
-
Jonathan Horacio Villatoro Córdoba