[arch-general] valgrind - memory exclusion files on the fritz?
All, valgrind is reporting an additional allocation and free along with and additional 1M of memory used. Below, 11 bytes should be allocated and freed. Instead valgrind reports 1035 bytes. (it's worse if you use strdup). For example: #include <stdio.h> #include <stdlib.h> #include <string.h> enum { MAXC = 16 }; int main (void) { char buf[] = "0123456789"; size_t len = strlen (buf); char *s = NULL; s = malloc (len + 1); snprintf (s, len + 1, "%s", buf); printf (" s : %s\n", s); free (s); return 0; } The report from valgrind: valgrind ./bin/valgrindchk ==3917== Memcheck, a memory error detector ==3917== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==3917== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info ==3917== Command: ./bin/valgrindchk ==3917== s : 0123456789 ==3917== ==3917== HEAP SUMMARY: ==3917== in use at exit: 0 bytes in 0 blocks ==3917== total heap usage: 2 allocs, 2 frees, 1,035 bytes allocated ==3917== ==3917== All heap blocks were freed -- no leaks are possible ==3917== ==3917== For counts of detected and suppressed errors, rerun with: -v ==3917== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Anybody else experience this problem? Anybody got a fix? I don't find anything valgrind related with a cursory search, did a patch get missed with the latest build? -- David C. Rankin, J.D.,P.E.
On Wed, Nov 23, 2016 at 2:26 PM, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
All,
valgrind is reporting an additional allocation and free along with and additional 1M of memory used. Below, 11 bytes should be allocated and freed. Instead valgrind reports 1035 bytes. (it's worse if you use strdup). For example:
#include <stdio.h> #include <stdlib.h> #include <string.h>
enum { MAXC = 16 };
int main (void) {
char buf[] = "0123456789"; size_t len = strlen (buf); char *s = NULL;
s = malloc (len + 1); snprintf (s, len + 1, "%s", buf);
printf (" s : %s\n", s);
free (s);
return 0; }
The report from valgrind:
valgrind ./bin/valgrindchk ==3917== Memcheck, a memory error detector ==3917== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==3917== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info ==3917== Command: ./bin/valgrindchk ==3917== s : 0123456789 ==3917== ==3917== HEAP SUMMARY: ==3917== in use at exit: 0 bytes in 0 blocks ==3917== total heap usage: 2 allocs, 2 frees, 1,035 bytes allocated ==3917== ==3917== All heap blocks were freed -- no leaks are possible ==3917== ==3917== For counts of detected and suppressed errors, rerun with: -v ==3917== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Anybody else experience this problem? Anybody got a fix? I don't find anything valgrind related with a cursory search, did a patch get missed with the latest build?
-- David C. Rankin, J.D.,P.E.
In glibc, printf calls malloc. See http://stackoverflow.com/questions/6743034/does-fprintf-use-malloc-under-the...
On 11/23/2016 12:31 AM, Chi-Hsuan Yen via arch-general wrote:
In glibc, printf calls malloc. See http://stackoverflow.com/questions/6743034/does-fprintf-use-malloc-under-the...
Yes, That's not the point. There is generally a know exclusion file that covers the memory used by fprintf, etc.. The reporting of the allocation and free by fprintf (if that is indeed the case here) should not be reported by valgrind. The same code run on 4.8.7-1-ARCH #1 SMP PREEMPT Thu Nov 10 17:22:48 CET 2016 x86_64 GNU/Linux and on SuSE: valgrind ./bin/valgrindchk ==1974== Memcheck, a memory error detector ==1974== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==1974== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info ==1974== Command: ./bin/valgrindchk ==1974== s : 0123456789 ==1974== ==1974== HEAP SUMMARY: ==1974== in use at exit: 0 bytes in 0 blocks ==1974== total heap usage: 1 allocs, 1 frees, 11 bytes allocated ==1974== ==1974== All heap blocks were freed -- no leaks are possible ==1974== ==1974== For counts of detected and suppressed errors, rerun with: -v ==1974== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1) Something is wrong with the Arch package. -- David C. Rankin, J.D.,P.E.
On Wed, Nov 23, 2016 at 2:46 PM, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
On 11/23/2016 12:31 AM, Chi-Hsuan Yen via arch-general wrote:
In glibc, printf calls malloc. See http://stackoverflow.com/questions/6743034/does-fprintf-use-malloc-under-the...
Yes,
That's not the point. There is generally a know exclusion file that covers the memory used by fprintf, etc.. The reporting of the allocation and free by fprintf (if that is indeed the case here) should not be reported by valgrind.
The same code run on 4.8.7-1-ARCH #1 SMP PREEMPT Thu Nov 10 17:22:48 CET 2016 x86_64 GNU/Linux and on SuSE:
valgrind ./bin/valgrindchk ==1974== Memcheck, a memory error detector ==1974== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==1974== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info ==1974== Command: ./bin/valgrindchk ==1974== s : 0123456789 ==1974== ==1974== HEAP SUMMARY: ==1974== in use at exit: 0 bytes in 0 blocks ==1974== total heap usage: 1 allocs, 1 frees, 11 bytes allocated ==1974== ==1974== All heap blocks were freed -- no leaks are possible ==1974== ==1974== For counts of detected and suppressed errors, rerun with: -v ==1974== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1)
Something is wrong with the Arch package.
-- David C. Rankin, J.D.,P.E.
As far as I know it's impossible (yet) to skip specific functions in memory usage counters. Only memory leak reports can be filtered. And things may be different on SuSE and Arch as they ship with different glibc versions and/or patches. Best, Yen, Chi-Hsuan
On 11/23/2016 08:49 AM, Chi-Hsuan Yen via arch-general wrote:
As far as I know it's impossible (yet) to skip specific functions in memory usage counters. Only memory leak reports can be filtered. And things may be different on SuSE and Arch as they ship with different glibc versions and/or patches.
Best,
Yen, Chi-Hsuan
I have a couple of old arch boxes I cranked up just to test: $ uname -a Linux providence 3.5.3-1-ARCH #1 SMP PREEMPT Sun Aug 26 08:15:06 UTC 2012 i686 GNU/Linux $ valgrind ./bin/valgrindchk ==7060== Memcheck, a memory error detector ==7060== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==7060== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==7060== Command: ./bin/valgrindchk ==7060== s : 0123456789 ==7060== ==7060== HEAP SUMMARY: ==7060== in use at exit: 0 bytes in 0 blocks ==7060== total heap usage: 1 allocs, 1 frees, 11 bytes allocated ==7060== ==7060== All heap blocks were freed -- no leaks are possible ==7060== ==7060== For counts of detected and suppressed errors, rerun with: -v ==7060== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) and a second old arch box: $ uname -a Linux nirvana 3.4.6-1-ARCH #1 SMP PREEMPT Fri Jul 20 08:21:26 CEST 2012 x86_64 GNU/Linux $ valgrind ./bin/valgrindchk ==3481== Memcheck, a memory error detector ==3481== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==3481== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==3481== Command: ./bin/valgrindchk ==3481== s : 0123456789 ==3481== ==3481== HEAP SUMMARY: ==3481== in use at exit: 0 bytes in 0 blocks ==3481== total heap usage: 1 allocs, 1 frees, 11 bytes allocated ==3481== ==3481== All heap blocks were freed -- no leaks are possible ==3481== ==3481== For counts of detected and suppressed errors, rerun with: -v ==3481== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1) Both boxes show the correct valgrind behavior. So sometime (I don't know when), something has changed in the way valgrind is packaged or patched. I have some old ABS backups I'll take a look at and see if anything is apparent. The whole reason this jumped out at me is I know this used to work correctly on Arch which incorporated the proper exclusion files up to at least valgrind 3.10-0 as shown above. -- David C. Rankin, J.D.,P.E.
On Thu, Nov 24, 2016 at 8:45 AM, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
On 11/23/2016 08:49 AM, Chi-Hsuan Yen via arch-general wrote:
As far as I know it's impossible (yet) to skip specific functions in memory usage counters. Only memory leak reports can be filtered. And things may be different on SuSE and Arch as they ship with different glibc versions and/or patches.
Best,
Yen, Chi-Hsuan
I have a couple of old arch boxes I cranked up just to test:
$ uname -a Linux providence 3.5.3-1-ARCH #1 SMP PREEMPT Sun Aug 26 08:15:06 UTC 2012 i686 GNU/Linux
$ valgrind ./bin/valgrindchk ==7060== Memcheck, a memory error detector ==7060== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==7060== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==7060== Command: ./bin/valgrindchk ==7060== s : 0123456789 ==7060== ==7060== HEAP SUMMARY: ==7060== in use at exit: 0 bytes in 0 blocks ==7060== total heap usage: 1 allocs, 1 frees, 11 bytes allocated ==7060== ==7060== All heap blocks were freed -- no leaks are possible ==7060== ==7060== For counts of detected and suppressed errors, rerun with: -v ==7060== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
and a second old arch box:
$ uname -a Linux nirvana 3.4.6-1-ARCH #1 SMP PREEMPT Fri Jul 20 08:21:26 CEST 2012 x86_64 GNU/Linux
$ valgrind ./bin/valgrindchk ==3481== Memcheck, a memory error detector ==3481== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==3481== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==3481== Command: ./bin/valgrindchk ==3481== s : 0123456789 ==3481== ==3481== HEAP SUMMARY: ==3481== in use at exit: 0 bytes in 0 blocks ==3481== total heap usage: 1 allocs, 1 frees, 11 bytes allocated ==3481== ==3481== All heap blocks were freed -- no leaks are possible ==3481== ==3481== For counts of detected and suppressed errors, rerun with: -v ==3481== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1)
Both boxes show the correct valgrind behavior. So sometime (I don't know when), something has changed in the way valgrind is packaged or patched. I have some old ABS backups I'll take a look at and see if anything is apparent.
The whole reason this jumped out at me is I know this used to work correctly on Arch which incorporated the proper exclusion files up to at least valgrind 3.10-0 as shown above.
-- David C. Rankin, J.D.,P.E.
OK I found the cause of the extra malloc() call. Since glibc 2.23 printf uses malloc instead of mmap to allocate file buffers. If you want to keep the old behavior, keep the old glibc. See https://sourceware.org/bugzilla/show_bug.cgi?id=16734 Best, Yen, Chi-Hsuan
On 11/24/2016 09:02 AM, Chi-Hsuan Yen via arch-general wrote:
OK I found the cause of the extra malloc() call. Since glibc 2.23 printf uses malloc instead of mmap to allocate file buffers. If you want to keep the old behavior, keep the old glibc.
See https://sourceware.org/bugzilla/show_bug.cgi?id=16734
Best,
Yen, Chi-Hsuan
Ah Hah! Thank you! That's makes sense, so it is the glibc 2.23 change from mmap to malloc that does it. I suspect that valgrind will be fixed to exclude the printf allocation shortly. Keeping 2.22 isn't worth it just for the old valgrind behavior. I'll check with the valgrind folks and see if that is in the works. -- David C. Rankin, J.D.,P.E.
participants (2)
-
Chi-Hsuan Yen
-
David C. Rankin