[pacman-dev] [PATCH] makepkg: optionally attach minimal debuginfo to binaries
The "minidebug" option adds a .gdb_debugdata section to executables and shared libraries containing LZMA compressed function symbols. GDB (and by extension systemd-coredump) can use this information to display useful backtraces without full debuginfo being installed. (see https://sourceware.org/gdb/onlinedocs/gdb/MiniDebugInfo.html) This benefit comes at the cost of slightly increased binary sizes. (https://fedoraproject.org/wiki/Features/MiniDebugInfo#Detailed_Description) Signed-off-by: Leo Wolf <jclw@jclw.de> --- doc/makepkg.conf.5.txt | 10 +++++++ scripts/libmakepkg/tidy/strip.sh.in | 60 +++++++++++++++++++++++++++++++++---- scripts/makepkg.sh.in | 2 +- 3 files changed, 66 insertions(+), 6 deletions(-) diff --git a/doc/makepkg.conf.5.txt b/doc/makepkg.conf.5.txt index e35eb8a..b86fd04 100644 --- a/doc/makepkg.conf.5.txt +++ b/doc/makepkg.conf.5.txt @@ -191,6 +191,16 @@ Options DEBUG_CXXFLAGS to their counterpart buildflags. Creates a separate package containing the debug symbols when used with `strip'. + *minidebug*;; + Enable useful backtraces at the cost of slightly larger binaries by + adding a section containing compressed function symbols to stripped + executables and shared libraries. (see + https://sourceware.org/gdb/onlinedocs/gdb/MiniDebugInfo.html) Like + `debug', this option will add the user-specified debug flags as + specified in DEBUG_CFLAGS and DEBUG_CXXFLAGS to their counterpart + buildflags. `minidebug' is only useful with `strip', but can be used + with or without `debug'. + **INTEGRITY_CHECK=(**check1 ...**)**:: File integrity checks to use. Multiple checks may be specified; this affects both generation and checking. The current valid options are: diff --git a/scripts/libmakepkg/tidy/strip.sh.in b/scripts/libmakepkg/tidy/strip.sh.in index 1c7aacf..184502d 100644 --- a/scripts/libmakepkg/tidy/strip.sh.in +++ b/scripts/libmakepkg/tidy/strip.sh.in @@ -37,6 +37,7 @@ build_id() { strip_file() { local binary=$1; shift + local minidebug_applicable=$1; shift if check_option "debug" "y"; then local bid=$(build_id "$binary") @@ -76,7 +77,51 @@ strip_file() { fi fi + # Generate minidebuginfo if requested + if check_option "minidebug" "y" && [[ $minidebug_applicable == "y" ]]; then + # make sure not to operate on the same file twice + if ! check_option "debug" "y" && objdump -h "$binary" | grep -q .gnu_debugdata; then + return; + fi + if ! check_option "debug" "y"; then + local debug=$(mktemp) + objcopy --only-keep-debug "$binary" "$debug" + else + local debug="$dbgdir/$binary.debug" + fi + add_minidebug "$binary" "$debug" + if ! check_option "debug" "y"; then + rm -f "$debug" + fi + fi + strip $@ "$binary" + +} + + +# This function is based on the one written for use in RPM by Alexander Larsson. +# (https://bugzilla.redhat.com/attachment.cgi?id=594198) +add_minidebug() { + local dynsyms=$(mktemp) + local funcsyms=$(mktemp) + local keep_symbols=$(mktemp) + # Extract the dynamic symbols from the main binary, there is no need to also have these + LC_ALL=C nm -D "$1" --format=posix --defined-only | awk '{ print $1 }' | sort > "$dynsyms" + # Extract all function symbols (see https://bugzilla.redhat.com/show_bug.cgi?id=1052415#c11) + LC_ALL=C nm "$1" --format=sysv --defined-only | awk -F \| '{ if ($4 ~ "FUNC") print $1 }' | sort > "$funcsyms" + # Keep all the function symbols not already in the dynamic symbol table + LC_ALL=C comm -13 "$dynsyms" "$funcsyms" > "$keep_symbols" + if [[ -s "$keep_symbols" ]]; then + local minidebug=$(mktemp) + # Copy the full debuginfo, keeping only a minimal set of symbols and removing some unnecessary sections. + objcopy --strip-all --remove-section .gdb_index --remove-section .comment --keep-symbols="$keep_symbols" \ + "$2" "$minidebug" + xz "$minidebug" + objcopy --add-section .gnu_debugdata="$minidebug.xz" "$binary" + rm -f "$minidebug.xz" + fi + rm -f "$dynsyms" "$funcsyms" "$keep_symbols" } @@ -93,25 +138,30 @@ tidy_strip() { fi local binary strip_flags + local minidebug_applicable # MiniDebugInfo is only useful with shared libraries and executables find . -type f -perm -u+w -print0 2>/dev/null | while read -rd '' binary ; do case "$(file -bi "$binary")" in *application/x-sharedlib*) # Libraries (.so) - strip_flags="$STRIP_SHARED";; + strip_flags="$STRIP_SHARED" + minidebug_applicable="y";; *application/x-archive*) # Libraries (.a) - strip_flags="$STRIP_STATIC";; + strip_flags="$STRIP_STATIC" + minidebug_applicable="n";; *application/x-object*) case "$binary" in *.ko) # Kernel module - strip_flags="$STRIP_SHARED";; + strip_flags="$STRIP_SHARED" + minidebug_applicable="n";; *) continue;; esac;; *application/x-executable*) # Binaries - strip_flags="$STRIP_BINARIES";; + strip_flags="$STRIP_BINARIES" + minidebug_applicable="y";; *) continue ;; esac - strip_file "$binary" ${strip_flags} + strip_file "$binary" $minidebug_applicable ${strip_flags} done fi } diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 80b95f4..b6195b5 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -791,7 +791,7 @@ run_function() { unset CPPFLAGS CFLAGS CXXFLAGS LDFLAGS fi - if check_option "debug" "y"; then + if check_option "debug" "y" || check_option "minidebug" "y"; then CFLAGS+=" $DEBUG_CFLAGS" CXXFLAGS+=" $DEBUG_CXXFLAGS" fi -- 2.7.0
Hi, Enabling the option provided by this patch when building official packages would make it possible to get useful backtraces without the hassle of rebuilding the affected package with debug symbols enabled. I'm sure this would make some upstreams quite happy. Fedora has had this since Fedora 18 (January 2013). Going by [1], the increase in binary size should vary between 1-2% for normal packages and up to 7% for complex packages like firefox. Because of compression, the relative increase in package size would be larger than that. I don't have numbers for this right now. Using LC_ALL=C for the sort, awk and comm operations seemed like a good idea, but I don't really know. I wanted to make sure this feature works even when not generating separate -debug packages (which probably won't get mirrored any time soon), hence the code surrounded by 'if ! check_option "debug"'. Cheers, Leo
Going by [1]… This was supposed to be the link to the Fedora Wiki already referenced in the commit message: https://fedoraproject.org/wiki/Features/MiniDebugInfo#Detailed_Description
Sorry about that.
==> 0x00: Summary I think this looks kind of useful, minidebug enabled binaries could help debugging and understanding certain type of crashes without the need of rebuilding and install full debug symbols. Overall the size increase seems to be round about 3% per binary and slightly higher per tarball (mainly depending on the amount of binaries it contains). The mplayer testcase had some strange effects that may need some deeper investigation before considering to make this a default option. Binaries with minidebug option and a corresponding .gnu_debugdata section do not seem to affect reproducibility. ==> 0x01: Patch adjustments There are some small adjustments required to this patch, I would recommend the following changes: 1) scripts/libmakepkg/tidy/strip.sh.in The new minidebug option is required to be added to the packaging_options array to make it available in the PKGBUILD's options array: packaging_options+=('strip' 'debug' 'minidebug') 2) etc/makepkg.conf.in The new option should to be added to the OPTIONS array in the configuration file. Its default value should also to be added in the comment above that option, including a description text between both. ==> 0x02: Backtrace example Following backtrace example was taken from zsh with default and minidebug options. (gdb) bt #0 0x00007ffff71a0150 in __read_nocancel () from /usr/lib/libc.so.6 #1 0x000000000041c3a7 in bin_read () #2 0x000000000041d926 in execbuiltin () #3 0x000000000042b59a in ?? () #4 0x000000000042bf4e in ?? () #5 0x000000000042c311 in ?? () #6 0x000000000042db6c in execlist () #7 0x000000000045168b in execif () #8 0x000000000042a581 in ?? () #9 0x000000000042bf4e in ?? () #10 0x000000000042c311 in ?? () (gdb) bt #0 0x00007ffff71a0150 in __read_nocancel () from /usr/lib/libc.so.6 #1 0x000000000041c3a7 in bin_read () #2 0x000000000041d926 in execbuiltin () #3 0x000000000042b59a in execcmd () #4 0x000000000042bf4e in execpline2 () #5 0x000000000042c311 in execpline () #6 0x000000000042db6c in execlist () #7 0x000000000045168b in execif () #8 0x000000000042a581 in execcmd () #9 0x000000000042bf4e in execpline2 () #10 0x000000000042c311 in execpline () ==> 0x03: File size influence The file size list below roughly matches the expected size increase and ends up with around 3% bigger binaries. The percentage size increase per package is obviously higher compared to a single binaries because the debug sections are already compress. 1) package -> util-linux 8.70% 1.7M 1738096 1.9M 1889416 binary -> hexdump 3.71% 47K 47968 49K 49752 binary -> lsblk 3.26% 76K 77296 78K 79816 binary -> fdisk 2.74% 108K 109640 110K 112640 2) package -> bash 1.14% 1.3M 1271836 1.3M 1286376 binary -> bash 1.86% 773K 791320 788K 806080 3) package -> ncmpcpp 13.75% 610K 624140 694K 710592 binary -> ncmpcpp 3.41% 2.4M 2447816 2.5M 2531440 4) package -> zsh 2.81% 1.8M 1883400 1.9M 1936428 binary -> zsh 1.47% 758K 775592 769K 786968 5) package -> wireshark-gtk 4.49% 641K 655976 670K 685452 binary -> wireshark-gtk 1.48% 2.0M 2068552 2.1M 2099256 6) package -> wireshark-qt 3.25% 2.9M 2983856 3.0M 3080928 binary -> wireshark 1.52% 6.1M 6375200 6.2M 6472216 ==> 0x04: Strange observations Besides the package firefox, where the minidebug option had no effect at all, there was some strange effect while testing the mplayer package. The mplayer and memcoder binaries significantly increase in size while having missing .gnu_debugdata sections. As there may be more cases affected by this issue, it should maybe be investigated before considering to enable this by default. 1) mplayer 26.95% 17M 17333304 21M 22005264 2) mencoder 25.96% 16M 16105880 20M 20286256 cheers, Levente
On 16/02/16 11:42, Levente Polyak wrote:
==> 0x00: Summary
I think this looks kind of useful, minidebug enabled binaries could help debugging and understanding certain type of crashes without the need of rebuilding and install full debug symbols.
Thanks for taking a look at this. <snip>
==> 0x04: Strange observations
Besides the package firefox, where the minidebug option had no effect at all, there was some strange effect while testing the mplayer package.
The mplayer and memcoder binaries significantly increase in size while having missing .gnu_debugdata sections. As there may be more cases affected by this issue, it should maybe be investigated before considering to enable this by default.
1) mplayer 26.95% 17M 17333304 21M 22005264
2) mencoder 25.96% 16M 16105880 20M 20286256
Whether this option is enabled or not by a distribution (including Arch) is not an issue to be discussed here. What worries me about this is that this is a sign of a bug in the implementation (particularly the lack of a .gnu_debugdata section). I'm going to need this case investigate before this gets pulled into makepkg. A
participants (3)
-
Allan McRae
-
Leo Wolf
-
Levente Polyak