[arch-dev-public] Fun with LTO and stripping
With the latest devtools, LTO is enabled by default. This causes an issue with .a and .o archived when stripping. Look out for output like:
strip: ./usr/lib/st4RPjCb/libsyslog_ng_native_connector_a-native-grammar.o: plugin needed to handle lto object
That file is now nicely mangled!
Turns out this is a very well known issue in other distros, and has not been fixed in binutils since reported in 2017.
There are a couple of workarounds:
1) disable LTO for that package 2) add -ffat-lto-objects to the C{,XX}FLAGS for those packages
I can see how to fix this in makepkg, but that will take a few days to push out. In the meantime, pay attention to your build output!
Cheers, Allan
On 24/12/21 00:03, Allan McRae via arch-dev-public wrote:
With the latest devtools, LTO is enabled by default. This causes an issue with .a and .o archived when stripping. Look out for output like:
strip: ./usr/lib/st4RPjCb/libsyslog_ng_native_connector_a-native-grammar.o: plugin needed to handle lto object
That file is now nicely mangled!
Turns out this is a very well known issue in other distros, and has not been fixed in binutils since reported in 2017.
There are a couple of workarounds:
- disable LTO for that package
- add -ffat-lto-objects to the C{,XX}FLAGS for those packages
I can see how to fix this in makepkg, but that will take a few days to push out. In the meantime, pay attention to your build output!
I have discovered two things about LTO today:
1) the strip issue is issue really hard to detect in order to work around in makepkg... There are too many variables. 2) clang requires -flto in LDFLAGS for LTO to work, but GCC does not.
This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.
I will patch makepkg to add -flto to LDFLAGS when lto is enabled. It has no effect for gcc, and fixes clang. Until a new pacman package appears, I'd suggest packages compiling with clang either add the LDFLAG in the PKGBUILD or disable LTO.
Allan
Allan
On Fri, 24 Dec 2021 at 10:01, Allan McRae via arch-dev-public arch-dev-public@lists.archlinux.org wrote:
This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.
It's worth noting that -ffat-lto-objects is not supported by clang. I'm not sure if there's an equivalent option (seems unlikely).
Packages built with clang and -flto appear to have unusable static libraries as the object files are LLVM bitcode. Perhaps GCC can link these archives with -Wl,-plugin,LLVMgold.so but this is not a solution for official packages.
tl;dr: If a package in the "LTO fat objects" todo is compiled with clang, it'll need to have LTO disabled with options=('!lto'), unless a better solution is found.
Allan McRae via arch-dev-public arch-dev-public@lists.archlinux.org on Fri, 2021/12/24 18:01:
This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.
Do you want us to commit changes in PKGBUILD only or should we push rebuilt packages?
On Sat, Dec 25, 2021, 22:01 Christian Hesse via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:
Allan McRae via arch-dev-public arch-dev-public@lists.archlinux.org on Fri, 2021/12/24 18:01:
This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.
Do you want us to commit changes in PKGBUILD only or should we push rebuilt packages?
I would build the packages so that we notice problems now and not after it gets combined with an unrelated change.
"Jan Alexander Steffens (heftig)" heftig@archlinux.org on Sat, 2021/12/25 22:07:
On Sat, Dec 25, 2021, 22:01 Christian Hesse via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:
Allan McRae via arch-dev-public arch-dev-public@lists.archlinux.org on Fri, 2021/12/24 18:01:
This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.
Do you want us to commit changes in PKGBUILD only or should we push rebuilt packages?
I would build the packages so that we notice problems now and not after it gets combined with an unrelated change.
Probably makes sense. :)
Just an additional headsup as we are on the topic of "LTO fun"; it seems like any Go packages utilizing cgo/the external linker seems to be just broken with LTO.
The error looks like: net._cgo_3c1cec0c9a4e_C2func_getaddrinfo: relocation target _cgo_3c1cec0c9a4e_C2func_getaddrinfo not defined
Please do just disable LTO if you spot this issue in your builds.
On Sat, Dec 25, 2021 at 11:51:49AM +0100, Morten Linderud via arch-dev-public wrote:
Just an additional headsup as we are on the topic of "LTO fun"; it seems like any Go packages utilizing cgo/the external linker seems to be just broken with LTO.
The error looks like: net._cgo_3c1cec0c9a4e_C2func_getaddrinfo: relocation target _cgo_3c1cec0c9a4e_C2func_getaddrinfo not defined
Please do just disable LTO if you spot this issue in your builds.
-- Morten Linderud PGP: 9C02FF419FECBE16
For anyone having trouble to do this: Set the `options` field in the PKGBUILD. For example:
options=('!lto')
Merry Christmas,
Chris
participants (6)
-
Allan McRae
-
Christian Hesse
-
Christian Rebischke
-
Evangelos Foutras
-
Jan Alexander Steffens (heftig)
-
Morten Linderud