[arch-dev-public] Fun with LTO and stripping - Arch-dev-public - lists.archlinux.org

newer
[arch-dev-public] RFC Final...

[arch-dev-public] Fun with LTO and stripping

older
[arch-dev-public] News draft:...

Allan McRae

23 Dec 2021 23 Dec '21

3:03 p.m.

With the latest devtools, LTO is enabled by default. This causes an issue with .a and .o archived when stripping. Look out for output like: strip: ./usr/lib/st4RPjCb/libsyslog_ng_native_connector_a-native-grammar.o: plugin needed to handle lto object That file is now nicely mangled! Turns out this is a very well known issue in other distros, and has not been fixed in binutils since reported in 2017. There are a couple of workarounds: 1) disable LTO for that package 2) add -ffat-lto-objects to the C{,XX}FLAGS for those packages I can see how to fix this in makepkg, but that will take a few days to push out. In the meantime, pay attention to your build output! Cheers, Allan

Show replies by date

Allan McRae

24 Dec 24 Dec

9:01 a.m.

On 24/12/21 00:03, Allan McRae via arch-dev-public wrote:

With the latest devtools, LTO is enabled by default. This causes an issue with .a and .o archived when stripping. Look out for output like:

strip: ./usr/lib/st4RPjCb/libsyslog_ng_native_connector_a-native-grammar.o: plugin needed to handle lto object

That file is now nicely mangled!

Turns out this is a very well known issue in other distros, and has not been fixed in binutils since reported in 2017.

There are a couple of workarounds:

1) disable LTO for that package 2) add -ffat-lto-objects to the C{,XX}FLAGS for those packages

I can see how to fix this in makepkg, but that will take a few days to push out. In the meantime, pay attention to your build output!

I have discovered two things about LTO today: 1) the strip issue is issue really hard to detect in order to work around in makepkg... There are too many variables. 2) clang requires -flto in LDFLAGS for LTO to work, but GCC does not. This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list. I will patch makepkg to add -flto to LDFLAGS when lto is enabled. It has no effect for gcc, and fixes clang. Until a new pacman package appears, I'd suggest packages compiling with clang either add the LDFLAG in the PKGBUILD or disable LTO. Allan Allan

Evangelos Foutras

8:23 p.m.

On Fri, 24 Dec 2021 at 10:01, Allan McRae via arch-dev-public wrote:

This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.

It's worth noting that -ffat-lto-objects is not supported by clang. I'm not sure if there's an equivalent option (seems unlikely). Packages built with clang and -flto appear to have unusable static libraries as the object files are LLVM bitcode. Perhaps GCC can link these archives with -Wl,-plugin,LLVMgold.so but this is not a solution for official packages. tl;dr: If a package in the "LTO fat objects" todo is compiled with clang, it'll need to have LTO disabled with options=('!lto'), unless a better solution is found.

Christian Hesse

25 Dec 25 Dec

10:01 p.m.

Allan McRae via arch-dev-public on Fri, 2021/12/24 18:01:

This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.

Do you want us to commit changes in PKGBUILD only or should we push rebuilt packages? -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}

Jan Alexander Steffens (heftig)

10:07 p.m.

On Sat, Dec 25, 2021, 22:01 Christian Hesse via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:

Allan McRae via arch-dev-public on Fri, 2021/12/24 18:01:

...
This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.

Do you want us to commit changes in PKGBUILD only or should we push rebuilt packages?

I would build the packages so that we notice problems now and not after it gets combined with an unrelated change.

Christian Hesse

11:18 p.m.

"Jan Alexander Steffens (heftig)" on Sat, 2021/12/25 22:07:

On Sat, Dec 25, 2021, 22:01 Christian Hesse via arch-dev-public < arch-dev-public@lists.archlinux.org> wrote:

...
Allan McRae via arch-dev-public on Fri, 2021/12/24 18:01:

...
This fix for the strip issue is for people to add CFLAGS+=" -ffat-lto-objects" to their PKGBUILDs if they use LTO and contain a .a or .o archive. This affects ~300 packages in our repos (~2.5%). I will create a TODO list.

Do you want us to commit changes in PKGBUILD only or should we push rebuilt packages?

I would build the packages so that we notice problems now and not after it gets combined with an unrelated change.

Probably makes sense. :) -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}

Morten Linderud

11:51 a.m.

Just an additional headsup as we are on the topic of "LTO fun"; it seems like any Go packages utilizing cgo/the external linker seems to be just broken with LTO. The error looks like: net._cgo_3c1cec0c9a4e_C2func_getaddrinfo: relocation target _cgo_3c1cec0c9a4e_C2func_getaddrinfo not defined Please do just disable LTO if you spot this issue in your builds. -- Morten Linderud PGP: 9C02FF419FECBE16

Christian Rebischke

12:50 p.m.

On Sat, Dec 25, 2021 at 11:51:49AM +0100, Morten Linderud via arch-dev-public wrote:

Just an additional headsup as we are on the topic of "LTO fun"; it seems like any Go packages utilizing cgo/the external linker seems to be just broken with LTO.

The error looks like: net._cgo_3c1cec0c9a4e_C2func_getaddrinfo: relocation target _cgo_3c1cec0c9a4e_C2func_getaddrinfo not defined

Please do just disable LTO if you spot this issue in your builds.

-- Morten Linderud PGP: 9C02FF419FECBE16

For anyone having trouble to do this: Set the `options` field in the PKGBUILD. For example: options=('!lto') Merry Christmas, Chris

851

Age (days ago)

853

Last active (days ago)

Download

7 comments

6 participants

tags

participants (6)

Allan McRae
Christian Hesse
Christian Rebischke
Evangelos Foutras
Jan Alexander Steffens (heftig)
Morten Linderud