[arch-general] Fedora Speeds Up Python 3.
Hi, I thought this might be of interest. https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup By building with -fno-semantic-interposition they remove the PLT that provides a level of indirection when calling a libpython function. libpython often calls itself and the PLT adds L1-cache pressure plus prevents inlining. Gives gains of 25% on some workloads. -- Cheers, Ralph.
Ah, that's interesting. I thought that would break my GIL profiling project that uses LD_PRELOAD (shameless plug: https://github.com/chrisjbillington/gil_load), but since I think I'm only overriding libc functions, it should be fine. I'm sure there are other things it will break (I could have overridden libpython functions instead - I wonder why I didn't, it seems simpler but there was probably a reason), but if someone wants to use code that hacks on the interpreter itself, having to install a custom python to do so is not so unreasonable. Those speedups are nothing to scoff at. -Chris On Tue, Nov 26, 2019 at 5:07 AM Ralph Corderoy <ralph@inputplus.co.uk> wrote:
Hi,
I thought this might be of interest. https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup
By building with -fno-semantic-interposition they remove the PLT that provides a level of indirection when calling a libpython function. libpython often calls itself and the PLT adds L1-cache pressure plus prevents inlining. Gives gains of 25% on some workloads.
-- Cheers, Ralph.
Oh I remember why I didn't override libpython functions with LD_PRELOAD. Python is statically linked to libpython in Ubuntu, which I was using at the time, so it didn't work. Fedora decided not to go that far, but yeah, Ubuntu has already taken that plunge. If LD_PRELOADing libpython doesn't work in Ubuntu, then there probably aren't many projects using it. On Tue, Nov 26, 2019 at 9:31 AM Chris Billington <chrisjbillington@gmail.com> wrote:
Ah, that's interesting.
I thought that would break my GIL profiling project that uses LD_PRELOAD (shameless plug: https://github.com/chrisjbillington/gil_load), but since I think I'm only overriding libc functions, it should be fine.
I'm sure there are other things it will break (I could have overridden libpython functions instead - I wonder why I didn't, it seems simpler but there was probably a reason), but if someone wants to use code that hacks on the interpreter itself, having to install a custom python to do so is not so unreasonable. Those speedups are nothing to scoff at.
-Chris
On Tue, Nov 26, 2019 at 5:07 AM Ralph Corderoy <ralph@inputplus.co.uk> wrote:
Hi,
I thought this might be of interest.
https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup
By building with -fno-semantic-interposition they remove the PLT that provides a level of indirection when calling a libpython function. libpython often calls itself and the PLT adds L1-cache pressure plus prevents inlining. Gives gains of 25% on some workloads.
-- Cheers, Ralph.
From: Ralph Corderoy <ralph@inputplus.co.uk> Sent: Tue Nov 26 11:07:36 CET 2019 To: <arch-general@archlinux.org> Subject: [arch-general] Fedora Speeds Up Python 3.
Hi,
I thought this might be of interest. https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup
By building with -fno-semantic-interposition they remove the PLT that provides a level of indirection when calling a libpython function. libpython often calls itself and the PLT adds L1-cache pressure plus prevents inlining. Gives gains of 25% on some workloads.
-- Cheers, Ralph.
Does it work alongside "-fno-plt" flag that Arch uses? Yours sincerely G. K.
participants (3)
-
Chris Billington
-
Geo Kozey
-
Ralph Corderoy