[arch-dev-public] Proposal: enabling full ASLR on x86_64 via hardening-wrapper
Arch's single biggest security weakness is that it's not benefiting from
address space layout randomization (ASLR). Fixing this issue would be a
major step towards being a leader in this area. Many distributions enable
ASLR, stack protector, etc. for a chosen set of "security critical" stuff
but very few enable them across the board.
Executable permissions on memory (NX bit) prevent an attacker from directly
injecting code via memory corruption vulnerabilities. It's only truly useful
in combination with ASLR because if the existing code is in a known location
it can be reused by the attacker.[1]
[1] https://en.wikipedia.org/wiki/Return-oriented_programming
The cost of ASLR is that it requires position independent (relocatable) code
(PIC), as is already required for all code in dynamic libraries.
On i686, PIC has a significant cost (up to 30% or more). The vanilla kernel
also lacks emulation of the NX bit and has no brute force protection to make
up for the smaller 32-bit address space. Only the subset of users using
grsecurity / PaX would truly benefit. I don't care about i686 anyway.
On x86_64, there's hardware support for position independent code so it's
essentially free. It does currently cost ~1% due to linker and compiler
limitations (indirection to access globals) but this has been fixed in the
binutils/gcc master branches. It's much cheaper than the
-fstack-protector-strong switch which we're already using, and the security
value is higher.
The only real barrier to enabling it is the lack of support in GCC for
simply flipping it on by default. Library code is already built with -fPIC
and is then linked with -shared. Full ASLR requires building the executable
code with -fPIE (or -fPIC, which isn't as cheap) and then linking with -pie.
There are two approaches to this:
1) Patching the toolchain's spec files (Hardened Gentoo)
2) Wrapper scripts for clang/gcc/ld.bfd/ld.gold (Debian, Fedora, Ubuntu)
Upstream hasn't accepted various forms of the first option, so I don't think
it's a suitable approach for us. The hardening-wrapper package implements
the second option and also enforces our existing hardening flags for build
systems ignoring CFLAGS/CXXFLAGS/LDFLAGS.
All it would take is either adding hardening-wrapper to the default devtools
packages list or making it part of base-devel.
Here's an example with a small C program:
strcat@thinktank i ~ master % cat test.c
#include
On 19 December 2014 at 00:31, Daniel Micay
Arch's single biggest security weakness is that it's not benefiting from address space layout randomization (ASLR). Fixing this issue would be a major step towards being a leader in this area. Many distributions enable ASLR, stack protector, etc. for a chosen set of "security critical" stuff but very few enable them across the board.
Executable permissions on memory (NX bit) prevent an attacker from directly injecting code via memory corruption vulnerabilities. It's only truly useful in combination with ASLR because if the existing code is in a known location it can be reused by the attacker.[1]
[1] https://en.wikipedia.org/wiki/Return-oriented_programming
The cost of ASLR is that it requires position independent (relocatable) code (PIC), as is already required for all code in dynamic libraries.
On i686, PIC has a significant cost (up to 30% or more). The vanilla kernel also lacks emulation of the NX bit and has no brute force protection to make up for the smaller 32-bit address space. Only the subset of users using grsecurity / PaX would truly benefit. I don't care about i686 anyway.
On x86_64, there's hardware support for position independent code so it's essentially free. It does currently cost ~1% due to linker and compiler limitations (indirection to access globals) but this has been fixed in the binutils/gcc master branches. It's much cheaper than the -fstack-protector-strong switch which we're already using, and the security value is higher.
The only real barrier to enabling it is the lack of support in GCC for simply flipping it on by default. Library code is already built with -fPIC and is then linked with -shared. Full ASLR requires building the executable code with -fPIE (or -fPIC, which isn't as cheap) and then linking with -pie. There are two approaches to this:
1) Patching the toolchain's spec files (Hardened Gentoo) 2) Wrapper scripts for clang/gcc/ld.bfd/ld.gold (Debian, Fedora, Ubuntu)
Upstream hasn't accepted various forms of the first option, so I don't think it's a suitable approach for us. The hardening-wrapper package implements the second option and also enforces our existing hardening flags for build systems ignoring CFLAGS/CXXFLAGS/LDFLAGS.
All it would take is either adding hardening-wrapper to the default devtools packages list or making it part of base-devel.
Here's an example with a small C program:
strcat@thinktank i ~ master % cat test.c #include
static void foo() {} static int bar = 5;
int main(void) { int baz = 5; printf("function: %p, data: %p, stack: %p\n", foo, &bar, &baz); return 0; }
Without the hardening-wrapper package installed:
strcat@thinktank i ~ master % gcc test.c -o test strcat@thinktank i ~ master % checksec --file test RELRO STACK CANARY NX PIE RPATH RUNPATH FILE No RELRO No canary found NX enabled No PIE No RPATH No RUNPATH test strcat@thinktank i ~ master % ./test function: 0x400506, data: 0x600980, stack: 0x7fff12ba3a7c strcat@thinktank i ~ master % ./test function: 0x400506, data: 0x600980, stack: 0x7fffa47958fc
With the hardening-wrapper package installed:
strcat@thinktank i ~ master % gcc test.c -o test strcat@thinktank i ~ master % checksec --file test RELRO STACK CANARY NX PIE RPATH RUNPATH FILE Partial RELRO Canary found NX enabled PIE enabled No RPATH No RUNPATH test strcat@thinktank i ~ master % ./test function: 0x7f267e569800, data: 0x7f267e76a050, stack: 0x7fff3ee5ea14 strcat@thinktank i ~ master % ./test function: 0x7f6a06b3e790, data: 0x7f6a06d3f048, stack: 0x7fff76a891bc
The defaults in /etc/hardening-wrapper.conf can be overridden via env variables:
strcat@thinktank i ~ master % HARDENING_STACK_PROTECTOR=0 gcc test.c -o test strcat@thinktank i ~ master % checksec --file test RELRO STACK CANARY NX PIE RPATH RUNPATH FILE Partial RELRO No canary found NX enabled PIE enabled No RPATH No RUNPATH test
The need to use wrapper scripts is a bit ugly, but I think it's a nicer approach than tracking down and patching dozens of build systems to respect CFLAGS/LDFLAGS and it's the only realistic way to enable PIE. The only alternative short of patching GCC spec files is manually specifying the flags in thousands of executable-only packages, and patching any build system producing both binaries and libraries (yikes!).
No matter how much I like the idea of making Arch more secure, there is one thing that makes compiling the whole system with ASLR one big no-go for me (please correct me if I'm wrong). As far as I know, the ASLR makes core dumps completely useless, and also makes it impossible to make any sense from addresses in backtrace (assume that you get a backtrace from an application without debugging symbols). I guess the same thing would happen with valgrind, too. I would be OK with building things from core with ASLR, as they should very stable, but not whole Arch. Lukas
On 19/12/14 03:50 AM, Lukas Jirkovsky wrote:
No matter how much I like the idea of making Arch more secure, there is one thing that makes compiling the whole system with ASLR one big no-go for me (please correct me if I'm wrong). As far as I know, the ASLR makes core dumps completely useless, and also makes it impossible to make any sense from addresses in backtrace (assume that you get a backtrace from an application without debugging symbols). I guess the same thing would happen with valgrind, too.
I would be OK with building things from core with ASLR, as they should very stable, but not whole Arch.
Lukas
The address of dynamic libraries, the stack and the heap (both sbrk and
the mmap base) is already randomized today so the backtrace is already
going to include randomized addresses for anything defined in a library.
PIE makes it possible for the executable base to be relocated, which
*also* randomizes the code and data defined in the executable and some
global data structures like the GOT / PLT. ASLR needs PIE to be a truly
useful exploit mitigation but it's still there without it.
An executable is compiled as PIE is compatible with full ASLR but it
doesn't force users to use it. ASLR can be disabled by setting
/proc/sys/kernel/randomize_va_space to 0. It's also possible to do it
for a single process (far better idea), which this wrapper tool will do:
#include
Hello Daniel, thank you for the detailed explanation.
The address of dynamic libraries, the stack and the heap (both sbrk and the mmap base) is already randomized today so the backtrace is already going to include randomized addresses for anything defined in a library.
Sure, but knowing at least the source line where it crashed in the executable may help a lot even if the rest of the backtrace is useless.
An executable is compiled as PIE is compatible with full ASLR but it doesn't force users to use it. ASLR can be disabled by setting /proc/sys/kernel/randomize_va_space to 0. It's also possible to do it for a single process (far better idea)
Oh, I didn't know that it's so easy to disable it. I would still prefer to have it enabled only for the core system and the applications that are a common point of entry such as web browser or web server, but I can cope with that if I can handle that by myself when everything is compiled with ASLR. Lukas
On 21/12/14 04:51 AM, Lukas Jirkovsky wrote:
Hello Daniel, thank you for the detailed explanation.
The address of dynamic libraries, the stack and the heap (both sbrk and the mmap base) is already randomized today so the backtrace is already going to include randomized addresses for anything defined in a library.
Sure, but knowing at least the source line where it crashed in the executable may help a lot even if the rest of the backtrace is useless.
It's possible to figure out the base address by using gdb on the core dump and looking at `i aux`. The AT_PHDR value can be rounded down to the page size and that's the base address. There might be an easier way to get it... From a non-PIE executable, where 0x400000 is the default base addr: 3 AT_PHDR Program headers for program 0x400040 This is with PIE and randomization disabled (0x555555554000): 3 AT_PHDR Program headers for program 0x555555554040 Here is a randomized one (0x7f2887981000): 3 AT_PHDR Program headers for program 0x7f2887981040 It's possible to map these back to the code just as you can do without PIE. The debugger itself is fully aware of the base address, even with no debugging symbols: #4 0x00007f21baf0c040 in __libc_start_main () from /usr/lib/libc.so.6 (i.e. it translates addresses to symbols whenever it can)
An executable is compiled as PIE is compatible with full ASLR but it doesn't force users to use it. ASLR can be disabled by setting /proc/sys/kernel/randomize_va_space to 0. It's also possible to do it for a single process (far better idea)
Oh, I didn't know that it's so easy to disable it. I would still prefer to have it enabled only for the core system and the applications that are a common point of entry such as web browser or web server, but I can cope with that if I can handle that by myself when everything is compiled with ASLR.
The problem is that it's important for anything doing networking, any executable with setuid/setcap/setgid and anything run by users on untrusted input like image viewers, text editors and tools like file and strings. If `python` or `ruby` isn't PIE, then it's trivial to exploit heap buffer overflows in native libraries used by anything written in those languages, etc. There are cases where you can rule out untrusted input, but I think they're a rare exception. For example, there's no use in hardening the `ldd` binary - but `lddtree` doesn't trust the code so it'd be nice if the interpreter it runs as (Python) was hardened.
The problem is that it's important for anything doing networking, any executable with setuid/setcap/setgid and anything run by users on untrusted input like image viewers, text editors and tools like file and strings. If `python` or `ruby` isn't PIE, then it's trivial to exploit heap buffer overflows in native libraries used by anything written in those languages, etc.
There are cases where you can rule out untrusted input, but I think they're a rare exception. For example, there's no use in hardening the `ldd` binary - but `lddtree` doesn't trust the code so it'd be nice if the interpreter it runs as (Python) was hardened.
These recent imagemagick vulnerabilities are a nice example: http://www.openwall.com/lists/oss-security/2014/12/24/1 It's often exposed to attackers through web apps where they're used to resize, compress and/or convert images provided by the users. PIE would be a forbidable defense for a situation like this because the attacker is not going to be able to use an info leak to obtain the base address. They will be forced to brute force through the ASLR, which will take a really long time, especially if the web app rate limits uploads. PaX has brute force protection so it'd take decades or even centuries to exploit it. It also ignores our CFLAGS, so it doesn't have SSP if it's not built with hardening-wrapper or patched: % checksec --file /usr/bin/convert RELRO STACK CANARY NX PIE RPATH RUNPATH FILE Partial RELRO No canary found NX enabled No PIE No RPATH No RUNPATH /usr/bin/convert However, SSP isn't going to help against issues like the heap overflows that were found but it will mitigate some vulnerabilities.
One more thing to note about this is that we'd need to do a rebuild of the remaining 186 packages with static libraries. In many cases, those libraries will probably just vanish thanks to the !staticlibs default. Static libraries aren't currently built as position independent unless they're meant to be usable in dynamic libraries. I think the only case where setting HARDENING_PIE=0 would be necessary is to deal with proprietary static libraries that are non-PIC/PIE but AFAIK there are none in the repositories anyway.
On 22/12/14 06:53, Daniel Micay wrote:
One more thing to note about this is that we'd need to do a rebuild of the remaining 186 packages with static libraries. In many cases, those libraries will probably just vanish thanks to the !staticlibs default.
Yet we have already rebuilt ALL packages since adding this default.The static libraries left have no shared coutnerpart.
Static libraries aren't currently built as position independent unless they're meant to be usable in dynamic libraries.
I think the only case where setting HARDENING_PIE=0 would be necessary is to deal with proprietary static libraries that are non-PIC/PIE but AFAIK there are none in the repositories anyway.
On 21/12/14 06:44 PM, Allan McRae wrote:
On 22/12/14 06:53, Daniel Micay wrote:
Yet we have already rebuilt ALL packages since adding this default.The static libraries left have no shared coutnerpart.
Ah, I forgot that there was a rebuild for that already. They'd just need to be rebuilt again for -fPIE to kick in from the wrapper.
On Thu, 18 Dec 2014 18:31:57 -0500
Daniel Micay
Arch's single biggest security weakness is that it's not benefiting from address space layout randomization (ASLR). Fixing this issue would be a major step towards being a leader in this area. Many distributions enable ASLR, stack protector, etc. for a chosen set of "security critical" stuff but very few enable them across the board.
Executable permissions on memory (NX bit) prevent an attacker from directly injecting code via memory corruption vulnerabilities. It's only truly useful in combination with ASLR because if the existing code is in a known location it can be reused by the attacker.[1]
[1] https://en.wikipedia.org/wiki/Return-oriented_programming
The cost of ASLR is that it requires position independent (relocatable) code (PIC), as is already required for all code in dynamic libraries.
On i686, PIC has a significant cost (up to 30% or more). The vanilla kernel also lacks emulation of the NX bit and has no brute force protection to make up for the smaller 32-bit address space. Only the subset of users using grsecurity / PaX would truly benefit. I don't care about i686 anyway.
On x86_64, there's hardware support for position independent code so it's essentially free. It does currently cost ~1% due to linker and compiler limitations (indirection to access globals) but this has been fixed in the binutils/gcc master branches. It's much cheaper than the -fstack-protector-strong switch which we're already using, and the security value is higher.
The only real barrier to enabling it is the lack of support in GCC for simply flipping it on by default. Library code is already built with -fPIC and is then linked with -shared. Full ASLR requires building the executable code with -fPIE (or -fPIC, which isn't as cheap) and then linking with -pie. There are two approaches to this:
1) Patching the toolchain's spec files (Hardened Gentoo) 2) Wrapper scripts for clang/gcc/ld.bfd/ld.gold (Debian, Fedora, Ubuntu)
Upstream hasn't accepted various forms of the first option, so I don't think it's a suitable approach for us. The hardening-wrapper package implements the second option and also enforces our existing hardening flags for build systems ignoring CFLAGS/CXXFLAGS/LDFLAGS.
All it would take is either adding hardening-wrapper to the default devtools packages list or making it part of base-devel.
Here's an example with a small C program:
strcat@thinktank i ~ master % cat test.c #include
static void foo() {} static int bar = 5;
int main(void) { int baz = 5; printf("function: %p, data: %p, stack: %p\n", foo, &bar, &baz); return 0; }
Without the hardening-wrapper package installed:
strcat@thinktank i ~ master % gcc test.c -o test strcat@thinktank i ~ master % checksec --file test RELRO STACK CANARY NX PIE RPATH RUNPATH FILE No RELRO No canary found NX enabled No PIE No RPATH No RUNPATH test strcat@thinktank i ~ master % ./test function: 0x400506, data: 0x600980, stack: 0x7fff12ba3a7c strcat@thinktank i ~ master % ./test function: 0x400506, data: 0x600980, stack: 0x7fffa47958fc
With the hardening-wrapper package installed:
strcat@thinktank i ~ master % gcc test.c -o test strcat@thinktank i ~ master % checksec --file test RELRO STACK CANARY NX PIE RPATH RUNPATH FILE Partial RELRO Canary found NX enabled PIE enabled No RPATH No RUNPATH test strcat@thinktank i ~ master % ./test function: 0x7f267e569800, data: 0x7f267e76a050, stack: 0x7fff3ee5ea14 strcat@thinktank i ~ master % ./test function: 0x7f6a06b3e790, data: 0x7f6a06d3f048, stack: 0x7fff76a891bc
The defaults in /etc/hardening-wrapper.conf can be overridden via env variables:
strcat@thinktank i ~ master % HARDENING_STACK_PROTECTOR=0 gcc test.c -o test strcat@thinktank i ~ master % checksec --file test RELRO STACK CANARY NX PIE RPATH RUNPATH FILE Partial RELRO No canary found NX enabled PIE enabled No RPATH No RUNPATH test
The need to use wrapper scripts is a bit ugly, but I think it's a nicer approach than tracking down and patching dozens of build systems to respect CFLAGS/LDFLAGS and it's the only realistic way to enable PIE. The only alternative short of patching GCC spec files is manually specifying the flags in thousands of executable-only packages, and patching any build system producing both binaries and libraries (yikes!).
I already use hardening-wrapper for nginx and I'd like to see it enabled by default for all our packages, so +1 from me. B
On 19/12/14 09:31, Daniel Micay wrote:
The only real barrier to enabling it is the lack of support in GCC for simply flipping it on by default. Library code is already built with -fPIC and is then linked with -shared. Full ASLR requires building the executable code with -fPIE (or -fPIC, which isn't as cheap) and then linking with -pie. There are two approaches to this:
1) Patching the toolchain's spec files (Hardened Gentoo) 2) Wrapper scripts for clang/gcc/ld.bfd/ld.gold (Debian, Fedora, Ubuntu)
Upstream hasn't accepted various forms of the first option,
https://gcc.gnu.org/ml/gcc-patches/2014-11/msg01905.html Best patch I have seen yet - and had no negative comments from upstream. I'd guess it has a good change to be included in gcc-5.0. If it gets committed I can backport immediately. I am not in favour of using the hardening script because I don't find it adheres to what we consider KISS. Our build system is supposed to be simple and entirely transparent when looking at the PKGBUILD and default makepkg.conf. Any user can run "abs" and "makepkg" and get (roughly) the same package. Allan
On 25/12/14 07:56 PM, Allan McRae wrote:
I'd guess it has a good change to be included in gcc-5.0. If it gets committed I can backport immediately.
I am not in favour of using the hardening script because I don't find it adheres to what we consider KISS.
I can understand that. It works the same way as ccache/distcc though, which have integration in makepkg via PATH injection. It's an ugly hack, but it's not the only place it's used and I think the practical benefit of enforcing hardening flags outweighs the loss of purity. I could file a few hundred bugs on our tracker for packages ignoring LDFLAGS, but it's going to take a lot of effort to do the same for CFLAGS because of false positives. I'll start doing that if it's the only option but I don't think anyone - myself or the packagers - is going to be very happy about it. The lack of ASLR is very disappointing, because it's so easy to enable it and there aren't tangible drawbacks. It's a very difficult obstacle to overcome in most cases too. I can't recommend that anyone who cares about their security and privacy use a distribution without it. It's even enabled across the board on Windows, OS X and Android... I think that's a pretty high cost to pay for a sense of purity. I'll continue waiting to see what happens with the GCC patches but I'm not too optimistic about that. The reasoning behind the rejection of the past bugs / patches was primarily that this should be handled in autotools (ignoring that most projects don't use it) and that still applies to this attempt.
Our build system is supposed to be simple and entirely transparent when looking at the PKGBUILD and default makepkg.conf. Any user can run "abs" and "makepkg" and get (roughly) the same package.
It's still just as reproducible. A user may have a different version or configuration of GCC. The hardening-wrapper package exists so users may have it installed, whether or not it's pulled in by default. The best you can get to a reproducible build is by using devtools but even that is going to pull in the current set of packages rather than whatever the packager used. There are many packages that don't build anymore.
On 26/12/14 12:01, Daniel Micay wrote:
I'll continue waiting to see what happens with the GCC patches but I'm not too optimistic about that. The reasoning behind the rejection of the past bugs / patches was primarily that this should be handled in autotools (ignoring that most projects don't use it) and that still applies to this attempt.
This time there was even a comment asking why there was a need for a configuration option and it should bee made default. I think momentum has swung. Allan
Am 26.12.2014 01:56, schrieb Allan McRae:
I am not in favour of using the hardening script because I don't find it adheres to what we consider KISS. Our build system is supposed to be simple and entirely transparent when looking at the PKGBUILD and default makepkg.conf. Any user can run "abs" and "makepkg" and get (roughly) the same package.
I agree, using such hacks kind of violates the kiss principle and our policy to follow upstream and don't patch or fork. I suggest to revistd this proposal once the needed changes are available upstream. Greetings, Pierre -- Pierre Schmitz, https://pierre-schmitz.com
On 31/12/14 04:47 AM, Pierre Schmitz wrote:
Am 26.12.2014 01:56, schrieb Allan McRae:
I am not in favour of using the hardening script because I don't find it adheres to what we consider KISS. Our build system is supposed to be simple and entirely transparent when looking at the PKGBUILD and default makepkg.conf. Any user can run "abs" and "makepkg" and get (roughly) the same package.
I agree, using such hacks kind of violates the kiss principle and our policy to follow upstream and don't patch or fork. I suggest to revistd this proposal once the needed changes are available upstream.
It's not necessarily going to land upstream. The fact that it can be done without changes to GCC via build systems or hardening scripts is the main reason it has been rejected in the past. On a package-by-package basis, carrying out-of-tree patches for missing SSP, RELRO and/or _FORTIFY_SOURCE is a lot less simple than simply adding makedepends=(hardening-wrapper). Lack of full ASLR in a package with a prominent attack surface is a higher priority bug than the other flags, but since it's a problem nearly across the board there's little point in filing them. I gave up on doing this manually almost as soon as I started: https://wiki.archlinux.org/index.php/DeveloperWiki:Security#Packages_not_res... If I could I would just write a high latency version of hardening-wrapper where it files a bug when CFLAGS / LDFLAGS wasn't respected rather than just injecting the flags itself. Not going to work thanks to stuff like autoconf.
participants (5)
-
Allan McRae
-
Bartłomiej Piotrowski
-
Daniel Micay
-
Lukas Jirkovsky
-
Pierre Schmitz