[pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

newer
[pacman-dev] [PATCH 1/5] Check if...

Steven Noonan

10 Feb 2014 10 Feb '14

12:26 a.m.

I noticed when I built a kernel package with options 'strip' and 'debug' that the debug package was missing the debug information for the kernel modules. This patch adds kernel modules to the targets for 'strip' and is compatible with the split-out debug information packages. v2: Correct logic error -- did a 'continue' after the 2nd-level case statement instead of in the default case. v3: Oops, forgot to terminate outer case. Ran this version through the test suite and all tests passed. Signed-off-by: Steven Noonan --- scripts/makepkg.sh.in | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 940e947..027fcc3 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1659,6 +1659,13 @@ tidy_install() { strip_flags="$STRIP_SHARED";; *application/x-archive*) # Libraries (.a) strip_flags="$STRIP_STATIC";; + *application/x-object*) + case "$binary" in + *.ko) # Kernel module + strip_flags="$STRIP_SHARED";; + *) + continue;; + esac;; *application/x-executable*) # Binaries strip_flags="$STRIP_BINARIES";; *) -- 1.8.5.4

Show replies by date

Allan McRae

10 Feb 10 Feb

5:32 a.m.

New subject: [pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

On 10/02/14 09:26, Steven Noonan wrote:

...

I noticed when I built a kernel package with options 'strip' and 'debug' that the debug package was missing the debug information for the kernel modules. This patch adds kernel modules to the targets for 'strip' and is compatible with the split-out debug information packages.

v2: Correct logic error -- did a 'continue' after the 2nd-level case statement instead of in the default case. v3: Oops, forgot to terminate outer case. Ran this version through the test suite and all tests passed.

For future reference, put comments like these ....

...

Signed-off-by: Steven Noonan ---

... under this line. That way they are ignored when committing the patch.

...

scripts/makepkg.sh.in | 7 +++++++ 1 file changed, 7 insertions(+)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 940e947..027fcc3 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1659,6 +1659,13 @@ tidy_install() { strip_flags="$STRIP_SHARED";; *application/x-archive*) # Libraries (.a) strip_flags="$STRIP_STATIC";; + *application/x-object*) + case "$binary" in + *.ko) # Kernel module + strip_flags="$STRIP_SHARED";; + *) + continue;; + esac;;

This looks fine to me. I notice the Arch Linux kernel package uses options=('!strip'). I am lead to believe that stripping vmlinux lead to issues with building modules. Also the Arch PKGBUILD compresses the modules before it gets to stripping the files. So, I guess my question is... Well, I'm not sure! I think I just need a more complete picture to asses this. Is my interpretation of the issue with stripping vmlinux right? Does having separate debug symbols for vmlinux still allow modules be built? How do compressed modules and separate debug symbols interact? Allan

Steven Noonan

11:56 a.m.

New subject: [pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

On Sun, Feb 9, 2014 at 8:32 PM, Allan McRae wrote:

...

On 10/02/14 09:26, Steven Noonan wrote:

...
I noticed when I built a kernel package with options 'strip' and 'debug' that the debug package was missing the debug information for the kernel modules. This patch adds kernel modules to the targets for 'strip' and is compatible with the split-out debug information packages.

v2: Correct logic error -- did a 'continue' after the 2nd-level case statement instead of in the default case. v3: Oops, forgot to terminate outer case. Ran this version through the test suite and all tests passed.

For future reference, put comments like these ....

...
Signed-off-by: Steven Noonan ---

... under this line. That way they are ignored when committing the patch.

I'll bear that in mind, thanks for pointing it out.

...

...
scripts/makepkg.sh.in | 7 +++++++ 1 file changed, 7 insertions(+)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 940e947..027fcc3 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1659,6 +1659,13 @@ tidy_install() { strip_flags="$STRIP_SHARED";; *application/x-archive*) # Libraries (.a) strip_flags="$STRIP_STATIC";; + *application/x-object*) + case "$binary" in + *.ko) # Kernel module + strip_flags="$STRIP_SHARED";; + *) + continue;; + esac;;

This looks fine to me.

I notice the Arch Linux kernel package uses options=('!strip'). I am lead to believe that stripping vmlinux lead to issues with building modules. Also the Arch PKGBUILD compresses the modules before it gets to stripping the files.

I maintain the Arch Linux EC2 kernel, and I'm doing adventurous things with the packaging to make debugging a bit easier. I am looking at enabling CONFIG_DEBUG_INFO and adding -debug packages. That will indeed require removing the compression stage for the modules. In my experience this has had no negative impact. In fact, tools like 'perf' look for separate debug symbols. As for module builds, you're correct: MODPOST 1 modules FATAL: vmlinux has no symtab? /usr/src/linux-3.13.2-1-ec2/scripts/Makefile.modpost:90: recipe for target '__modpost' failed make[1]: *** [__modpost] Error 1 The build still fails even if the -debug packages are installed alongside. I'd argue however that this doesn't impact this patch directly -- splitting debug symbols from the kernel modules is OK, but stripping vmlinux is not. This could be worked around by: 1. Moving vmlinux into linux-headers. vmlinux is kind of an oddball (only item in /usr/src/linux for the main package) -- maybe I'm missing context as to why it's in the 'linux' package? it seems like you'd need the -headers package for a module build anyway, so it can't be there just for module builds. 2. Change linux-headers to use options=('!strip'). This also avoids the strange side effect of having a -debug for linux *and* linux-headers. I just tested the above and successfully built a module. So I guess the question is if vmlinux really needs to be in the primary package (and I'd be curious to know why, if so).

...

So, I guess my question is... Well, I'm not sure! I think I just need a more complete picture to asses this. Is my interpretation of the issue with stripping vmlinux right? Does having separate debug symbols for vmlinux still allow modules be built? How do compressed modules and separate debug symbols interact?

Well, as mentioned above, 'perf' behavior is now more reasonable (e.g. 'Test object code reading' in 'perf test' passes now). I'm also getting better results out of sampling with 'perf'. This behavioral improvement sort of makes sense since the primary developers of it are RedHat folks and splitting out debuginfo is how RedHat packages their kernels.

Allan McRae

11 Feb 11 Feb

1:03 a.m.

New subject: [pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

On 10/02/14 20:56, Steven Noonan wrote:

...

On Sun, Feb 9, 2014 at 8:32 PM, Allan McRae wrote:

...
On 10/02/14 09:26, Steven Noonan wrote:

...
I noticed when I built a kernel package with options 'strip' and 'debug' that the debug package was missing the debug information for the kernel modules. This patch adds kernel modules to the targets for 'strip' and is compatible with the split-out debug information packages.

v2: Correct logic error -- did a 'continue' after the 2nd-level case statement instead of in the default case. v3: Oops, forgot to terminate outer case. Ran this version through the test suite and all tests passed.

For future reference, put comments like these ....

...
Signed-off-by: Steven Noonan ---

... under this line. That way they are ignored when committing the patch.

I'll bear that in mind, thanks for pointing it out.

...
...
scripts/makepkg.sh.in | 7 +++++++ 1 file changed, 7 insertions(+)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 940e947..027fcc3 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1659,6 +1659,13 @@ tidy_install() { strip_flags="$STRIP_SHARED";; *application/x-archive*) # Libraries (.a) strip_flags="$STRIP_STATIC";; + *application/x-object*) + case "$binary" in + *.ko) # Kernel module + strip_flags="$STRIP_SHARED";; + *) + continue;; + esac;;

This looks fine to me.

I notice the Arch Linux kernel package uses options=('!strip'). I am lead to believe that stripping vmlinux lead to issues with building modules. Also the Arch PKGBUILD compresses the modules before it gets to stripping the files.

I maintain the Arch Linux EC2 kernel, and I'm doing adventurous things with the packaging to make debugging a bit easier. I am looking at enabling CONFIG_DEBUG_INFO and adding -debug packages. That will indeed require removing the compression stage for the modules. In my experience this has had no negative impact. In fact, tools like 'perf' look for separate debug symbols.

As for module builds, you're correct:

MODPOST 1 modules FATAL: vmlinux has no symtab? /usr/src/linux-3.13.2-1-ec2/scripts/Makefile.modpost:90: recipe for target '__modpost' failed make[1]: *** [__modpost] Error 1

The build still fails even if the -debug packages are installed alongside.

I'd argue however that this doesn't impact this patch directly -- splitting debug symbols from the kernel modules is OK, but stripping vmlinux is not. This could be worked around by:

1. Moving vmlinux into linux-headers. vmlinux is kind of an oddball (only item in /usr/src/linux for the main package) -- maybe I'm missing context as to why it's in the 'linux' package? it seems like you'd need the -headers package for a module build anyway, so it can't be there just for module builds. 2. Change linux-headers to use options=('!strip'). This also avoids the strange side effect of having a -debug for linux *and* linux-headers.

I just tested the above and successfully built a module. So I guess the question is if vmlinux really needs to be in the primary package (and I'd be curious to know why, if so).

That makes sense to me. Can you create a feature request on the Arch bug tracker to implement this. Anyway - now that I am more clear on the situation, I think your patch makes a lot of sense and will pull it. Allan

Steven Noonan

8:14 a.m.

New subject: [pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

On Mon, Feb 10, 2014 at 4:03 PM, Allan McRae wrote:

...

On 10/02/14 20:56, Steven Noonan wrote:

...
On Sun, Feb 9, 2014 at 8:32 PM, Allan McRae wrote:

...
On 10/02/14 09:26, Steven Noonan wrote:

...
I noticed when I built a kernel package with options 'strip' and 'debug' that the debug package was missing the debug information for the kernel modules. This patch adds kernel modules to the targets for 'strip' and is compatible with the split-out debug information packages.

v2: Correct logic error -- did a 'continue' after the 2nd-level case statement instead of in the default case. v3: Oops, forgot to terminate outer case. Ran this version through the test suite and all tests passed.

For future reference, put comments like these ....

...
Signed-off-by: Steven Noonan ---

... under this line. That way they are ignored when committing the patch.

I'll bear that in mind, thanks for pointing it out.

...
...
scripts/makepkg.sh.in | 7 +++++++ 1 file changed, 7 insertions(+)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 940e947..027fcc3 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1659,6 +1659,13 @@ tidy_install() { strip_flags="$STRIP_SHARED";; *application/x-archive*) # Libraries (.a) strip_flags="$STRIP_STATIC";; + *application/x-object*) + case "$binary" in + *.ko) # Kernel module + strip_flags="$STRIP_SHARED";; + *) + continue;; + esac;;

This looks fine to me.

I notice the Arch Linux kernel package uses options=('!strip'). I am lead to believe that stripping vmlinux lead to issues with building modules. Also the Arch PKGBUILD compresses the modules before it gets to stripping the files.

I maintain the Arch Linux EC2 kernel, and I'm doing adventurous things with the packaging to make debugging a bit easier. I am looking at enabling CONFIG_DEBUG_INFO and adding -debug packages. That will indeed require removing the compression stage for the modules. In my experience this has had no negative impact. In fact, tools like 'perf' look for separate debug symbols.

As for module builds, you're correct:

MODPOST 1 modules FATAL: vmlinux has no symtab? /usr/src/linux-3.13.2-1-ec2/scripts/Makefile.modpost:90: recipe for target '__modpost' failed make[1]: *** [__modpost] Error 1

The build still fails even if the -debug packages are installed alongside.

I'd argue however that this doesn't impact this patch directly -- splitting debug symbols from the kernel modules is OK, but stripping vmlinux is not. This could be worked around by:

1. Moving vmlinux into linux-headers. vmlinux is kind of an oddball (only item in /usr/src/linux for the main package) -- maybe I'm missing context as to why it's in the 'linux' package? it seems like you'd need the -headers package for a module build anyway, so it can't be there just for module builds. 2. Change linux-headers to use options=('!strip'). This also avoids the strange side effect of having a -debug for linux *and* linux-headers.

I just tested the above and successfully built a module. So I guess the question is if vmlinux really needs to be in the primary package (and I'd be curious to know why, if so).

That makes sense to me. Can you create a feature request on the Arch bug tracker to implement this.

Anyway - now that I am more clear on the situation, I think your patch makes a lot of sense and will pull it.

Thanks, Allan! I've submitted the feature request: https://bugs.archlinux.org/task/38869

Thomas Bächler

10:58 p.m.

New subject: [pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

Am 10.02.2014 11:56, schrieb Steven Noonan:

...

I maintain the Arch Linux EC2 kernel, and I'm doing adventurous things with the packaging to make debugging a bit easier. I am looking at enabling CONFIG_DEBUG_INFO and adding -debug packages. That will indeed require removing the compression stage for the modules. In my experience this has had no negative impact. In fact, tools like 'perf' look for separate debug symbols.

I was unaware that there is a way to debug kernel modules. And I am sceptical. So you can debug modules, but not built-in stuff? This all sounds very weird.

...

As for module builds, you're correct:

MODPOST 1 modules FATAL: vmlinux has no symtab? /usr/src/linux-3.13.2-1-ec2/scripts/Makefile.modpost:90: recipe for target '__modpost' failed make[1]: *** [__modpost] Error 1

vmlinux is not even needed to build modules. The only reason we install vmlinux is 1) we used to need it for our Xen machines (this is obsolete now) and 2) it is needed for oprofile iirc. I do not think it is in any way related to building modules, or needed for that matter.

Allan McRae

12 Feb 12 Feb

12:13 a.m.

New subject: [pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

On 12/02/14 07:58, Thomas Bächler wrote:

...

Am 10.02.2014 11:56, schrieb Steven Noonan:

...
I maintain the Arch Linux EC2 kernel, and I'm doing adventurous things with the packaging to make debugging a bit easier. I am looking at enabling CONFIG_DEBUG_INFO and adding -debug packages. That will indeed require removing the compression stage for the modules. In my experience this has had no negative impact. In fact, tools like 'perf' look for separate debug symbols.

I was unaware that there is a way to debug kernel modules. And I am sceptical. So you can debug modules, but not built-in stuff? This all sounds very weird.

...
As for module builds, you're correct:

MODPOST 1 modules FATAL: vmlinux has no symtab? /usr/src/linux-3.13.2-1-ec2/scripts/Makefile.modpost:90: recipe for target '__modpost' failed make[1]: *** [__modpost] Error 1

vmlinux is not even needed to build modules. The only reason we install vmlinux is 1) we used to need it for our Xen machines (this is obsolete now) and 2) it is needed for oprofile iirc.

I do not think it is in any way related to building modules, or needed for that matter.

I have seen a few people report that stripping that file breaks module builds. This and your statement that it is not related to module builds seem inconsistent. Allan

Thomas Bächler

12:52 a.m.

New subject: [pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

Am 12.02.2014 00:13, schrieb Allan McRae:

...

I have seen a few people report that stripping that file breaks module builds. This and your statement that it is not related to module builds seem inconsistent.

So, I just deleted the file and built a module successfully. I don't see where you get your information, but that seems pretty solid.

Dave Reisner

12:59 a.m.

New subject: [pacman-dev] [PATCH v3] makepkg: implement support for stripping kernel modules

On Wed, Feb 12, 2014 at 12:52:11AM +0100, Thomas Bächler wrote:

...

Am 12.02.2014 00:13, schrieb Allan McRae:

...
I have seen a few people report that stripping that file breaks module builds. This and your statement that it is not related to module builds seem inconsistent.

So, I just deleted the file and built a module successfully. I don't see where you get your information, but that seems pretty solid.

vmlinux present, unstripped -- modpost succeeds vmlinux present, stripped -- modpost fails vmlinux not present -- modpost succeeds Above results apply to kdbus and nvidia. No idea....