Re: [pacman-dev] stripping binaries in makepkg

newer
[pacman-dev] [GIT] The official...

Sven-Hendrik Haase

23 Sep 2010 23 Sep '10

4:55 a.m.

I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case. -- Sven-Hendrik

Show replies by date

Dan McGee

23 Sep 23 Sep

5:08 a.m.

New subject: [pacman-dev] stripping binaries in makepkg

On Wed, Sep 22, 2010 at 11:55 PM, Sven-Hendrik Haase wrote:

...

I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.

-- Sven-Hendrik

Commit 721ceee1e2c9b18425d84cf39f6541b2f04072b3 might offer some insight, and then there is the original code from the git import: +# strip binaries +if [ "$NOSTRIP" = "0" ]; then + msg "Stripping debugging symbols from libraries..." + find pkg/{,usr,usr/local,opt/*}/lib -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip --strip-debug '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" + msg "Stripping symbols from binaries..." + find pkg/{,usr,usr/local,opt/*}/{bin,sbin} -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" +fi This leads me to believe stripping *.exe or *.dll files is bad news, and since we don't hardcode that anymore, we also limit where we traverse in the package filesystem. -Dan

Allan McRae

27 Sep 27 Sep

5:03 a.m.

New subject: [pacman-dev] stripping binaries in makepkg

On 23/09/10 15:08, Dan McGee wrote:

...

On Wed, Sep 22, 2010 at 11:55 PM, Sven-Hendrik Haase wrote:

...
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.

-- Sven-Hendrik

Commit 721ceee1e2c9b18425d84cf39f6541b2f04072b3 might offer some insight, and then there is the original code from the git import:

+# strip binaries +if [ "$NOSTRIP" = "0" ]; then + msg "Stripping debugging symbols from libraries..." + find pkg/{,usr,usr/local,opt/*}/lib -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip --strip-debug '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" + msg "Stripping symbols from binaries..." + find pkg/{,usr,usr/local,opt/*}/{bin,sbin} -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" +fi

This leads me to believe stripping *.exe or *.dll files is bad news, and since we don't hardcode that anymore, we also limit where we traverse in the package filesystem.

This should not be an issue now as we do more testing on file types before stripping. makepkg will not touch .dll files and .exe ones are probably cross-compiled and should have !strip in their PKGBUILD anyway. So... with that in mind I did some testing on this. For the "gcc-snapshot" package in the AUR, the time taken by the current stripping is 9 sec. Adjusting the strip to look at all files takes that to 9.3 sec. So even with the addition of running "file" on all headers etc, the majority of the time seems to be taken with the strip command. Another example, python 3 takes ~28 sec with the current method and ~29 when testing all files. Lets take a very extreme example - glest-data, a 110M arch=any package. That takes 0.01 sec with the current method and 9 sec when checking every file. That is about as extreme as possible... My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD. Sound OK? Allan

Florian Pritz

9:27 a.m.

New subject: [pacman-dev] stripping binaries in makepkg

On 27.09.2010 07:03, Allan McRae wrote:

...

My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD.

Sound OK?

Sounds simpler than the current method and somewhat more logical. -- Florian Pritz -- {flo,bluewind}@server-speed.net

Jonathan Conder

9:41 a.m.

New subject: [pacman-dev] stripping binaries in makepkg

On 27/09/10 18:03, Allan McRae wrote:

...

On 23/09/10 15:08, Dan McGee wrote:

...
On Wed, Sep 22, 2010 at 11:55 PM, Sven-Hendrik Haase wrote:

...
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.

-- Sven-Hendrik

Commit 721ceee1e2c9b18425d84cf39f6541b2f04072b3 might offer some insight, and then there is the original code from the git import:

+# strip binaries +if [ "$NOSTRIP" = "0" ]; then + msg "Stripping debugging symbols from libraries..." + find pkg/{,usr,usr/local,opt/*}/lib -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip --strip-debug '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" + msg "Stripping symbols from binaries..." + find pkg/{,usr,usr/local,opt/*}/{bin,sbin} -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" +fi

This leads me to believe stripping *.exe or *.dll files is bad news, and since we don't hardcode that anymore, we also limit where we traverse in the package filesystem.

This should not be an issue now as we do more testing on file types before stripping. makepkg will not touch .dll files and .exe ones are probably cross-compiled and should have !strip in their PKGBUILD anyway.

So... with that in mind I did some testing on this.

For the "gcc-snapshot" package in the AUR, the time taken by the current stripping is 9 sec. Adjusting the strip to look at all files takes that to 9.3 sec. So even with the addition of running "file" on all headers etc, the majority of the time seems to be taken with the strip command.

Another example, python 3 takes ~28 sec with the current method and ~29 when testing all files.

Lets take a very extreme example - glest-data, a 110M arch=any package. That takes 0.01 sec with the current method and 9 sec when checking every file. That is about as extreme as possible...

My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD.

Sound OK?

Allan

Why not set !strip automatically for arch=any packages?

Florian Pritz

9:43 a.m.

New subject: [pacman-dev] stripping binaries in makepkg

On 27.09.2010 11:41, Jonathan Conder wrote:

...

On 27/09/10 18:03, Allan McRae wrote:

...
My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD.

Sound OK?

Allan

Why not set !strip automatically for arch=any packages?

That's magic and might cause confusion. -- Florian Pritz -- {flo,bluewind}@server-speed.net

Allan McRae

10:20 a.m.

New subject: [pacman-dev] stripping binaries in makepkg

On 27/09/10 19:43, Florian Pritz wrote:

...

On 27.09.2010 11:41, Jonathan Conder wrote:

...
On 27/09/10 18:03, Allan McRae wrote:

...
My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD.

Sound OK?

Allan

Why not set !strip automatically for arch=any packages?

That's magic and might cause confusion.

We have had this discussion previously. The main concern is that there might be a valid case where an arch=any package wants to be stripped. I can not think of one, but if it occurred it would then require manual stripping. Adding !strip is easy enough to do. Allan

Allan McRae

23 Sep 23 Sep

5:12 a.m.

New subject: [pacman-dev] stripping binaries in makepkg

On 23/09/10 14:55, Sven-Hendrik Haase wrote:

...

I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.

It has been that way since 2005-03-15 with the initial commit in the git repo: http://projects.archlinux.org/pacman.git/diff/scripts/makepkg?id=d04baabafa . Except then the path was hard-coded in makepkg. The configurable STRIP_DIRS came much later: http://projects.archlinux.org/pacman.git/commit/?id=ae9e33ed As to why... I have never thought about it other than it seems a waste to search /usr/share for binaries to strip. Allan

Sven-Hendrik Haase

5:23 a.m.

New subject: [pacman-dev] stripping binaries in makepkg

On 23.09.2010 07:12, Allan McRae wrote:

...

On 23/09/10 14:55, Sven-Hendrik Haase wrote:

...
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.

It has been that way since 2005-03-15 with the initial commit in the git repo: http://projects.archlinux.org/pacman.git/diff/scripts/makepkg?id=d04baabafa . Except then the path was hard-coded in makepkg. The configurable STRIP_DIRS came much later: http://projects.archlinux.org/pacman.git/commit/?id=ae9e33ed

As to why... I have never thought about it other than it seems a waste to search /usr/share for binaries to strip.

Allan

Alright then. Could you add "usr/lib32" to the end of the STRIP_DIRS array for the default makepkg.conf to avoid any problems that come with building multilib packages? I can also make you a patch for this if it's necessary for such a small change. -- Sven-Hendrik

4960

Age (days ago)

4964

Last active (days ago)

List overview

Download

8 comments

5 participants

participants (5)

Allan McRae
Dan McGee
Florian Pritz
Jonathan Conder
Sven-Hendrik Haase