Re: [pacman-dev] stripping binaries in makepkg
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case. -- Sven-Hendrik
On Wed, Sep 22, 2010 at 11:55 PM, Sven-Hendrik Haase <sh@lutzhaase.com> wrote:
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.
-- Sven-Hendrik
Commit 721ceee1e2c9b18425d84cf39f6541b2f04072b3 might offer some insight, and then there is the original code from the git import: +# strip binaries +if [ "$NOSTRIP" = "0" ]; then + msg "Stripping debugging symbols from libraries..." + find pkg/{,usr,usr/local,opt/*}/lib -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip --strip-debug '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" + msg "Stripping symbols from binaries..." + find pkg/{,usr,usr/local,opt/*}/{bin,sbin} -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" +fi This leads me to believe stripping *.exe or *.dll files is bad news, and since we don't hardcode that anymore, we also limit where we traverse in the package filesystem. -Dan
On 23/09/10 15:08, Dan McGee wrote:
On Wed, Sep 22, 2010 at 11:55 PM, Sven-Hendrik Haase<sh@lutzhaase.com> wrote:
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.
-- Sven-Hendrik
Commit 721ceee1e2c9b18425d84cf39f6541b2f04072b3 might offer some insight, and then there is the original code from the git import:
+# strip binaries +if [ "$NOSTRIP" = "0" ]; then + msg "Stripping debugging symbols from libraries..." + find pkg/{,usr,usr/local,opt/*}/lib -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip --strip-debug '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" + msg "Stripping symbols from binaries..." + find pkg/{,usr,usr/local,opt/*}/{bin,sbin} -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" +fi
This leads me to believe stripping *.exe or *.dll files is bad news, and since we don't hardcode that anymore, we also limit where we traverse in the package filesystem.
This should not be an issue now as we do more testing on file types before stripping. makepkg will not touch .dll files and .exe ones are probably cross-compiled and should have !strip in their PKGBUILD anyway. So... with that in mind I did some testing on this. For the "gcc-snapshot" package in the AUR, the time taken by the current stripping is 9 sec. Adjusting the strip to look at all files takes that to 9.3 sec. So even with the addition of running "file" on all headers etc, the majority of the time seems to be taken with the strip command. Another example, python 3 takes ~28 sec with the current method and ~29 when testing all files. Lets take a very extreme example - glest-data, a 110M arch=any package. That takes 0.01 sec with the current method and 9 sec when checking every file. That is about as extreme as possible... My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD. Sound OK? Allan
On 27.09.2010 07:03, Allan McRae wrote:
My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD.
Sound OK?
Sounds simpler than the current method and somewhat more logical. -- Florian Pritz -- {flo,bluewind}@server-speed.net
On 27/09/10 18:03, Allan McRae wrote:
On 23/09/10 15:08, Dan McGee wrote:
On Wed, Sep 22, 2010 at 11:55 PM, Sven-Hendrik Haase<sh@lutzhaase.com> wrote:
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.
-- Sven-Hendrik
Commit 721ceee1e2c9b18425d84cf39f6541b2f04072b3 might offer some insight, and then there is the original code from the git import:
+# strip binaries +if [ "$NOSTRIP" = "0" ]; then + msg "Stripping debugging symbols from libraries..." + find pkg/{,usr,usr/local,opt/*}/lib -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip --strip-debug '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" + msg "Stripping symbols from binaries..." + find pkg/{,usr,usr/local,opt/*}/{bin,sbin} -type f -not -name "*.dll" -not -name "*.exe" \ + -exec /usr/bin/strip '{}' \; 2>&1 \ + | grep -v "No such file" | grep -v "format not recognized" +fi
This leads me to believe stripping *.exe or *.dll files is bad news, and since we don't hardcode that anymore, we also limit where we traverse in the package filesystem.
This should not be an issue now as we do more testing on file types before stripping. makepkg will not touch .dll files and .exe ones are probably cross-compiled and should have !strip in their PKGBUILD anyway.
So... with that in mind I did some testing on this.
For the "gcc-snapshot" package in the AUR, the time taken by the current stripping is 9 sec. Adjusting the strip to look at all files takes that to 9.3 sec. So even with the addition of running "file" on all headers etc, the majority of the time seems to be taken with the strip command.
Another example, python 3 takes ~28 sec with the current method and ~29 when testing all files.
Lets take a very extreme example - glest-data, a 110M arch=any package. That takes 0.01 sec with the current method and 9 sec when checking every file. That is about as extreme as possible...
My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD.
Sound OK?
Allan
Why not set !strip automatically for arch=any packages?
On 27.09.2010 11:41, Jonathan Conder wrote:
On 27/09/10 18:03, Allan McRae wrote:
My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD.
Sound OK?
Allan
Why not set !strip automatically for arch=any packages?
That's magic and might cause confusion. -- Florian Pritz -- {flo,bluewind}@server-speed.net
On 27/09/10 19:43, Florian Pritz wrote:
On 27.09.2010 11:41, Jonathan Conder wrote:
On 27/09/10 18:03, Allan McRae wrote:
My conclusion? For most binary packages, the time taken in the stripping step is mainly used for the actual stripping of files and not testing which files to strip. I think this justifies removing the STRIP_DIRS variable from makepkg. For large architecture independent packages where the difference from this change is noticeable, the packager should add !strip to their PKGBUILD.
Sound OK?
Allan
Why not set !strip automatically for arch=any packages?
That's magic and might cause confusion.
We have had this discussion previously. The main concern is that there might be a valid case where an arch=any package wants to be stripped. I can not think of one, but if it occurred it would then require manual stripping. Adding !strip is easy enough to do. Allan
On 23/09/10 14:55, Sven-Hendrik Haase wrote:
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.
It has been that way since 2005-03-15 with the initial commit in the git repo: http://projects.archlinux.org/pacman.git/diff/scripts/makepkg?id=d04baabafa . Except then the path was hard-coded in makepkg. The configurable STRIP_DIRS came much later: http://projects.archlinux.org/pacman.git/commit/?id=ae9e33ed As to why... I have never thought about it other than it seems a waste to search /usr/share for binaries to strip. Allan
On 23.09.2010 07:12, Allan McRae wrote:
On 23/09/10 14:55, Sven-Hendrik Haase wrote:
I'd also like an answer to this. Why not do away with the individual directories? Locating the binaries even in a fairly large package will be fairly quick in any case.
It has been that way since 2005-03-15 with the initial commit in the git repo: http://projects.archlinux.org/pacman.git/diff/scripts/makepkg?id=d04baabafa . Except then the path was hard-coded in makepkg. The configurable STRIP_DIRS came much later: http://projects.archlinux.org/pacman.git/commit/?id=ae9e33ed
As to why... I have never thought about it other than it seems a waste to search /usr/share for binaries to strip.
Allan
Alright then. Could you add "usr/lib32" to the end of the STRIP_DIRS array for the default makepkg.conf to avoid any problems that come with building multilib packages? I can also make you a patch for this if it's necessary for such a small change. -- Sven-Hendrik
participants (5)
-
Allan McRae
-
Dan McGee
-
Florian Pritz
-
Jonathan Conder
-
Sven-Hendrik Haase