What is the advantage to all this beyond people adding "-flto" to their CFLAGS if they want LTO?
The advantage is not having to do that. Theoretically, the compiler devs expect LTO to always work and want reports when it does not. We should be able to enable LTO for every build, but my experience is that--occasionally--the linker fails. Users so inclined could "set it and forget it" most of the time, and occasionally disable it if a package does not build. I find this more efficient than modifying the PKGBUILD of each package that could benefit from LTO.
All modern builds should handle the rest by themselves.
Should, but don't last time I checked. A number of builds failed for not having {AR,RANLIB,NM}FLAGS set when attempting LTO. I see the complexity as another reason to have this built into makepkg instead of having to do all this tediousness on a per-package basis. The less anyone has to edit PKGBUILDs for generic things like optimization, the better I think.
I'm guessing this is untested...
That's kinda like.. not testing it, yeah? I've been using this for quite some time. Nearly everything compiles successfully with pgo. There were a couple of wrapper leftovers to fix in my initial post, see below. The problem with only me doing the testing is no one will ever beleive my success. I have been successful, believe it--or try it for yourself.
Anyway, I don't think PGO has a place in makepkg itself
Why not, specifically? We have other cflag stuff in there; besides, it should be disabled by default so users could opt-in when they feel ready. Do you know you can PGO just about any package in the repositories? It's still takes two build steps, but this macro saves a lot of tediousness over editing the individual PKGBUILDS. I don't see how it's any more or less worthy than ccache, upx, etc. As for the particular cflags I've used for PGO and LTO, these are *known to work* but may not be the only options, might be overkill, or there could even be a few more minor flags worth setting. I'd appreciate some commentary on this matter, and some tests to prove or disprove which flags are required. "-fwhopr" might be a safter idea than "-flto", to avoid the compiler running out of memory and crashing while compiling large programs. New patchset: In makepkg, "-Wl,-option" must be at the end of LDFLAGS: --- makepkg 2016-02-23 11:52:34.000000000 +0900 +++ makepkg 2016-03-20 16:57:07.260935565 +0900 @@ -866,6 +866,32 @@ export DISTCC_HOSTS fi + + # Use lto if it is requested (check buildenv and PKGBUILD opts) + if check_buildoption "lto" "y" && [[ -f "$(gcc -print-search-dirs | grep install | awk '{print $2 "liblto_plugin.so"}')" ]]; then + CFLAGS+=" -flto=$(getconf _NPROCESSORS_ONLN)" + CXXFLAGS+=" -flto=$(getconf _NPROCESSORS_ONLN)" + LDFLAGS+=" $CFLAGS $CXXFLAGS -Wl,-fuse-linker-plugin" + LTOPLUGIN="$(gcc -print-search-dirs | grep install | awk '{print $2 "liblto_plugin.so"}')" + ARFLAGS+=" --plugin $LTOPLUGIN" + RANLIBFLAGS+=" --plugin $LTOPLUGIN" + NMFLAGS+=" --plugin $LTOPLUGIN" + fi + + # Generate or utilize pgo if it is requested (check buildenv and PKGBUILD opts) + if check_buildoption "pgo" "y"; then + if [ ! -d "$PROFDEST/$pkgbase.gen" ]; then + mkdir "$PROFDEST/$pkgbase.gen" + CFLAGS+=" -fprofile-generate -fprofile-dir=$PROFDEST/$pkgbase.gen" + CXXFLAGS+=" -fprofile-generate -fprofile-dir=$PROFDEST/$pkgbase.gen" + LDFLAGS+=" -Wl,-lgcov" + elif [ ! -d "$PROFDEST/$pkgbase.used" ]; then + mv "$PROFDEST/$pkgbase.gen" "$PROFDEST/$pkgbase.used" + CFLAGS+=" -fprofile-correction -fprofile-use -fprofile-dir=$PROFDEST/$pkgbase.used" + CXXFLAGS+=" -fprofile-correction -fprofile-use -fprofile-dir=$PROFDEST/$pkgbase.used" + fi + fi + run_function_safe "build" } @@ -1865,6 +2028,7 @@ [[ -n ${PKGDEST} ]] && _PKGDEST=$(canonicalize_path ${PKGDEST}) [[ -n ${SRCDEST} ]] && _SRCDEST=$(canonicalize_path ${SRCDEST}) [[ -n ${SRCPKGDEST} ]] && _SRCPKGDEST=$(canonicalize_path ${SRCPKGDEST}) +[[ -n ${PROFDEST} ]] && _PROFDEST=$(canonicalize_path ${PROFDEST}) [[ -n ${LOGDEST} ]] && _LOGDEST=$(canonicalize_path ${LOGDEST}) [[ -n ${BUILDDIR} ]] && _BUILDDIR=$(canonicalize_path ${BUILDDIR}) [[ -n ${PKGEXT} ]] && _PKGEXT=${PKGEXT} @@ -1961,6 +2125,14 @@ IGNOREARCH=1 fi +PROFDEST=${_PROFDEST:-$PROFDEST} +PROFDEST=${PROFDEST:-$startdir} #default to $startdir if undefined +if [[ ! -w $PROFDEST ]] ; then + error "$(gettext "You do not have write permission to store profiles in %s.")" "$PROFDEST" + plain "$(gettext "Aborting...")" + exit 1 +fi + LOGDEST=${_LOGDEST:-$LOGDEST} LOGDEST=${LOGDEST:-$startdir} #default to $startdir if undefined if (( LOGGING )) && [[ ! -w $LOGDEST ]]; then In makepkg.conf, I should have specified lto and pgo as BUILDENV options, not package options: --- makepkg.conf 2016-03-20 17:07:12.708226549 +0900 +++ makepkg.conf 2016-03-20 17:12:09.358079792 +0900 @@ -83,6 +83,8 @@ #-- ccache: Use ccache to cache compilation #-- check: Run the check() function if present in the PKGBUILD #-- sign: Generate PGP signature file +#-- lto: Use link-time optimization to reduce compiled binary size and possibly improve performance +#-- pgo: Generate or use profile guided optimization to reduce compile binary size and improve performance # BUILDENV=(!distcc color !ccache !check !sign) # @@ -142,6 +144,8 @@ #SRCDEST=/home/sources #-- Source packages: specify a fixed directory where all src packages will be placed #SRCPKGDEST=/home/srcpackages +#-- Profile cache: specify a fixed directory where profiles will be stored +#PROFDEST=/home/sources #-- Log files: specify a fixed directory where all log files will be placed #LOGDEST=/home/makepkglogs #-- Packager: name/email of the person or organization building packages I have tested these patches. They work for me and have no ill side effects.