[pacman-dev] LTO and PGO build options

[Que Quotion]

>What is the advantage to all this beyond people adding "-flto" to their
CFLAGS if they want LTO?

The advantage is not having to do that. Theoretically, the compiler
devs expect LTO to always work and want reports when it does not. We
should be able to enable LTO for every build, but my experience is
that--occasionally--the linker fails.

Users so inclined could "set it and forget it" most of the time, and
occasionally disable it if a package does not build. I find this more
efficient than modifying the PKGBUILD of each package that could
benefit from LTO.

>All modern builds should handle the rest by themselves.

Should, but don't last time I checked. A number of builds failed for
not having {AR,RANLIB,NM}FLAGS set when attempting LTO. I see the
complexity as another reason to have this built into makepkg instead
of having to do all this tediousness on a per-package basis.

The less anyone has to edit PKGBUILDs for generic things like
optimization, the better I think.

>I'm guessing this is untested...

That's kinda like.. not testing it, yeah?

I've been using this for quite some time. Nearly everything compiles
successfully with pgo. There were a couple of wrapper leftovers to fix
in my initial post, see below. The problem with only me doing the
testing is no one will ever beleive my success. I have been
successful, believe it--or try it for yourself.

>Anyway, I don't think PGO has a place in makepkg itself

Why not, specifically? We have other cflag stuff in there; besides, it
should be disabled by default so users could opt-in when they feel
ready. Do you know you can PGO just about any package in the
repositories? It's still takes two build steps, but this macro saves a
lot of tediousness over editing the individual PKGBUILDS. I don't see
how it's any more or less worthy than ccache, upx, etc.

As for the particular cflags I've used for PGO and LTO, these are
*known to work* but may not be the only options, might be overkill, or
there could even be a few more minor flags worth setting. I'd
appreciate some commentary on this matter, and some tests to prove or
disprove which flags are required.

"-fwhopr" might be a safter idea than "-flto", to avoid the compiler
running out of memory and crashing while compiling large programs.

New patchset:

In makepkg, "-Wl,-option" must be at the end of LDFLAGS:

--- makepkg	2016-02-23 11:52:34.000000000 +0900
+++ makepkg	2016-03-20 16:57:07.260935565 +0900
@@ -866,6 +866,32 @@
 		export DISTCC_HOSTS
 	fi

+
+	# Use lto if it is requested (check buildenv and PKGBUILD opts)
+	if check_buildoption "lto" "y" && [[ -f "$(gcc -print-search-dirs |
grep install | awk '{print $2 "liblto_plugin.so"}')" ]]; then
+		CFLAGS+=" -flto=$(getconf _NPROCESSORS_ONLN)"
+		CXXFLAGS+=" -flto=$(getconf _NPROCESSORS_ONLN)"
+		LDFLAGS+=" $CFLAGS $CXXFLAGS -Wl,-fuse-linker-plugin"
+		LTOPLUGIN="$(gcc -print-search-dirs | grep install | awk '{print $2
"liblto_plugin.so"}')"
+		ARFLAGS+=" --plugin $LTOPLUGIN"
+		RANLIBFLAGS+=" --plugin $LTOPLUGIN"
+		NMFLAGS+=" --plugin $LTOPLUGIN"
+	fi
+
+	# Generate or utilize pgo if it is requested (check buildenv and
PKGBUILD opts)
+	if check_buildoption "pgo" "y"; then
+		if [ ! -d "$PROFDEST/$pkgbase.gen" ]; then
+			mkdir "$PROFDEST/$pkgbase.gen"
+			CFLAGS+=" -fprofile-generate -fprofile-dir=$PROFDEST/$pkgbase.gen"
+			CXXFLAGS+=" -fprofile-generate -fprofile-dir=$PROFDEST/$pkgbase.gen"
+			LDFLAGS+=" -Wl,-lgcov"
+		elif [ ! -d "$PROFDEST/$pkgbase.used" ]; then
+			mv "$PROFDEST/$pkgbase.gen" "$PROFDEST/$pkgbase.used"
+			CFLAGS+=" -fprofile-correction -fprofile-use
-fprofile-dir=$PROFDEST/$pkgbase.used"
+			CXXFLAGS+=" -fprofile-correction -fprofile-use
-fprofile-dir=$PROFDEST/$pkgbase.used"
+		fi
+	fi
+
 	run_function_safe "build"
 }

@@ -1865,6 +2028,7 @@
 [[ -n ${PKGDEST} ]] && _PKGDEST=$(canonicalize_path ${PKGDEST})
 [[ -n ${SRCDEST} ]] && _SRCDEST=$(canonicalize_path ${SRCDEST})
 [[ -n ${SRCPKGDEST} ]] && _SRCPKGDEST=$(canonicalize_path ${SRCPKGDEST})
+[[ -n ${PROFDEST} ]] && _PROFDEST=$(canonicalize_path ${PROFDEST})
 [[ -n ${LOGDEST} ]] && _LOGDEST=$(canonicalize_path ${LOGDEST})
 [[ -n ${BUILDDIR} ]] && _BUILDDIR=$(canonicalize_path ${BUILDDIR})
 [[ -n ${PKGEXT} ]] && _PKGEXT=${PKGEXT}
@@ -1961,6 +2125,14 @@
 	IGNOREARCH=1
 fi

+PROFDEST=${_PROFDEST:-$PROFDEST}
+PROFDEST=${PROFDEST:-$startdir} #default to $startdir if undefined
+if [[ ! -w $PROFDEST ]] ; then
+	error "$(gettext "You do not have write permission to store profiles
in %s.")" "$PROFDEST"
+	plain "$(gettext "Aborting...")"
+	exit 1
+fi
+
 LOGDEST=${_LOGDEST:-$LOGDEST}
 LOGDEST=${LOGDEST:-$startdir} #default to $startdir if undefined
 if (( LOGGING )) && [[ ! -w $LOGDEST ]]; then

In makepkg.conf, I should have specified lto and pgo as BUILDENV
options, not package options:

--- makepkg.conf	2016-03-20 17:07:12.708226549 +0900
+++ makepkg.conf	2016-03-20 17:12:09.358079792 +0900
@@ -83,6 +83,8 @@
 #-- ccache:   Use ccache to cache compilation
 #-- check:    Run the check() function if present in the PKGBUILD
 #-- sign:     Generate PGP signature file
+#-- lto:      Use link-time optimization to reduce compiled binary
size and possibly improve performance
+#-- pgo:      Generate or use profile guided optimization to reduce
compile binary size and improve performance
 #
 BUILDENV=(!distcc color !ccache !check !sign)
 #
@@ -142,6 +144,8 @@
 #SRCDEST=/home/sources
 #-- Source packages: specify a fixed directory where all src packages
will be placed
 #SRCPKGDEST=/home/srcpackages
+#-- Profile cache: specify a fixed directory where profiles will be stored
+#PROFDEST=/home/sources
 #-- Log files: specify a fixed directory where all log files will be placed
 #LOGDEST=/home/makepkglogs
 #-- Packager: name/email of the person or organization building packages

I have tested these patches. They work for me and have no ill side effects.