[pacman-dev] [PATCH v2] makepkg: record build information in .BUILDINFO
This information can be used to reproduce build conditions, which can then be used to determine if a package builds reproducibly. Signed-off-by: Allan McRae <allan@archlinux.org> --- v2: Add build directory, PKGBUILD sha256sum and debug packaging flags. scripts/makepkg.sh.in | 49 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 14 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 6ededa3..db96a30 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -223,7 +223,7 @@ run_pacman() { else cmd=("$PACMAN_PATH" "$@") fi - if [[ $1 != -@(T|Qq) ]]; then + if [[ $1 != -@(T|Qq|Q) ]]; then if type -p sudo >/dev/null; then cmd=(sudo "${cmd[@]}") else @@ -1143,19 +1143,30 @@ write_pkginfo() { [[ $optdepends ]] && printf "optdepend = %s\n" "${optdepends[@]//+([[:space:]])/ }" [[ $makedepends ]] && printf "makedepend = %s\n" "${makedepends[@]}" [[ $checkdepends ]] && printf "checkdepend = %s\n" "${checkdepends[@]}" +} - local it - for it in "${packaging_options[@]}"; do - check_option "$it" "y" - case $? in - 0) - printf "makepkgopt = %s\n" "$it" - ;; - 1) - printf "makepkgopt = %s\n" "!$it" - ;; - esac - done +write_buildinfo() { + msg2 "$(gettext "Generating %s file...")" ".BUILDINFO" + + printf "builddir = %s\n" "${BUILDDIR}" + + local sum="$(openssl dgst -sha256 "${BUILDFILE}")" + sum=${sum##* } + + printf "pkgbuild_sha256sum = %s\n" $sum + + printf "buildenv = %s\n" "${BUILDENV[@]}" + printf "options = %s\n" "${OPTIONS[@]}" + + printf "cppflags = %s\n" "$cppflags" + printf "cflags = %s\n" "$cflags" + printf "cxxflags = %s\n" "$cxxflags" + printf "ldflags = %s\n" "$ldflags" + printf "debug_cflags = %s\n" "$debug_cflags" + printf "debug_cxxflags = %s\n" "$debug_cxxflags" + + local pkglist=($(run_pacman -Q | sed "s# #-#")) + printf "installed = %s\n" "${pkglist[@]}" } create_package() { @@ -1172,8 +1183,9 @@ create_package() { pkgarch=$(get_pkg_arch) write_pkginfo > .PKGINFO + write_buildinfo > .BUILDINFO - local comp_files=('.PKGINFO') + local comp_files=('.PKGINFO' '.BUILDINFO') # check for changelog/install files for i in 'changelog/.CHANGELOG' 'install/.INSTALL'; do @@ -1958,6 +1970,15 @@ GPGKEY=${_GPGKEY:-$GPGKEY} PACKAGER=${_PACKAGER:-$PACKAGER} CARCH=${_CARCH:-$CARCH} +# record initial build environment +cppflags="$CPPFLAGS" +cflags="$CFLAGS" +cxxflags="$CXXFLAGS" +ldflags="$LDFLAGS" +debug_cflags="$DEBUG_CFLAGS" +debug_cxxflags="$DEBUG_CXXFLAGS" + + if (( ! INFAKEROOT )); then if (( EUID == 0 )); then error "$(gettext "Running %s as root is not allowed as it can cause permanent,\n\ -- 2.6.1
On Sat, Oct 17, 2015 at 10:57:12AM +1000, Allan McRae wrote:
This information can be used to reproduce build conditions, which can then be used to determine if a package builds reproducibly.
Signed-off-by: Allan McRae <allan@archlinux.org> ---
v2: Add build directory, PKGBUILD sha256sum and debug packaging flags.
scripts/makepkg.sh.in | 49 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 14 deletions(-)
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 6ededa3..db96a30 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -223,7 +223,7 @@ run_pacman() { else cmd=("$PACMAN_PATH" "$@") fi - if [[ $1 != -@(T|Qq) ]]; then + if [[ $1 != -@(T|Qq|Q) ]]; then if type -p sudo >/dev/null; then cmd=(sudo "${cmd[@]}") else @@ -1143,19 +1143,30 @@ write_pkginfo() { [[ $optdepends ]] && printf "optdepend = %s\n" "${optdepends[@]//+([[:space:]])/ }" [[ $makedepends ]] && printf "makedepend = %s\n" "${makedepends[@]}" [[ $checkdepends ]] && printf "checkdepend = %s\n" "${checkdepends[@]}" +}
- local it - for it in "${packaging_options[@]}"; do - check_option "$it" "y" - case $? in - 0) - printf "makepkgopt = %s\n" "$it" - ;; - 1) - printf "makepkgopt = %s\n" "!$it" - ;; - esac - done +write_buildinfo() { + msg2 "$(gettext "Generating %s file...")" ".BUILDINFO" + + printf "builddir = %s\n" "${BUILDDIR}" + + local sum="$(openssl dgst -sha256 "${BUILDFILE}")" + sum=${sum##* } + + printf "pkgbuild_sha256sum = %s\n" $sum + + printf "buildenv = %s\n" "${BUILDENV[@]}" + printf "options = %s\n" "${OPTIONS[@]}" + + printf "cppflags = %s\n" "$cppflags" + printf "cflags = %s\n" "$cflags" + printf "cxxflags = %s\n" "$cxxflags" + printf "ldflags = %s\n" "$ldflags" + printf "debug_cflags = %s\n" "$debug_cflags" + printf "debug_cxxflags = %s\n" "$debug_cxxflags"
I'm still confused about why we only list these specific vars... why not dump the entire environment? Off the top of my head, other vars like CC, AS, LD, and PYTHON could all have a huge effect on the resulting build... It'd also be nice to see this in a more easily reuseable format. Something like: while read var; do printf '%s=%q\n' "$var" "${!var}" done < <(compgen -A variable)
+ + local pkglist=($(run_pacman -Q | sed "s# #-#")) + printf "installed = %s\n" "${pkglist[@]}" }
create_package() { @@ -1172,8 +1183,9 @@ create_package() {
pkgarch=$(get_pkg_arch) write_pkginfo > .PKGINFO + write_buildinfo > .BUILDINFO
- local comp_files=('.PKGINFO') + local comp_files=('.PKGINFO' '.BUILDINFO')
# check for changelog/install files for i in 'changelog/.CHANGELOG' 'install/.INSTALL'; do @@ -1958,6 +1970,15 @@ GPGKEY=${_GPGKEY:-$GPGKEY} PACKAGER=${_PACKAGER:-$PACKAGER} CARCH=${_CARCH:-$CARCH}
+# record initial build environment +cppflags="$CPPFLAGS" +cflags="$CFLAGS" +cxxflags="$CXXFLAGS" +ldflags="$LDFLAGS" +debug_cflags="$DEBUG_CFLAGS" +debug_cxxflags="$DEBUG_CXXFLAGS" + + if (( ! INFAKEROOT )); then if (( EUID == 0 )); then error "$(gettext "Running %s as root is not allowed as it can cause permanent,\n\ -- 2.6.1
On 17/10/15 22:38, Dave Reisner wrote:
On Sat, Oct 17, 2015 at 10:57:12AM +1000, Allan McRae wrote:
This information can be used to reproduce build conditions, which can then be used to determine if a package builds reproducibly.
Signed-off-by: Allan McRae <allan@archlinux.org> ---
v2: Add build directory, PKGBUILD sha256sum and debug packaging flags.
scripts/makepkg.sh.in | 49 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 14 deletions(-)
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 6ededa3..db96a30 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -223,7 +223,7 @@ run_pacman() { else cmd=("$PACMAN_PATH" "$@") fi - if [[ $1 != -@(T|Qq) ]]; then + if [[ $1 != -@(T|Qq|Q) ]]; then if type -p sudo >/dev/null; then cmd=(sudo "${cmd[@]}") else @@ -1143,19 +1143,30 @@ write_pkginfo() { [[ $optdepends ]] && printf "optdepend = %s\n" "${optdepends[@]//+([[:space:]])/ }" [[ $makedepends ]] && printf "makedepend = %s\n" "${makedepends[@]}" [[ $checkdepends ]] && printf "checkdepend = %s\n" "${checkdepends[@]}" +}
- local it - for it in "${packaging_options[@]}"; do - check_option "$it" "y" - case $? in - 0) - printf "makepkgopt = %s\n" "$it" - ;; - 1) - printf "makepkgopt = %s\n" "!$it" - ;; - esac - done +write_buildinfo() { + msg2 "$(gettext "Generating %s file...")" ".BUILDINFO" + + printf "builddir = %s\n" "${BUILDDIR}" + + local sum="$(openssl dgst -sha256 "${BUILDFILE}")" + sum=${sum##* } + + printf "pkgbuild_sha256sum = %s\n" $sum + + printf "buildenv = %s\n" "${BUILDENV[@]}" + printf "options = %s\n" "${OPTIONS[@]}" + + printf "cppflags = %s\n" "$cppflags" + printf "cflags = %s\n" "$cflags" + printf "cxxflags = %s\n" "$cxxflags" + printf "ldflags = %s\n" "$ldflags" + printf "debug_cflags = %s\n" "$debug_cflags" + printf "debug_cxxflags = %s\n" "$debug_cxxflags"
I'm still confused about why we only list these specific vars... why not dump the entire environment? Off the top of my head, other vars like CC, AS, LD, and PYTHON could all have a huge effect on the resulting build...
This is about recording variables for a reproducible build. All these need to be specified within the PKGBUILD for a package to be reproducible. Allan
On 10/17/15 at 11:10pm, Allan McRae wrote:
On 17/10/15 22:38, Dave Reisner wrote:
On Sat, Oct 17, 2015 at 10:57:12AM +1000, Allan McRae wrote:
This information can be used to reproduce build conditions, which can then be used to determine if a package builds reproducibly.
Signed-off-by: Allan McRae <allan@archlinux.org> ---
v2: Add build directory, PKGBUILD sha256sum and debug packaging flags.
scripts/makepkg.sh.in | 49 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 14 deletions(-)
<snip>
I'm still confused about why we only list these specific vars... why not dump the entire environment? Off the top of my head, other vars like CC, AS, LD, and PYTHON could all have a huge effect on the resulting build...
This is about recording variables for a reproducible build. All these need to be specified within the PKGBUILD for a package to be reproducible.
Allan
I agree that essential environment variables and the like should be specified in the PKGBUILD, but when I (briefly) looked at reproducible builds a while back I took it a step further. My thought at the time was to have a makepkg wrapper that would use a clean makepkg.conf so that any makepkg variables would have to be specified in the PKGBUILD as well. This wrapper would then record whatever environment information was required itself. Given the variety of environmental factors that will have to be recorded and reproduced for some packages, I think it makes more sense to have a wrapper that does all of the environment handling on its own without directly involving makepkg. apg
participants (3)
-
Allan McRae
-
Andrew Gregory
-
Dave Reisner