[pacman-dev] [PATCH] makepkg: respect $SOURCE_DATE_EPOCH to activate reproducible builds
If SOURCE_DATE_EPOCH is set, `touch` all source files before running
build() to fix the modification times. This works around build systems
and compilers that embed the file modification times into the file
contents of release artifacts.
Signed-off-by: Eli Schwartz
On 07/07/17 03:26, Eli Schwartz wrote:
If SOURCE_DATE_EPOCH is set, `touch` all source files before running build() to fix the modification times. This works around build systems and compilers that embed the file modification times into the file contents of release artifacts.
I think this is a reasonable compromise. @Levente: are you happy with this approach?
If SOURCE_DATE_EPOCH is set, `touch` all source files before running
build() to fix the modification times. This works around build systems
and compilers that embed the file modification times into the file
contents of release artifacts.
Signed-off-by: Eli Schwartz
On 07/28/2017 04:36 AM, Allan McRae wrote:
On 07/07/17 03:26, Eli Schwartz wrote:
If SOURCE_DATE_EPOCH is set, `touch` all source files before running build() to fix the modification times. This works around build systems and compilers that embed the file modification times into the file contents of release artifacts.
I think this is a reasonable compromise.
@Levente: are you happy with this approach?
The conditional REPRODUCIBLE touch block should be outside of run_build as former would imply that a build() function is mandatory inside every PKGBUILD to make this work. Its quite common (f.e. for python) to not have a build() function and I believe it should generally work without enforcing to have such function. Therefor I would recommend we move that block between run_build and run_prepare, f.e. a place where this works is line 1696 in commit bcc9c417. A tested/verified adjustment doing the above would be: https://github.com/anthraxx/pacman/commit/520acf93d83774c4b74ec8c19c1ba31fdd... Technically this approach will work, but personally i believe this is a bit too much hidden requirement to make it work. So for clarity I would either recommend we add a section describing this behaviour inside the manpage (as manually defining SOURCE_DATE_EPOCH will be mandatory to make it work) or we use makepkg flags plus config (which will also make it more transparent). I don't have any strong favorites, Eli implementation will do its job, the only difference will be that manually setting SOURCE_DATE_EPOCH is mandatory for the first initial invocation as well. cheers, Levente PS: thanks :P
On 07/28/2017 07:59 AM, Levente Polyak wrote:
On 07/28/2017 04:36 AM, Allan McRae wrote:
On 07/07/17 03:26, Eli Schwartz wrote:
If SOURCE_DATE_EPOCH is set, `touch` all source files before running build() to fix the modification times. This works around build systems and compilers that embed the file modification times into the file contents of release artifacts.
I think this is a reasonable compromise.
@Levente: are you happy with this approach?
The conditional REPRODUCIBLE touch block should be outside of run_build as former would imply that a build() function is mandatory inside every PKGBUILD to make this work. Its quite common (f.e. for python) to not have a build() function and I believe it should generally work without enforcing to have such function. Therefor I would recommend we move that block between run_build and run_prepare, f.e. a place where this works is line 1696 in commit bcc9c417. A tested/verified adjustment doing the above would be: https://github.com/anthraxx/pacman/commit/520acf93d83774c4b74ec8c19c1ba31fdd...
I kind of feel like this is a sign that people are doing the wrong thing, since you shouldn't be modifying $srcdir during package(). Personally, I am very careful about making sure `python setup.py build` is run during build(), the same way I am careful about making sure `make` is run during build() even when `make install` would cause `make all` to run anyway. I don't feel like we should make a split between GNU Make and other build systems, or between a build system that executes a C compiler vs. one that merely runs pod2man and copies files around. I won't even mention npm except to say, look at aur/rapydscript-ng-git.
Technically this approach will work, but personally i believe this is a bit too much hidden requirement to make it work. So for clarity I would either recommend we add a section describing this behaviour inside the manpage (as manually defining SOURCE_DATE_EPOCH will be mandatory to make it work) or we use makepkg flags plus config (which will also make it more transparent). I don't have any strong favorites, Eli implementation will do its job, the only difference will be that manually setting SOURCE_DATE_EPOCH is mandatory for the first initial invocation as well.
Okay. I personally prefer the implementation that doesn't add more flags or makepkg.conf options, because at a minimum anyone who wants to make use of this will need to have tooling to read from a previous invocation regardless or on top of those flags for the second invocation, so we might as well follow the same approach the whole time. Perhaps a REPRODUCIBILITY section in makepkg(1) right after ADDITIONAL FEATURES, then, to describe what efforts makepkg will go to to ensure a reproducible build? And then add SOURCE_DATE_EPOCH for environment variables makepkg will respect. -- Eli Schwartz
If SOURCE_DATE_EPOCH is set, `touch` all source files before running
build() to fix the modification times. This works around build systems
and compilers that embed the file modification times into the file
contents of release artifacts.
Signed-off-by: Eli Schwartz
On 24/08/17 07:12, Eli Schwartz wrote:
If SOURCE_DATE_EPOCH is set, `touch` all source files before running build() to fix the modification times. This works around build systems and compilers that embed the file modification times into the file contents of release artifacts.
Signed-off-by: Eli Schwartz
--- v3: add makepkg(8) documentation
doc/makepkg.8.txt | 16 ++++++++++++++++ scripts/makepkg.sh.in | 14 +++++++++++++- 2 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/doc/makepkg.8.txt b/doc/makepkg.8.txt index 2dff1b19..4258e6bd 100644 --- a/doc/makepkg.8.txt +++ b/doc/makepkg.8.txt @@ -206,6 +206,7 @@ Options *\--printsrcinfo*:: Generate and print the SRCINFO file to stdout.
+ Additional Features ------------------- makepkg supports building development versions of packages without having to @@ -214,6 +215,19 @@ separate utility 'versionpkg'. See linkman:PKGBUILD[5] for details on how to set up a development PKGBUILD.
+Reproducibility +--------------- +makepkg is designed to be compatible with +link:https://reproducible-builds.org/docs/[Reproducible Builds]. If the +**SOURCE_DATE_EPOCH** environment variable is set, it will be exported to +subprocesses, and source and package file modification times and package +metadata will be unified based on the timestamp specified. + +If the **SOURCE_DATE_EPOCH** environment variable is not set, makepkg will use +its own start date for internal use, but is not responsible for ensuring the +package files themselves are built reproducibly. +
I don't like the phrasing there. How about : If the **SOURCE_DATE_EPOCH** environment variable is not set, makepkg will use its own start date for internal use, but will not unify source file timestamps before building.
+ Environment Variables --------------------- **PACMAN**:: @@ -265,6 +279,8 @@ Environment Variables Specify a key to use when signing packages, overriding the GPGKEY setting in linkman:makepkg.conf[5]
+**SOURCE_DATE_EPOCH=**"<date>":: + Used for link:https://reproducible-builds.org/docs/[Reproducible Builds].
Configuration ------------- diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 20e9dd7e..77d39ca5 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -79,6 +79,7 @@ PKGFUNC=0 PKGVERFUNC=0 PREPAREFUNC=0 REPKG=0 +REPRODUCIBLE=0 RMDEPS=0 SKIPCHECKSUMS=0 SKIPPGPCHECK=0 @@ -87,7 +88,12 @@ SPLITPKG=0 SOURCEONLY=0 VERIFYSOURCE=0
-export SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH:-$(date +%s)} +if [[ -n $SOURCE_DATE_EPOCH ]]; then + REPRODUCIBLE=1 +else + SOURCE_DATE_EPOCH=$(date +%s) +fi +export SOURCE_DATE_EPOCH
PACMAN_OPTS=()
@@ -475,6 +481,12 @@ run_prepare() { }
run_build() { + if (( REPRODUCIBLE )); then + # We have activated reproducible builds, so unify source times before + # building + find "$srcdir" -exec touch -h -d @$SOURCE_DATE_EPOCH {} + + fi +
I don't like this in run_build(). That will introduce an undocumented requirement that a PKGBUILD has a build() function to have its source file timestamps unified. I am happy with the location suggested here: https://github.com/anthraxx/pacman/commit/520acf93
run_function_safe "build" }
If SOURCE_DATE_EPOCH is set, `touch` all source files as part of
prepare() to fix the modification times. This works around build systems
and compilers that embed the file modification times into the file
contents of release artifacts.
Signed-off-by: Eli Schwartz
participants (4)
-
Allan McRae
-
Eli Schwartz
-
Eli Schwartz
-
Levente Polyak