- Pacman-dev - lists.archlinux.org

[pacman-dev] [PATCH] libmakepkg: reproducibilty for python packages
by Allan McRae 08 Aug '21

08 Aug '21

Arch Linux has been setting PYTHONHASHSEED=0 to create deterministic .pyc files. After a thorough review by the Arch Security Team, setting this variable was determined not to generated vulnerable .pyc files, as when the loader loads the .pyc file and unmarshalls it, the internal runtime will just populate the unordered data structures and use a new runtime hash for them. Signed-off-by: Allan McRae <allan(a)archlinux.org> --- Despite effectively being a 1 line patch to add "export PYTHONHASHSEED=0", I decided to add a libmakepkg submodule dedicated to reproducible packages. This way anybody can drop in mdoules to test easily. I'd like to include this for pacman-6.0.1. It is a bit more than I would usually like to include in a point release, but it is an easy win to for the reproducibile builds effort. Any objections to including it? scripts/libmakepkg/meson.build | 1 + scripts/libmakepkg/reproducible.sh.in | 29 ++++++++++++++++++++ scripts/libmakepkg/reproducible/meson.build | 17 ++++++++++++ scripts/libmakepkg/reproducible/python.sh.in | 29 ++++++++++++++++++++ 4 files changed, 76 insertions(+) create mode 100644 scripts/libmakepkg/reproducible.sh.in create mode 100644 scripts/libmakepkg/reproducible/meson.build create mode 100644 scripts/libmakepkg/reproducible/python.sh.in diff --git a/scripts/libmakepkg/meson.build b/scripts/libmakepkg/meson.build index b0697f78..50eb905b 100644 --- a/scripts/libmakepkg/meson.build +++ b/scripts/libmakepkg/meson.build @@ -5,6 +5,7 @@ libmakepkg_modules = [ { 'name' : 'lint_config', 'has_subdir' : true }, { 'name' : 'lint_package', 'has_subdir' : true }, { 'name' : 'lint_pkgbuild', 'has_subdir' : true }, + { 'name' : 'reproducible', 'has_subdir' : true }, { 'name' : 'source', 'has_subdir' : true }, { 'name' : 'srcinfo', }, { 'name' : 'tidy', 'has_subdir' : true }, diff --git a/scripts/libmakepkg/reproducible.sh.in b/scripts/libmakepkg/reproducible.sh.in new file mode 100644 index 00000000..7db2a3cc --- /dev/null +++ b/scripts/libmakepkg/reproducible.sh.in @@ -0,0 +1,29 @@ +#!/bin/bash +# +# reproducible.sh - utilities for improving package reproducibility +# +# Copyright (c) 2021 Pacman Development Team <pacman-dev(a)archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[[ -n "$LIBMAKEPKG_REPRODUCIBLE_SH" ]] && return +LIBMAKEPKG_REPRODUCIBLE_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + + +for lib in "$LIBRARY/reproducible/"*.sh; do + source "$lib" +done diff --git a/scripts/libmakepkg/reproducible/meson.build b/scripts/libmakepkg/reproducible/meson.build new file mode 100644 index 00000000..0738fe9c --- /dev/null +++ b/scripts/libmakepkg/reproducible/meson.build @@ -0,0 +1,17 @@ +libmakepkg_module = 'reproducible' + +sources = [ + 'python.sh.in', +] + +foreach src : sources + output_dir = join_paths(get_option('datadir'), 'makepkg', libmakepkg_module) + + custom_target( + libmakepkg_module + '_' + src.underscorify(), + command : [ SCRIPT_EDITOR, '@INPUT@', '@OUTPUT@' ], + input : src, + output : '@BASENAME@', + install : true, + install_dir : output_dir) +endforeach diff --git a/scripts/libmakepkg/reproducible/python.sh.in b/scripts/libmakepkg/reproducible/python.sh.in new file mode 100644 index 00000000..45fd4521 --- /dev/null +++ b/scripts/libmakepkg/reproducible/python.sh.in @@ -0,0 +1,29 @@ +#!/bin/bash +# +# python.sh - creating reproducible python packages +# +# Copyright (c) 2021 Pacman Development Team <pacman-dev(a)archlinux.org> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +[[ -n "$LIBMAKEPKG_REPRODUCIBLE_PYTHON_SH" ]] && return +LIBMAKEPKG_REPRODUCIBLE_PYTHON_SH=1 + + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + + +# disable hash randomization when creating .pyc files +export PYTHONHASHSEED=0 -- 2.32.0

1 0

[pacman-dev] [PATCH] Gitlab CI: add bsdtar to Fedora install
by Allan McRae 04 Aug '21

04 Aug '21

Signed-off-by: Allan McRae <allan(a)archlinux.org> --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 507e1332..668b97ba 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -103,7 +103,7 @@ fedora: - > dnf -y install git findutils patch sed - meson gcc libtool + meson gcc libtool bsdtar gpgme-devel libarchive-devel libcurl-devel openssl-devel gettext-devel asciidoc python3 dash gawk fakeroot fakechroot -- 2.32.0

1 0

[pacman-dev] [PATCH] PKGBUILD.5: Fix typo
by Andre Schröder 04 Aug '21

04 Aug '21

Signed-off-by: André Schröder <andre.schroedr(a)gmail.com> --- doc/PKGBUILD.5.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/PKGBUILD.5.asciidoc b/doc/PKGBUILD.5.asciidoc index d207d92d..dee15f5e 100644 --- a/doc/PKGBUILD.5.asciidoc +++ b/doc/PKGBUILD.5.asciidoc @@ -361,7 +361,7 @@ function. is skipped. *build() Function*:: - The optional `build()` function is use to compile and/or adjust the source + The optional `build()` function is used to compile and/or adjust the source files in preparation to be installed by the `package()` function. *check() Function*:: -- 2.32.0

2 2

[pacman-dev] [PATCH] makepkg: fix stripping of relocatable binaries with binutils>=2.37
by Allan McRae 04 Aug '21

04 Aug '21

Binutils commit 93df3340fd5ad32f784214fc125de71811da72ff enabled readelf to report "Position-Independent Executable" files. Fix stripping to account for this change. Signed-off-by: Allan McRae <allan(a)archlinux.org> --- scripts/libmakepkg/tidy/strip.sh.in | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/libmakepkg/tidy/strip.sh.in b/scripts/libmakepkg/tidy/strip.sh.in index 5d03b28f..96602538 100644 --- a/scripts/libmakepkg/tidy/strip.sh.in +++ b/scripts/libmakepkg/tidy/strip.sh.in @@ -135,6 +135,8 @@ tidy_strip() { case "$(LC_ALL=C readelf -h "$binary" 2>/dev/null)" in *Type:*'DYN (Shared object file)'*) # Libraries (.so) or Relocatable binaries strip_flags="$STRIP_SHARED";; + *Type:*'DYN (Position-Independent Executable file)'*) # Relocatable binaries + strip_flags="$STRIP_SHARED";; *Type:*'EXEC (Executable file)'*) # Binaries strip_flags="$STRIP_BINARIES";; *Type:*'REL (Relocatable file)'*) # Libraries (.a) or objects -- 2.32.0

1 0

[pacman-dev] [PATCH] pacman: correct inconsistent spelling of Packages
by Torsten Schmitz 26 Jul '21

26 Jul '21

When using VerbosePkgLists the header spells Package instead of Packages as it does normally. Signed-off-by: Torsten Schmitz <noreply.torsten(a)gmail.com> --- src/pacman/util.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/pacman/util.c b/src/pacman/util.c index 5486e7a5..36ae0bf0 100644 --- a/src/pacman/util.c +++ b/src/pacman/util.c @@ -824,7 +824,7 @@ static alpm_list_t *create_verbose_header(size_t count) alpm_list_t *ret = NULL; char *header; - pm_asprintf(&header, "%s (%zu)", _("Package"), count); + pm_asprintf(&header, "%s (%zu)", _("Packages"), count); add_table_cell(&ret, header, CELL_TITLE | CELL_FREE); add_table_cell(&ret, _("Old Version"), CELL_TITLE); -- 2.32.0

2 2

[pacman-dev] [PATCH] libmakepkg: allow correctly sourcing when $LIBRARY is not set
by Eli Schwartz 04 Jul '21

04 Jul '21

We usually set this up to default to the build time configured install location, but a couple of files crept in without this. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- scripts/libmakepkg/buildenv.sh.in | 2 ++ scripts/libmakepkg/executable.sh.in | 2 ++ scripts/libmakepkg/util/pkgbuild.sh.in | 2 ++ 3 files changed, 6 insertions(+) diff --git a/scripts/libmakepkg/buildenv.sh.in b/scripts/libmakepkg/buildenv.sh.in index 0f4178a90..07b32ce93 100644 --- a/scripts/libmakepkg/buildenv.sh.in +++ b/scripts/libmakepkg/buildenv.sh.in @@ -22,6 +22,8 @@ [[ -n "$LIBMAKEPKG_BUILDENV_SH" ]] && return LIBMAKEPKG_BUILDENV_SH=1 +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + declare -a buildenv_functions build_options for lib in "$LIBRARY/buildenv/"*.sh; do diff --git a/scripts/libmakepkg/executable.sh.in b/scripts/libmakepkg/executable.sh.in index ea40289ba..dfa8a0bdf 100644 --- a/scripts/libmakepkg/executable.sh.in +++ b/scripts/libmakepkg/executable.sh.in @@ -21,6 +21,8 @@ [[ -n "$LIBMAKEPKG_EXECUTABLE_SH" ]] && return LIBMAKEPKG_EXECUTABLE_SH=1 +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + declare -a executable_functions for lib in "$LIBRARY/executable/"*.sh; do diff --git a/scripts/libmakepkg/util/pkgbuild.sh.in b/scripts/libmakepkg/util/pkgbuild.sh.in index ceda7824f..e3c5b5902 100644 --- a/scripts/libmakepkg/util/pkgbuild.sh.in +++ b/scripts/libmakepkg/util/pkgbuild.sh.in @@ -21,6 +21,8 @@ [[ -n "$LIBMAKEPKG_UTIL_PKGBUILD_SH" ]] && return LIBMAKEPKG_UTIL_PKGBUILD_SH=1 +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + source "$LIBRARY/util/schema.sh" -- 2.32.0

1 0

[pacman-dev] [Git][pacman/pacman][master] doc: use localdate instead of pacman_date to improve reproducibility
by Allan McRae (＠allan) 01 Jul '21

01 Jul '21

Allan McRae pushed to branch master at Pacman / Pacman Commits: b4383b8d by Jonas Witschel at 2021-07-01T11:19:18+10:00 doc: use localdate instead of pacman_date to improve reproducibility pacman_date is set to the current date during build without respecting SOURCE_DATE_EPOCH. As a result, a build cannot be fully reproduced on a later date because the date embedded into the man pages does not match. In contrast, the built-in asciidoc attribute "localdate" respects SOURCE_DATE_EPOCH and has the desired ISO 8601 format, so simply use that instead of the custom "pacman_date" attribute. Fixes: FS#71154 Signed-off-by: Jonas Witschel <diabonas at archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> - - - - - 2 changed files: - doc/asciidoc.conf - doc/meson.build Changes: ===================================== doc/asciidoc.conf ===================================== @@ -50,7 +50,7 @@ ifdef::backend-docbook[] template::[header-declarations] <refentry> <refentryinfo> -<date>{pacman_date}</date> +<date>{localdate}</date> </refentryinfo> <refmeta> <refentrytitle>{mantitle}</refentrytitle> ===================================== doc/meson.build ===================================== @@ -25,7 +25,6 @@ asciidoc_conf = join_paths(meson.current_source_dir(), 'asciidoc.conf') asciidoc_opts = [ '-f', asciidoc_conf, '-a', 'pacman_version="@0@"'.format(PACKAGE_VERSION), - '-a', 'pacman_date=@0@'.format(run_command('date', '+%Y-%m-%d').stdout().strip()), '-a', 'srcext=@0@'.format(get_option('src-ext')), '-a', 'pkgext=@0@'.format(get_option('pkg-ext')), '-a', 'pkgdatadir=@0@'.format(PKGDATADIR), View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/b4383b8d007e28aa76324f4… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/b4383b8d007e28aa76324f4… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[pacman-dev] Multiple progress bars not compatible with dumb terminals
by miha＠kamnitnik.top 29 Jun '21

29 Jun '21

On less capable terminals, pacman should probably display only one active progress bar simultaneously. An example of such a terminal is Emacs's M-x shell ($TERM == "dumb"). It can't display the additional "total" properly because it doesn't implement cursor movement.

2 2

[pacman-dev] [PATCH] doc: make man pages reproducible
by Allan McRae 25 Jun '21

25 Jun '21

Use asciidoc's localdate which respects SOURCE_DATE_EPOCH. Thanks to Jonas Witschel <diabonas(a)archlinux.org> for investigating this issue. Fixes FS#71154. --- doc/asciidoc.conf | 2 +- doc/meson.build | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/asciidoc.conf b/doc/asciidoc.conf index e2e72672..830cc908 100644 --- a/doc/asciidoc.conf +++ b/doc/asciidoc.conf @@ -50,7 +50,7 @@ ifdef::backend-docbook[] template::[header-declarations] <refentry> <refentryinfo> -<date>{pacman_date}</date> +<date>{localdate}</date> </refentryinfo> <refmeta> <refentrytitle>{mantitle}</refentrytitle> diff --git a/doc/meson.build b/doc/meson.build index 4aaac554..5a8eb3a4 100644 --- a/doc/meson.build +++ b/doc/meson.build @@ -25,7 +25,6 @@ asciidoc_conf = join_paths(meson.current_source_dir(), 'asciidoc.conf') asciidoc_opts = [ '-f', asciidoc_conf, '-a', 'pacman_version="@0@"'.format(PACKAGE_VERSION), - '-a', 'pacman_date=@0@'.format(run_command('date', '+%Y-%m-%d').stdout().strip()), '-a', 'srcext=@0@'.format(get_option('src-ext')), '-a', 'pkgext=@0@'.format(get_option('pkg-ext')), '-a', 'pkgdatadir=@0@'.format(PKGDATADIR), -- 2.32.0

1 0

[pacman-dev] [PATCH] doc: use localdate instead of pacman_date to improve reproducibility
by Jonas Witschel 25 Jun '21

25 Jun '21

pacman_date is set to the current date during build without respecting SOURCE_DATE_EPOCH. As a result, a build cannot be fully reproduced on a later date because the date embedded into the man pages does not match. In contrast, the built-in asciidoc attribute "localdate" respects SOURCE_DATE_EPOCH and has the desired ISO 8601 format, so simply use that instead of the custom "pacman_date" attribute. Fixes: FS#71154 Signed-off-by: Jonas Witschel <diabonas(a)archlinux.org> --- doc/asciidoc.conf | 2 +- doc/meson.build | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/asciidoc.conf b/doc/asciidoc.conf index e2e72672..830cc908 100644 --- a/doc/asciidoc.conf +++ b/doc/asciidoc.conf @@ -50,7 +50,7 @@ ifdef::backend-docbook[] template::[header-declarations] <refentry> <refentryinfo> -<date>{pacman_date}</date> +<date>{localdate}</date> </refentryinfo> <refmeta> <refentrytitle>{mantitle}</refentrytitle> diff --git a/doc/meson.build b/doc/meson.build index 4aaac554..5a8eb3a4 100644 --- a/doc/meson.build +++ b/doc/meson.build @@ -25,7 +25,6 @@ asciidoc_conf = join_paths(meson.current_source_dir(), 'asciidoc.conf') asciidoc_opts = [ '-f', asciidoc_conf, '-a', 'pacman_version="@0@"'.format(PACKAGE_VERSION), - '-a', 'pacman_date=@0@'.format(run_command('date', '+%Y-%m-%d').stdout().strip()), '-a', 'srcext=@0@'.format(get_option('src-ext')), '-a', 'pkgext=@0@'.format(get_option('pkg-ext')), '-a', 'pkgdatadir=@0@'.format(PKGDATADIR), -- 2.32.0

1 0