August 2023 - Pacman-dev - lists.archlinux.org

Supporting openssh-signing of packages and databases
by Max Gautier 08 Dec '23

08 Dec '23

Would pacman be interested in supporting signature verification of packages (and databases) using the (relatively new) signing protocol introduced and implemented by Openssh [1], alongside GPG signatures ? The intended benefits are be easier packagers setup and workflows (compared to GPG), as well as more out of the box support for signing with FIDO2 tokens (as openssh has sk-* keys to natively support those). The ALLOWED_SIGNERS (documented in man ssh-keygen) file and thus the signing namespace or namespaces would be up to the distribution using pacman (presumably, different distributions should not use the same namespace(s)). If there is interest in this, I'm interested in trying my hands at an implementation; in that case, I'd welcome any advice or things to avoid in doing so. Thanks, [1]: https://raw.githubusercontent.com/openssh/openssh-portable/master/PROTOCOL.… -- Max Gautier

2 6

[PATCH v3] alpm_run_chroot: Fix the execution of pacman hooks
by Dragan Simic 19 Oct '23

19 Oct '23

Running "pacman -Syyu" to perform a full system update resulted in a rather conspicuous error message in the output of pacman: ( 3/18) Creating temporary files... Assertion 'fd' failed at src/tmpfiles/tmpfiles.c:843, function fd_set_perms(). Aborting. /usr/share/libalpm/scripts/systemd-hook: line 28: 1735 Aborted (core dumped) /usr/bin/systemd-tmpfiles --create error: command failed to execute correctly Expectedly, running "systemd-tmpfiles --create" manually afterwards went just fine and resulted in no errors, leading to a conclusion that executing /usr/share/libalpm/hooks/30-systemd-tmpfiles.hook failed, but only when it was run from within pacman. After a detailed and rather lengthy investigation, it turned out the code in lib/libalpm/util.c that executes the hooks by forking a child has some bugs that allow the error to occur under certain circumstances. In particular, function _alpm_run_chroot() that executes hooks in a fork()ed child does not employ dup2() properly, but instead executes close() followed by dup2(). The man page for dup2() clearly states in the quotation below that attempts to re-implement the equivalent functionality, which is the case in function _alpm_run_chroot(), must be avoided: The dup2() system call performs the same task as dup(), but instead of using the lowest-numbered unused file descriptor, it uses the file descriptor number specified in newfd. In other words, the file descriptor newfd is adjusted so that it now refers to the same open file description as oldfd. If the file descriptor newfd was previously open, it is closed before being reused; the close is performed silently (i.e., any errors during the close are not reported by dup2()). The steps of closing and reusing the file descriptor newfd are performed atomically. This is important, because trying to implement equivalent functionality using close(2) and dup() would be subject to race conditions, whereby newfd might be reused between the two steps. Such reuse could happen because the main program is interrupted by a signal handler that allocates a file descriptor, or because a parallel thread allocates a file descriptor. As a result, a condition can occur in which the file descriptor 0 is closed by calling close(0), and left closed after the while loop that fails to execute dup2() because of receiving EBUSY, resulting in the described issues. Also, failed attempts to execute dup2() should be treated as fatal errors instead of being silently ignored. Let's improve the code to prevent the described issues. While there, perform a minor cleanup as well, to make the formatting of the code a tiny bit more consistent. Signed-off-by: Dragan Simic <dsimic(a)manjaro.org> --- lib/libalpm/util.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/lib/libalpm/util.c b/lib/libalpm/util.c index dffa3b51..0897c5ea 100644 --- a/lib/libalpm/util.c +++ b/lib/libalpm/util.c @@ -639,28 +639,40 @@ int _alpm_run_chroot(alpm_handle_t *handle, const char *cmd, char *const argv[], if(pid == 0) { /* this code runs for the child only (the actual chroot/exec) */ - close(0); - close(1); - close(2); - while(dup2(child2parent_pipefd[HEAD], 1) == -1 && errno == EINTR); - while(dup2(child2parent_pipefd[HEAD], 2) == -1 && errno == EINTR); - while(dup2(parent2child_pipefd[TAIL], 0) == -1 && errno == EINTR); - close(parent2child_pipefd[TAIL]); close(parent2child_pipefd[HEAD]); close(child2parent_pipefd[TAIL]); + while(dup2(child2parent_pipefd[HEAD], STDERR_FILENO) == -1) { + if(errno != EINTR) { + /* the child cannot talk through the parent at this point, but try yelling anyway */ + fprintf(stderr, _("could not redirect standard error (%s)\n"), strerror(errno)); + exit(1); + } + } + while(dup2(parent2child_pipefd[TAIL], STDIN_FILENO) == -1) { + if(errno != EINTR) { + /* use fprintf() instead of _alpm_log() to send output through the parent */ + fprintf(stderr, _("could not redirect standard input (%s)\n"), strerror(errno)); + exit(1); + } + } + close(parent2child_pipefd[TAIL]); + while(dup2(child2parent_pipefd[HEAD], STDOUT_FILENO) == -1) { + if(errno != EINTR) { + fprintf(stderr, _("could not redirect standard output (%s)\n"), strerror(errno)); + exit(1); + } + } close(child2parent_pipefd[HEAD]); if(cwdfd >= 0) { close(cwdfd); } - /* use fprintf instead of _alpm_log to send output through the parent */ if(chroot(handle->root) != 0) { fprintf(stderr, _("could not change the root directory (%s)\n"), strerror(errno)); exit(1); } if(chdir("/") != 0) { - fprintf(stderr, _("could not change directory to %s (%s)\n"), - "/", strerror(errno)); + fprintf(stderr, _("could not change directory to %s (%s)\n"), "/", strerror(errno)); exit(1); } /* bash assumes it's being run under rsh/ssh if stdin is a socket and

2 5

[Git][pacman/pacman][master] repo-add: fix exit on mktemp failure
by Allan McRae (＠allan) 28 Aug '23

28 Aug '23

Allan McRae pushed to branch master at Pacman / Pacman Commits: ffde12ce by Nicolas Rolans at 2023-08-28T12:43:56+00:00 repo-add: fix exit on mktemp failure - - - - - 1 changed file: - scripts/repo-add.sh.in View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/ffde12cebdd146ac5246490… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/ffde12cebdd146ac5246490… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[Git][pacman/pacman][master] Replace md5sums with sha256sums in documentation
by Allan McRae (＠allan) 28 Aug '23

28 Aug '23

Allan McRae pushed to branch master at Pacman / Pacman Commits: 37dae721 by Jouke Witteveen at 2023-08-28T12:27:28+00:00 Replace md5sums with sha256sums in documentation As noted in !24, md5sums are outdated. - - - - - 2 changed files: - doc/PKGBUILD-example.txt - doc/makepkg-template.1.asciidoc View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/37dae721d37647981514557… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/37dae721d37647981514557… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[Git][pacman/pacman][master] proto: Change the default checksum from md5 to sha256
by Allan McRae (＠allan) 28 Aug '23

28 Aug '23

Allan McRae pushed to branch master at Pacman / Pacman Commits: aa3a1bc3 by Ben Westover at 2023-08-28T22:25:10+10:00 proto: Change the default checksum from md5 to sha256 MD5 isn't a very good checksum, and the PKGBUILD page on the Arch Wiki states that it should not be used, instead recommending sha256 or b2. This patch changes the default from md5 to sha256 because that seems to be the most commonly used checksum today. Signed-off-by: Ben Westover <kwestover.kw(a)gmail.com> - - - - - 3 changed files: - proto/PKGBUILD-split.proto - proto/PKGBUILD-vcs.proto - proto/PKGBUILD.proto View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/aa3a1bc3b50d797fb75278f… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/aa3a1bc3b50d797fb75278f… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[Git][pacman/pacman][master] pacman: do not check file md5sums
by Allan McRae (＠allan) 28 Aug '23

28 Aug '23

Allan McRae pushed to branch master at Pacman / Pacman Commits: 298755c9 by Allan McRae at 2023-08-28T22:17:51+10:00 pacman: do not check file md5sums The file md5sums are removed from the .MTREE file, so pacman should not attempt to check them. Signed-off-by: Allan McRae <allan(a)archlinux.org> - - - - - 1 changed file: - src/pacman/check.c View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/298755c905fdd40f9611abe… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/298755c905fdd40f9611abe… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[Git][pacman/pacman][master] 3 commits: makepkg: immutable git sources by hashing the checkout content
by Allan McRae (＠allan) 28 Aug '23

28 Aug '23

Allan McRae pushed to branch master at Pacman / Pacman Commits: 2fc2ab6c by Levente Polyak at 2023-08-28T22:15:03+10:00 makepkg: immutable git sources by hashing the checkout content This feature makes Git VCS build inputs immutable by adding support for pinning a Git checkout by a hash of its content using the deterministic export functionality `git archive`. This feature aids packagers by allowing them to use simple and convenient refnames (instead of full commit hashes) in the `PKGBUILD` while still preserving security implications of immutable build inputs using a trusted cryptographic hash function of the content. Previously VCS source downloads have been skipped for `--geninteg` and `--source` as both options did not need a checkout. This commit changes this behavior by forcing the download of all sources as integrity checks and generation requires to have an up to date state. Signed-off-by: Levente Polyak <anthraxx(a)archlinux.org> - - - - - ca3c873d by Levente Polyak at 2023-08-28T22:15:03+10:00 makepkg: immutable mercurial sources by hashing the checkout content This feature makes Mercurial VCS build inputs immutable by adding support for pinning a Mercurial checkout by a hash of its content using the deterministic export functionality `hg archive`. This feature aids packagers by allowing them to use simple and convenient refnames (instead of full commit hashes) in the `PKGBUILD` while still preserving security implications of immutable build inputs using a trusted cryptographic hash function of the content. Signed-off-by: Levente Polyak <anthraxx(a)archlinux.org> - - - - - ee933acf by Levente Polyak at 2023-08-28T22:15:03+10:00 makepkg: immutable bzr by hashing the checkout content This feature makes bzr VCS build inputs immutable by adding support for pinning a bzr checkout by a hash of its content using the deterministic export functionality `bzr export`. This feature allows to preserve security implications of immutable build inputs using a trusted cryptographic hash function of the content. Signed-off-by: Levente Polyak <anthraxx(a)archlinux.org> - - - - - 8 changed files: - doc/PKGBUILD.5.asciidoc - scripts/libmakepkg/integrity/generate_checksum.sh.in - scripts/libmakepkg/integrity/verify_checksum.sh.in - scripts/libmakepkg/source/bzr.sh.in - scripts/libmakepkg/source/file.sh.in - scripts/libmakepkg/source/git.sh.in - scripts/libmakepkg/source/hg.sh.in - scripts/makepkg.sh.in View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/compare/843bf21e794c79c5b3bcf8… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/compare/843bf21e794c79c5b3bcf8… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[Git][pacman/pacman][master] libmakepkg: Fix non-reproducible binaries by processing debuginfo in order
by Allan McRae (＠allan) 28 Aug '23

28 Aug '23

Allan McRae pushed to branch master at Pacman / Pacman Commits: 843bf21e by kpcyrd at 2023-08-27T13:03:40+02:00 libmakepkg: Fix non-reproducible binaries by processing debuginfo in order - - - - - 1 changed file: - scripts/libmakepkg/tidy/strip.sh.in View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/843bf21e794c79c5b3bcf8a… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/843bf21e794c79c5b3bcf8a… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[PATCH] alpm_run_chroot: Fix the execution of pacman hooks
by Dragan Simic 21 Aug '23

21 Aug '23

During a full system update of a Manjaro ARM installation, on an ARM-based computer, of course, I spotted a highly conspicuous error message in the output of pacman: ( 3/18) Creating temporary files... Assertion 'fd' failed at src/tmpfiles/tmpfiles.c:843, function fd_set_perms(). Aborting. /usr/share/libalpm/scripts/systemd-hook: line 28: 1735 Aborted (core dumped) /usr/bin/systemd-tmpfiles --create error: command failed to execute correctly Here's also an excerpt from /var/log/pacman.log (the stack trace was rather unusable, so I'll skip it for the sake of brevity): [ALPM] running '30-systemd-tmpfiles.hook'... [ALPM-SCRIPTLET] Assertion 'fd' failed at src/tmpfiles/tmpfiles.c:843, function fd_set_perms(). Aborting. [ALPM-SCRIPTLET] /usr/share/libalpm/scripts/systemd-hook: line 28: 1735 Aborted (core dumped) /usr/bin/systemd-tmpfiles --create Running "systemd-tmpfiles --create" manually afterwards resulted in no errors, which made the error message even more conspicuous. Thus, executing /usr/share/libalpm/hooks/30-systemd-tmpfiles.hook failed, but only when it was run from within pacman. I also saw a few people complaining about the same error message on Manjaro and Arch Linux forums, even with one complaint dating more than a few years ago, but nobody came through with a fix or workaround. After a detailed and rather lengthy investigation, it turned out that the root cause was twofold, as described below: 1) For its pacman package, Arch Linux ARM applies a patch named 0003-Revert-alpm_run_chroot-always-connect-parent2child-p.patch that reverts rather old pacman commit 1d6583a5, for an unknown reason. This patch causes the error like clockwork. 2) The code in pacman's lib/libalpm/util.c that executes the hooks by forking a child has some rather subtle bugs that allow the error to occur under certain circumstances. Regarding the first point, the patch from Arch Linux ARM creates a condition in which the file descriptor 0 is closed by calling close(0) and left closed when the executed hook has no option "NeedsTargets" specified, which is the case for the hook mentioned above, /usr/share/libalpm/hooks/30-systemd-tmpfiles.hook. As a result, the first call to open() during the execution of the hook returns 0 as the file descriptor, because 0 is the lowest currently available value. It would all go unnoticed, but systemd-tmpfiles performed assert(fd) checks in its file src/tmpfiles/tmpfiles.c, which failed because fd equaled 0. These checks seem to have been removed in the meantime, which effectively made the error go away, but the original issue still remains. Regarding the second point, function _alpm_run_chroot() that executes hooks in a fork()ed child does not execute dup2() properly, but instead executes close() followed by dup2(). The man page for dup2() clearly states that such attempts to re-implement the equivalent functionality must be avoided, as visible in this quotation: The dup2() system call performs the same task as dup(), but instead of using the lowest-numbered unused file descriptor, it uses the file descriptor number specified in newfd. In other words, the file descriptor newfd is adjusted so that it now refers to the same open file description as oldfd. If the file descriptor newfd was previously open, it is closed before being reused; the close is performed silently (i.e., any errors during the close are not reported by dup2()). The steps of closing and reusing the file descriptor newfd are performed atomically. This is important, because trying to implement equivalent functionality using close(2) and dup() would be subject to race conditions, whereby newfd might be reused between the two steps. Such reuse could happen because the main program is interrupted by a signal handler that allocates a file descriptor, or because a parallel thread allocates a file descriptor. As a result, a condition can occur in which the file descriptor 0 is closed by calling close(0), and left closed after the while loop that fails to execute dup2() because of receiving EBUSY, resulting in the original issue. On top of that, failed attempts to execute dup2() should be treated as fatal errors instead of being silently ignored. Let's improve the code to prevent the issues described in the second point, while not applying the above-mentioned Arch Linux ARM package patch fixes the issues decribed in the first point. While there, perform a minor cleanup as well, to make the formatting of the code a tiny bit more consistent. Signed-off-by: Dragan Simic <dsimic(a)manjaro.org> --- lib/libalpm/util.c | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/lib/libalpm/util.c b/lib/libalpm/util.c index dffa3b51..97e87c6c 100644 --- a/lib/libalpm/util.c +++ b/lib/libalpm/util.c @@ -639,28 +639,39 @@ int _alpm_run_chroot(alpm_handle_t *handle, const char *cmd, char *const argv[], if(pid == 0) { /* this code runs for the child only (the actual chroot/exec) */ - close(0); - close(1); - close(2); - while(dup2(child2parent_pipefd[HEAD], 1) == -1 && errno == EINTR); - while(dup2(child2parent_pipefd[HEAD], 2) == -1 && errno == EINTR); - while(dup2(parent2child_pipefd[TAIL], 0) == -1 && errno == EINTR); - close(parent2child_pipefd[TAIL]); close(parent2child_pipefd[HEAD]); close(child2parent_pipefd[TAIL]); + while(dup2(child2parent_pipefd[HEAD], STDERR_FILENO) == -1) { + if(errno != EINTR) { + /* at this point, the child cannot talk through the parent */ + exit(1); + } + } + while(dup2(parent2child_pipefd[TAIL], STDIN_FILENO) == -1) { + if(errno != EINTR) { + /* use fprintf() instead of _alpm_log() to send output through the parent */ + fprintf(stderr, _("could not redirect standard input (%s)\n"), strerror(errno)); + exit(1); + } + } + close(parent2child_pipefd[TAIL]); + while(dup2(child2parent_pipefd[HEAD], STDOUT_FILENO) == -1) { + if(errno != EINTR) { + fprintf(stderr, _("could not redirect standard output (%s)\n"), strerror(errno)); + exit(1); + } + } close(child2parent_pipefd[HEAD]); if(cwdfd >= 0) { close(cwdfd); } - /* use fprintf instead of _alpm_log to send output through the parent */ if(chroot(handle->root) != 0) { fprintf(stderr, _("could not change the root directory (%s)\n"), strerror(errno)); exit(1); } if(chdir("/") != 0) { - fprintf(stderr, _("could not change directory to %s (%s)\n"), - "/", strerror(errno)); + fprintf(stderr, _("could not change directory to %s (%s)\n"), "/", strerror(errno)); exit(1); } /* bash assumes it's being run under rsh/ssh if stdin is a socket and -- 2.33.1

3 6

[PATCH v2] alpm_run_chroot: Fix the execution of pacman hooks
by Dragan Simic 20 Aug '23

20 Aug '23

Running "pacman -Syyu" to perform a full system update resulted in a rather conspicuous error message in the output of pacman: ( 3/18) Creating temporary files... Assertion 'fd' failed at src/tmpfiles/tmpfiles.c:843, function fd_set_perms(). Aborting. /usr/share/libalpm/scripts/systemd-hook: line 28: 1735 Aborted (core dumped) /usr/bin/systemd-tmpfiles --create error: command failed to execute correctly Expectedly, running "systemd-tmpfiles --create" manually afterwards went just fine and resulted in no errors, leading to a conclusion that executing /usr/share/libalpm/hooks/30-systemd-tmpfiles.hook failed, but only when it was run from within pacman. After a detailed and rather lengthy investigation, it turned out the code in lib/libalpm/util.c that executes the hooks by forking a child has some bugs that allow the error to occur under certain circumstances. In particular, function _alpm_run_chroot() that executes hooks in a fork()ed child does not employ dup2() properly, but instead executes close() followed by dup2(). The man page for dup2() clearly states in the quotation below that attempts to re-implement the equivalent functionality, which is the case in function _alpm_run_chroot(), must be avoided: The dup2() system call performs the same task as dup(), but instead of using the lowest-numbered unused file descriptor, it uses the file descriptor number specified in newfd. In other words, the file descriptor newfd is adjusted so that it now refers to the same open file description as oldfd. If the file descriptor newfd was previously open, it is closed before being reused; the close is performed silently (i.e., any errors during the close are not reported by dup2()). The steps of closing and reusing the file descriptor newfd are performed atomically. This is important, because trying to implement equivalent functionality using close(2) and dup() would be subject to race conditions, whereby newfd might be reused between the two steps. Such reuse could happen because the main program is interrupted by a signal handler that allocates a file descriptor, or because a parallel thread allocates a file descriptor. As a result, a condition can occur in which the file descriptor 0 is closed by calling close(0), and left closed after the while loop that fails to execute dup2() because of receiving EBUSY, resulting in the described issues. Also, failed attempts to execute dup2() should be treated as fatal errors instead of being silently ignored. Let's improve the code to prevent the described issues. While there, perform a minor cleanup as well, to make the formatting of the code a tiny bit more consistent. Signed-off-by: Dragan Simic <dsimic(a)manjaro.org> --- lib/libalpm/util.c | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/lib/libalpm/util.c b/lib/libalpm/util.c index dffa3b51..97e87c6c 100644 --- a/lib/libalpm/util.c +++ b/lib/libalpm/util.c @@ -639,28 +639,39 @@ int _alpm_run_chroot(alpm_handle_t *handle, const char *cmd, char *const argv[], if(pid == 0) { /* this code runs for the child only (the actual chroot/exec) */ - close(0); - close(1); - close(2); - while(dup2(child2parent_pipefd[HEAD], 1) == -1 && errno == EINTR); - while(dup2(child2parent_pipefd[HEAD], 2) == -1 && errno == EINTR); - while(dup2(parent2child_pipefd[TAIL], 0) == -1 && errno == EINTR); - close(parent2child_pipefd[TAIL]); close(parent2child_pipefd[HEAD]); close(child2parent_pipefd[TAIL]); + while(dup2(child2parent_pipefd[HEAD], STDERR_FILENO) == -1) { + if(errno != EINTR) { + /* at this point, the child cannot talk through the parent */ + exit(1); + } + } + while(dup2(parent2child_pipefd[TAIL], STDIN_FILENO) == -1) { + if(errno != EINTR) { + /* use fprintf() instead of _alpm_log() to send output through the parent */ + fprintf(stderr, _("could not redirect standard input (%s)\n"), strerror(errno)); + exit(1); + } + } + close(parent2child_pipefd[TAIL]); + while(dup2(child2parent_pipefd[HEAD], STDOUT_FILENO) == -1) { + if(errno != EINTR) { + fprintf(stderr, _("could not redirect standard output (%s)\n"), strerror(errno)); + exit(1); + } + } close(child2parent_pipefd[HEAD]); if(cwdfd >= 0) { close(cwdfd); } - /* use fprintf instead of _alpm_log to send output through the parent */ if(chroot(handle->root) != 0) { fprintf(stderr, _("could not change the root directory (%s)\n"), strerror(errno)); exit(1); } if(chdir("/") != 0) { - fprintf(stderr, _("could not change directory to %s (%s)\n"), - "/", strerror(errno)); + fprintf(stderr, _("could not change directory to %s (%s)\n"), "/", strerror(errno)); exit(1); } /* bash assumes it's being run under rsh/ssh if stdin is a socket and

1 0