[pacman-dev] [PATCH] makepkg: use builtin globbing to print files in package
- it comes with free collation when moving the LC_ALL declaration up a bit;
this fixes a bug where the .FILES were not being properly sorted and
their order depended on directory creation order, which broke
reproducible builds in the wild.
- it handles sorting null-delimited output everywhere, without sort -z;
this lets us get rid of sed hacks
- it is faster than invoking multiple find subprocesses
- dotfiles can be automatically printed *and the C locale sorts them first*
with a single ** glob
Signed-off-by: Eli Schwartz
On Tue, 21 Aug 2018 10:15:12 -0400, Eli Schwartz wrote:
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index ae1ef01b..1325b019 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -711,10 +711,14 @@ write_buildinfo() { # database files are placed at the beginning of the package regardless of # sorting list_package_files() { - (find . -path './.*' \! -name '.'; find . \! -path './.*' \! -name '.' | LC_ALL=C sort) | - sed -e 's|^\./||' | tr '\n' '\0' + ( + export LC_COLLATE=C + shopt -s dotglob globstar + printf '%s\0' ** + )
Since globbing is done in the same process, it should be sufficient to set LC_COLLATE; no need to export it. -- Happy hacking, ~ Luke Shumaker
On 8/21/18 11:41 AM, Luke Shumaker wrote:
On Tue, 21 Aug 2018 10:15:12 -0400, Eli Schwartz wrote:
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index ae1ef01b..1325b019 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -711,10 +711,14 @@ write_buildinfo() { # database files are placed at the beginning of the package regardless of # sorting list_package_files() { - (find . -path './.*' \! -name '.'; find . \! -path './.*' \! -name '.' | LC_ALL=C sort) | - sed -e 's|^\./||' | tr '\n' '\0' + ( + export LC_COLLATE=C + shopt -s dotglob globstar + printf '%s\0' ** + )
Since globbing is done in the same process, it should be sufficient to set LC_COLLATE; no need to export it.
I guess.... My initial version used LC_COLLATE without globstar and just replaced the first find command with printf, so it needed to be exported to sort as well. Then dreisner pointed out using globstar could totally drop the find subprocess. -- Eli Schwartz Bug Wrangler and Trusted User
On 08/21/18 at 10:15am, Eli Schwartz wrote:
- dotfiles can be automatically printed *and the C locale sorts them first* with a single ** glob
The glob will sort dotfiles before alphanumeric names, but there are several characters that will still sort before a period.
On 8/21/18 11:53 AM, Andrew Gregory wrote:
On 08/21/18 at 10:15am, Eli Schwartz wrote:
- dotfiles can be automatically printed *and the C locale sorts them first* with a single ** glob
The glob will sort dotfiles before alphanumeric names, but there are several characters that will still sort before a period.
Eh. In the wise words of dreisner (when I asked if I should amend it to fix that): yes technically if someone were to create a root directory starting with ! " # $ & % ' ( ) * + , - it would sort before the dotfiles the day someone does that, i'll perfect my implementation of stabbing people over standard tcp/ip connections. I rather agree, since none of those filenames are remotely sane, at least not as direct children of / -- which is why this matters, since our sorting goal is IIRC just to ensure our metadata dotfiles are the most efficient files to extract. If that fails... oh well, some crazy person's package will have somewhat less efficiently extracted metadata. Although, this could be fixed I guess with: printf '%s\0' .!(|.); printf '%s\0' !(.*) */** | sort -uz -- Eli Schwartz Bug Wrangler and Trusted User
participants (3)
-
Andrew Gregory
-
Eli Schwartz
-
Luke Shumaker