- Pacman-dev - lists.archlinux.org

[pacman-dev] [PATCH] makepkg: respect $SOURCE_DATE_EPOCH to activate reproducible builds
by Eli Schwartz 13 Sep '17

13 Sep '17

If SOURCE_DATE_EPOCH is set, `touch` all source files before running build() to fix the modification times. This works around build systems and compilers that embed the file modification times into the file contents of release artifacts. Signed-off-by: Eli Schwartz <eschwartz93(a)gmail.com> --- Guarded by checking for the variable, because this stomps all over incremental builds. I prefer looking for SOURCE_DATE_EPOCH in the environment because less bloat IMHO, but perhaps this would be preferable as a makepkg flag or makepkg.conf option? Is there anything else that needs to be done to force reproducible builds, other than merely setting SOURCE_DATE_EPOCH which should be harmless as a global thing? scripts/makepkg.sh.in | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 20e9dd7e..fb1d40a9 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -79,15 +79,22 @@ PKGFUNC=0 PKGVERFUNC=0 PREPAREFUNC=0 REPKG=0 +REPRODUCIBLE=0 RMDEPS=0 SKIPCHECKSUMS=0 SKIPPGPCHECK=0 SIGNPKG='' SPLITPKG=0 SOURCEONLY=0 + VERIFYSOURCE=0 -export SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH:-$(date +%s)} +if [[ -n $SOURCE_DATE_EPOCH ]]; then + REPRODUCIBLE=1 +else + SOURCE_DATE_EPOCH=$(date +%s) +fi +export SOURCE_DATE_EPOCH PACMAN_OPTS=() @@ -475,6 +482,12 @@ run_prepare() { } run_build() { + if (( REPRODUCIBLE )); then + # We have activated reproducible builds, so unify source times before + # building + find "$srcdir" -exec touch -h -d @$SOURCE_DATE_EPOCH {} + + fi + run_function_safe "build" } -- 2.13.2

4 7

[pacman-dev] [PATCH] libmakepkg/integrity: check for invalid tags
by Eli Schwartz 13 Sep '17

13 Sep '17

As per https://lists.archlinux.org/pipermail/arch-general/2017-July/043876.html git doesn't check that the tag name matches what an annotated tag object *thinks* it should be called. This is a bit of a theoretical attack and some would argue that we should always use commits since upstream can legitimately change a tag, but nevertheless this can result in a downgrade attack if the git download transport was manipulated. So, check the tag blob to make sure the tag actually matches the name we used for `git checkout` Signed-off-by: Eli Schwartz <eschwartz93(a)gmail.com> --- scripts/libmakepkg/integrity/verify_signature.sh.in | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/scripts/libmakepkg/integrity/verify_signature.sh.in b/scripts/libmakepkg/integrity/verify_signature.sh.in index 5468f977..3783dbb2 100644 --- a/scripts/libmakepkg/integrity/verify_signature.sh.in +++ b/scripts/libmakepkg/integrity/verify_signature.sh.in @@ -187,6 +187,13 @@ verify_git_signature() { printf " %s git repo ... " "${dir##*/}" >&2 + tagname="$(git -C "$dir" cat-file tag "$fragval" 2>/dev/null | awk 'FNR == 3 {print $2}')" + if [[ $fragtype = tag && -n $tagname && $tagname != $fragval ]]; then + printf "%s (%s)" "$(gettext "FAILED")" "$(gettext "forged tag, you have been hacked!")" >&2 + errors=1 + return 1 + fi + git -C "$dir" verify-$fragtype --raw "$fragval" > "$statusfile" 2>&1 if ! grep -qs NEWSIG "$statusfile"; then printf '%s\n' "$(gettext "SIGNATURE NOT FOUND")" >&2 -- 2.13.2

3 7

[pacman-dev] Request
by Shameem Fairooz 12 Aug '17

12 Aug '17

Hi I'm shameem, newbie. I've nailed down the basics of shell scripting. I'd like to contribute to pacman and other official projects. Any other skills that you'd recommend me to learn along with the process ? Thank You.

2 1

[pacman-dev] Variable execution time for pacman -Qdt?
by Will S 08 Aug '17

08 Aug '17

I have noticed that the first time I run pacman -Qdt after not running it for a while it can take a variable amount of time -- usually at least 20 seconds and as long as one minute. If I run it again shortly after that it completes in less than a second. Is there a simple explanation for this variation? I have 1705 packages installed and pacman -Qdt lists 26. The reason I'd like it to be fast is that yaourt calls this command before a system update, so I get a 20-60 second lag before being prompted for my password for sudo pacman -Syu.

2 1

[pacman-dev] [aur-general] Handling git submodules (Was: Re: TU Application: Dan Printzell)
by Eli Schwartz 31 Jul '17

31 Jul '17

On 07/31/2017 03:35 PM, Doug Newgard wrote: > On Mon, 31 Jul 2017 14:16:33 -0400 > Eli Schwartz <eschwartz(a)archlinux.org> wrote: >> You're both right and wrong. "$srcdir/$submodulename" will be at >> origin/master, but `git submodule update` in the primary repo will >> completely ignore that clone altogether and just cares about the git >> objects which it uses in yet another clone. >> > > This is a bit confusing. You set the URL of the submodule to the location of > the local clone, so it doesn't ignore it, it just clones from the local clone. > And yes, it does use the specific commit specified in the main repo, so setting > #commit= for the submodule is useless. Yeah, that. Although now I am starting to think... I don't have any PKGBUILDs that do this myself. So it's never really been on my radar. However, I am wondering if it would make more sense to allow/use: - $SRCDEST/$repo for the submodule url. The variable should be accessible, we just don't document its availability for use in a PKGBUILD. - Implement noextract=() for source/git.sh to avoid having two checkouts with that ugly layer of indirection. - Update the "VCS package guidelines" wiki page -- Eli Schwartz

1 0

[pacman-dev] Repository management
by Allan McRae 30 Jul '17

30 Jul '17

Hi all, Every time I attempt to work on repo-add, I find it to be a very difficult endeavour. Even though it is half the size of makepkg (without even including any of libmakepkg), it is much more convoluted to work on. We also have a weird repository database system. We have: - .db dbs with package information, signatures and delta information - .files dbs that are the same as .db dbs but additionally include filelists There are two reasons the .files dbs replicate all information in the .db dbs - .db and .files dbs getting out of sync could cause issues - a complete database is useful for things like archweb, mostly to avoid the above I would also like to include information on source packages to these databases. The files information is separate due to wanting our primary database to be small. Likewise, source package information needs to be separate (the signatures take most of the size in the .db dbs, so adding source package signatures effectively doubles the size). So two points up for discussion: 1) Sync repository layout? I don't see any point in leaving the tar based format, as reading of sync databases is not a bottleneck. (The local db format can be a bottleneck, but that is a separate discussion...) Do we split the information in .db out of .files and add a .full db with complete information? Then any .src db could follow suit and just have source package information. How do we get around the out of sync issue (e.g., a package is removed from .db, but we have an old .files database with it). Do we add timestamps, and print a warning on -F operations when the two are out of sync? 2) Do we need a better (read "more easily maintainable") tool for handling database generation and updates? libalpm already can read in information package files, so we could add libalpm/db_write.c with the database creation functions. Should we unify our repo format with our local database format which we already write? I am looking for ideas here. Please brainstorm to your hearts content. Cheers, Allan

5 7

[pacman-dev] [PATCH] Fix coding style issue in pacman/util.c
by Michael Straube 28 Jul '17

28 Jul '17

Add missing braces to some one-line if blocks. Signed-off-by: Michael Straube <straubem(a)gmx.de> --- src/pacman/util.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/src/pacman/util.c b/src/pacman/util.c index ae8a74d3..9784d627 100644 --- a/src/pacman/util.c +++ b/src/pacman/util.c @@ -1299,8 +1299,9 @@ void select_display(const alpm_list_t *pkglist) alpm_pkg_t *pkg = i->data; alpm_db_t *db = alpm_pkg_get_db(pkg); - if(!dbname) + if(!dbname) { dbname = alpm_db_get_name(db); + } if(strcmp(alpm_db_get_name(db), dbname) != 0) { display_repo_list(dbname, list, cols); FREELIST(list); @@ -1349,8 +1350,9 @@ static int multiselect_parse(char *array, int count, char *response) break; } len = strtrim(starts); - if(len == 0) + if(len == 0) { continue; + } if(*starts == '^') { starts++; @@ -1370,8 +1372,9 @@ static int multiselect_parse(char *array, int count, char *response) } } - if(parseindex(starts, &start, 1, count) != 0) + if(parseindex(starts, &start, 1, count) != 0) { return -1; + } if(!ends) { array[start - 1] = include; @@ -1495,8 +1498,9 @@ int select_question(int count) size_t len = strtrim(response); if(len > 0) { int n; - if(parseindex(response, &n, 1, count) != 0) + if(parseindex(response, &n, 1, count) != 0) { continue; + } return (n - 1); } } -- 2.13.2

2 1

[pacman-dev] Outdated comments in pacman.conf
by Andrey Azarov 16 Jul '17

16 Jul '17

Hello! I found what comments in ls /etc/pacman.conf about signature verification is out of date: # By default, pacman accepts packages signed by keys that its local keyring > # trusts (see pacman-key and its man page), as well as unsigned packages. > SigLevel = Required DatabaseOptional > LocalFileSigLevel = Optional > #RemoteFileSigLevel = Required So I propose following changes to make comments consistent with default settings: # By default, pacman accepts only packages signed by keys that its local > keyring trusts (see pacman-key and its man page) > # The only exclusion is packages from local filesystem, such packages can > be unsigned > SigLevel = Required DatabaseOptional > LocalFileSigLevel = Optional > #RemoteFileSigLevel = Required I include svn patch in attachments. With best wishes, Andrey

2 1

[pacman-dev] [PATCH v2] Fix brace coding style issues
by Michael Straube 15 Jul '17

15 Jul '17

- Add missing braces to one-line if blocks - Move opening brace to line end Signed-off-by: Michael Straube <straubem(a)gmx.de> --- lib/libalpm/delta.c | 3 +-- lib/libalpm/sync.c | 6 ++++-- src/pacman/util.c | 12 ++++++++---- 3 files changed, 13 insertions(+), 8 deletions(-) diff --git a/lib/libalpm/delta.c b/lib/libalpm/delta.c index 89bc32ff..0138aac1 100644 --- a/lib/libalpm/delta.c +++ b/lib/libalpm/delta.c @@ -238,8 +238,7 @@ static alpm_list_t *find_unused(alpm_list_t *deltas, const char *to, off_t quota for(i = vertices; i; i = i->next) { alpm_graph_t *v = i->data; alpm_delta_t *vdelta = v->data; - if(strcmp(vdelta->to, to) == 0) - { + if(strcmp(vdelta->to, to) == 0) { v->weight = vdelta->download_size; } } diff --git a/lib/libalpm/sync.c b/lib/libalpm/sync.c index ae0f1e49..73120b63 100644 --- a/lib/libalpm/sync.c +++ b/lib/libalpm/sync.c @@ -271,8 +271,9 @@ alpm_list_t SYMEXPORT *alpm_find_group_pkgs(alpm_list_t *dbs, alpm_db_t *db = i->data; alpm_group_t *grp = alpm_db_get_group(db, name); - if(!grp) + if(!grp) { continue; + } for(j = grp->packages; j; j = j->next) { alpm_pkg_t *pkg = j->data; @@ -288,8 +289,9 @@ alpm_list_t SYMEXPORT *alpm_find_group_pkgs(alpm_list_t *dbs, }; ignorelist = alpm_list_add(ignorelist, pkg); QUESTION(db->handle, &question); - if(!question.install) + if(!question.install) { continue; + } } if(!alpm_pkg_find(pkgs, pkg->name)) { pkgs = alpm_list_add(pkgs, pkg); diff --git a/src/pacman/util.c b/src/pacman/util.c index ae8a74d3..9784d627 100644 --- a/src/pacman/util.c +++ b/src/pacman/util.c @@ -1299,8 +1299,9 @@ void select_display(const alpm_list_t *pkglist) alpm_pkg_t *pkg = i->data; alpm_db_t *db = alpm_pkg_get_db(pkg); - if(!dbname) + if(!dbname) { dbname = alpm_db_get_name(db); + } if(strcmp(alpm_db_get_name(db), dbname) != 0) { display_repo_list(dbname, list, cols); FREELIST(list); @@ -1349,8 +1350,9 @@ static int multiselect_parse(char *array, int count, char *response) break; } len = strtrim(starts); - if(len == 0) + if(len == 0) { continue; + } if(*starts == '^') { starts++; @@ -1370,8 +1372,9 @@ static int multiselect_parse(char *array, int count, char *response) } } - if(parseindex(starts, &start, 1, count) != 0) + if(parseindex(starts, &start, 1, count) != 0) { return -1; + } if(!ends) { array[start - 1] = include; @@ -1495,8 +1498,9 @@ int select_question(int count) size_t len = strtrim(response); if(len > 0) { int n; - if(parseindex(response, &n, 1, count) != 0) + if(parseindex(response, &n, 1, count) != 0) { continue; + } return (n - 1); } } -- 2.13.3

1 0

[pacman-dev] [PATCH] test/pacman: fix spelling of distribution name and another typo
by Michael Straube 09 Jul '17

09 Jul '17

Signed-off-by: Michael Straube <straubem(a)gmx.de> --- test/pacman/ChangeLog | 2 +- test/pacman/README | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/test/pacman/ChangeLog b/test/pacman/ChangeLog index fec0eb23..fed468d0 100644 --- a/test/pacman/ChangeLog +++ b/test/pacman/ChangeLog @@ -25,5 +25,5 @@ Release 0.1 (06/01/30) ----------- Initial release. -Annoucement on the ArchLinux pacman-dev mailing list: +Announcement on the Arch Linux pacman-dev mailing list: http://www.archlinux.org/pipermail/pacman-dev/2006-January/000186.html diff --git a/test/pacman/README b/test/pacman/README index a5fdeaad..224acd84 100644 --- a/test/pacman/README +++ b/test/pacman/README @@ -1,7 +1,7 @@ README ====== -pactest is a test suite for the ArchLinux package manager: pacman. +pactest is a test suite for the Arch Linux package manager: pacman. It has a rather high level view of operations performed by pacman: it automatically creates a test environment based on a test case file @@ -9,7 +9,7 @@ description, the run pacman, and finally check the results of test according to a set of rules defined in the test case. It is written in Python and makes available most of what can be found in -pacman's code to create ArchLinux packages or read and write databases entries. +pacman's code to create Arch Linux packages or read and write databases entries. Each test case is defined in a separate file that is sourced in order to set the environment. @@ -149,7 +149,7 @@ Example: pkg1 = pmpkg("dummy", "2.1-1") pkg2 = pmpkg("foobar") -All fields from a ArchLinux package can be set and modified directly with no +All fields from a Arch Linux package can be set and modified directly with no methods to access them. Note: some fields are automatically set by pactest and should preferably not be modified by hand (i.e. "md5sum", "size", or "csize"). -- 2.13.2

1 0