July 2020 - Pacman-dev - lists.archlinux.org

[pacman-dev] [PATCH] Check that destfile_name exists before using it
by Anatol Pomozov 14 Jul '20

14 Jul '20

In some cases (when trust_remote_name is used for a URL without a filename and no Content-Disposition is provided by the server) destfile_name will be NULL. In this case payload data will be stored in tempfile_name and no destfile_name is set. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> --- lib/libalpm/dload.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 343f5c78..673e769f 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -897,15 +897,18 @@ int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls, for(i = payloads; i; i = i->next) { struct dload_payload *payload = i->data; - const char *filename; char *filepath; if(payload->signature) { continue; } - filename = mbasename(payload->destfile_name); - filepath = _alpm_filecache_find(handle, filename); + if(payload->destfile_name) { + const char *filename = mbasename(payload->destfile_name); + filepath = _alpm_filecache_find(handle, filename); + } else { + STRDUP(filepath, payload->tempfile_name, GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + } if(filepath) { alpm_list_append(fetched, filepath); } else { -- 2.27.0

2 2

[pacman-dev] [PATCH] Build signature remote name based on the main payload name
by Anatol Pomozov 14 Jul '20

14 Jul '20

The main payload final name might be affected by url redirects or Content-Disposition HTTP header value. We want to make sure that accompanion *.sig filename always matches the package filename. So ignore finalname/Content-Disposition for the *.sig file. It also helps to fix a corner case when the download URL does not contain a filename and server provides Content-Disposition for the main payload but not for the signature payload. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> --- lib/libalpm/dload.c | 61 +++++++++++++++++++++++++++------------------ 1 file changed, 37 insertions(+), 24 deletions(-) diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 78808eb3..1785dd6a 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -481,6 +481,31 @@ static int curl_check_finished_download(CURLM *curlm, CURLMsg *msg, curl_easy_getinfo(curl, CURLINFO_CONDITION_UNMET, &timecond); curl_easy_getinfo(curl, CURLINFO_EFFECTIVE_URL, &effective_url); + if(payload->trust_remote_name) { + if(payload->content_disp_name) { + /* content-disposition header has a better name for our file */ + free(payload->destfile_name); + payload->destfile_name = get_fullpath(localpath, + get_filename(payload->content_disp_name), ""); + } else { + const char *effective_filename = strrchr(effective_url, '/'); + + if(effective_filename && strlen(effective_filename) > 2) { + effective_filename++; + + /* if destfile was never set, we wrote to a tempfile. even if destfile is + * set, we may have followed some redirects and the effective url may + * have a better suggestion as to what to name our file. in either case, + * refactor destfile to this newly derived name. */ + if(!payload->destfile_name || strcmp(effective_filename, + strrchr(payload->destfile_name, '/') + 1) != 0) { + free(payload->destfile_name); + payload->destfile_name = get_fullpath(localpath, effective_filename, ""); + } + } + } + } + /* Let's check if client requested downloading accompanion *.sig file */ if(!payload->signature && payload->download_signature && curlerr == CURLE_OK && payload->respcode < 400) { struct dload_payload *sig = NULL; @@ -489,6 +514,18 @@ static int curl_check_finished_download(CURLM *curlm, CURLMsg *msg, CALLOC(sig, 1, sizeof(*sig), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); MALLOC(sig->fileurl, len, FREE(sig); GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); snprintf(sig->fileurl, len, "%s.sig", effective_url); + + if(payload->trust_remote_name) { + /* In this case server might provide a new name for the main payload. + * Choose *.sig filename based on this new name. + */ + const char* realname = payload->destfile_name ? payload->destfile_name : payload->tempfile_name; + const char *final_file = get_filename(realname); + int remote_name_len = strlen(final_file) + 5; + MALLOC(sig->remote_name, remote_name_len, FREE(sig->fileurl); FREE(sig); GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); + snprintf(sig->remote_name, remote_name_len, "%s.sig", final_file); + } + sig->signature = 1; sig->handle = handle; sig->force = payload->force; @@ -520,30 +557,6 @@ static int curl_check_finished_download(CURLM *curlm, CURLMsg *msg, GOTO_ERR(handle, ALPM_ERR_RETRIEVE, cleanup); } - if(payload->trust_remote_name) { - if(payload->content_disp_name) { - /* content-disposition header has a better name for our file */ - free(payload->destfile_name); - payload->destfile_name = get_fullpath(localpath, - get_filename(payload->content_disp_name), ""); - } else { - const char *effective_filename = strrchr(effective_url, '/'); - if(effective_filename && strlen(effective_filename) > 2) { - effective_filename++; - - /* if destfile was never set, we wrote to a tempfile. even if destfile is - * set, we may have followed some redirects and the effective url may - * have a better suggestion as to what to name our file. in either case, - * refactor destfile to this newly derived name. */ - if(!payload->destfile_name || strcmp(effective_filename, - strrchr(payload->destfile_name, '/') + 1) != 0) { - free(payload->destfile_name); - payload->destfile_name = get_fullpath(localpath, effective_filename, ""); - } - } - } - } - ret = 0; cleanup: -- 2.27.0

2 1

[pacman-dev] [PATCH] Do not free payload fields in the middle of this structure use
by Anatol Pomozov 13 Jul '20

13 Jul '20

At the end of payload use it calls _alpm_dload_payload_reset() that will free() these and other fields anyway. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> --- lib/libalpm/dload.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 1785dd6a..343f5c78 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -698,10 +698,6 @@ static int curl_add_payload(alpm_handle_t *handle, CURLM *curlm, return 0; cleanup: - FREE(payload->fileurl); - FREE(payload->tempfile_name); - FREE(payload->destfile_name); - FREE(payload->content_disp_name); curl_easy_cleanup(curl); return ret; } -- 2.27.0

1 0

[pacman-dev] Errors in pacman man pages
by Helge Kreutzmann 09 Jul '20

09 Jul '20

Dear pacman maintainer, the manpage-l10n project maintains a large number of translations of man pages both from a large variety of sources (including packman) as well for a large variety of target languages. During their work translators notice different possible issues in the original (english) man pages. Sometimes this is a straightforward typo, sometimes a hard to read sentence, sometimes this is a convention not held up and sometimes we simply do not understand the original. We use several distributions as sources and update regularly (at least every 2 month). This means we are fairly recent (some distributions like archlinux also update frequently) but might miss the latest upstream version once in a while, so the error might be already fixed. We apologize and ask you to close the issue immediately if this should be the case, but given the huge volume of projects and the very limited number of volunteers we are not able to double check each and every issue. Secondly we translators see the manpages in the neutral po format, i.e. converted and harmonized, but not the original source (be it man, groff, xml or other). So we cannot provide a true patch (where possible), but only an approximation which you need to convert into your source format. Finally the issues I'm reporting have accumulated over time and are not always discovered by me, so sometimes my description of the problem my be a bit limited - do not hesitate to ask so we can clarify them. I'm now reporting the errors for your project. If future reports should use another channel, please let me know. Man page: makepkg-template.1 Issue: "name" and "version" should be I<> as they are options of the program "Currently used keys are: name (mandatory) and version\\&. Template names are " "limited to alphanumerics, \"@\", \"+\", \"\\&.\", \"-\", and \"_\"\\&. " "Versions are limited to numbers and \"\\&.\"\\&." -- Man page: pacman.8 Issue: This and other options' parameters could be wrapped in 'I<' and '>' "I<pacman> E<lt>operationE<gt> [options] [targets]" -- Man page: pacman.8 Issue: Curly double quotes in some strings, others regular quotes. Uniformize. "Since version 3\\&.0\\&.0, pacman has been the front-end to B<libalpm>(3), " "the \\(lqArch Linux Package Management\\(rq library\\&. This library allows " "alternative front-ends to be written (for instance, a GUI front-end)\\&." -- Man page: pacman.8 Issue: 'Search' 'owns' 'group' 'list' 'check' = option-related words, wrap around with "I<--" and ">" (or "B<--" and ">"?) "Show less information for certain query operations\\&. This is useful when " "pacman\\(cqs output is processed in a script\\&. Search will only show " "package names and not version, group, and description information; owns will " "only show package names instead of \"file is owned by pkg\" messages; group " "will only show package names and omit group names; list will only show files " "and omit package names; check will only show pairs of package names and " "missing files; a bare query will only show package names rather than names " -- Man page: pacman.8 Issue: 'Search' 'group' 'list' = option-related words, wrap around with "I<--" and ">" "Show less information for certain sync operations\\&. This is useful when " "pacman\\(cqs output is processed in a script\\&. Search will only show " "package names and not repository, version, group, and description " "information; list will only show package names and omit databases and " "versions; group will only show package names and omit group names\\&." -- Man page: pacman.conf.5 Issue: wget → B<wget>(1) "If set, an external program will be used to download all remote files\\&. " "All instances of %u will be replaced with the download URL\\&. If present, " "instances of %o will be replaced with the local filename, plus a \\(lq\\&." "part\\(rq extension, which allows programs like wget to do file resumes " "properly\\&." "This option is useful for users who experience problems with built-in HTTP/" "FTP support, or need the more advanced proxy support that comes with " "utilities like wget\\&." -- Man page: pacman.conf.5 Issue: Incorrect markup of -U: "" → B<> "Set the signature verification level for installing packages using the \"-U" "\" operation on a local file\\&. Uses the value from SigLevel as the default" "\\&." -- Man page: pacman.conf.5 Issue: Missing markup of syslog(): syslog() → B<syslog>() "Log action messages through syslog()\\&. This will insert log entries into /" "var/log/messages or equivalent\\&." -- Man page: pacman.conf.5 Issue: Where did "%o" come from? The XferCommand command usage does not have an %o "If set, an external program will be used to download all remote files\\&. " "All instances of %u will be replaced with the download URL\\&. If present, " "instances of %o will be replaced with the local filename, plus a \\(lq\\&." "part\\(rq extension, which allows programs like wget to do file resumes " "properly\\&." -- Man page: pacman.conf.5 Issue: [options] should not be translated, wrap it with B<> "The I<SigLevel> directive is valid in both the [options] and repository " "sections\\&. If used in [options], it sets a default value for any " "repository that does not provide the setting\\&." -- Man page: pacman-conf.8 Issue: remote → repository "B<-r, --repo> E<lt>remoteE<gt>" -- Dr. Helge Kreutzmann debian(a)helgefjell.de Dipl.-Phys. http://www.helgefjell.de/debian.php 64bit GNU powered gpg signed mail preferred Help keep free software "libre": http://www.ffii.de/

2 2

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.2.1-105-gb01bcc7d
by Allan McRae 07 Jul '20

07 Jul '20

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via b01bcc7d3d680856bd60c4ae03e4ba3f6d889cb2 (commit) via f3dfba73d22b7eca3810a8114f2aab63da488b4c (commit) via 78d6dcec6c49bd2fa830237a46fd14337bc9fd4c (commit) via 34ba8d984d89393ab85ca67724b87af67ff004c3 (commit) via f078c2d3bcb72bafda0dce5fe2c9418ca462bb1a (commit) from 6b9c1b4d54225b4c2808b5fadc2b6e779ae1916a (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit b01bcc7d3d680856bd60c4ae03e4ba3f6d889cb2 Author: Anatol Pomozov <anatol.pomozov(a)gmail.com> Date: Tue May 26 19:12:08 2020 -0700 Fallback to detached signatures during keyring check Pacman has a 'key in keyring' verification step that makes sure the signatures have a valid keyid. Currently pacman parses embedded package signatures only. Add a fallback to detached signatures. If embedded signature is missing then it tries to read corresponding *.sig file and get keyid from there. Verification: debug: found cached pkg: /var/cache/pacman/pkg/glib-networking-2.64.3-1-x86_64.pkg.tar.zst debug: found detached signature /var/cache/pacman/pkg/glib-networking-2.64.3-1-x86_64.pkg.tar.zst.sig with size 310 debug: found signature key: A5E9288C4FA415FA debug: looking up key A5E9288C4FA415FA locally debug: key lookup success, key exists Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit f3dfba73d22b7eca3810a8114f2aab63da488b4c Author: Anatol Pomozov <anatol.pomozov(a)gmail.com> Date: Mon May 18 14:30:04 2020 -0700 FS#33992: force download *.sig file if it does not exist in the cache In case if *.pkg exists but *.sig file does not we still have to pass the pkg to multi_download API. To avoid redownloading *.pkg file we use CURLOPT_TIMECONDITION curl option. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 78d6dcec6c49bd2fa830237a46fd14337bc9fd4c Author: Anatol Pomozov <anatol.pomozov(a)gmail.com> Date: Thu Apr 23 17:28:43 2020 -0700 Add a utility function to check whether a file exists in the cache It is similar to _alpm_filecache_find() but does not return a dynamically allocated memory to user. Thus the user does not need to free this resource. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 34ba8d984d89393ab85ca67724b87af67ff004c3 Author: Anatol Pomozov <anatol.pomozov(a)gmail.com> Date: Mon May 18 14:13:11 2020 -0700 Do not use counter for error tracking Current code uses an incrementing counter to check whether a function returned error: errors += some_function(); if(errors) { goto finish } Replace with a more standard variable errors = some_function(); if(errors) { goto finish } Rename 'errors' variable to a more typical 'ret'. Avoid reporting both ALPM_EVENT_PKG_RETRIEVE_FAILED and ALPM_EVENT_PKG_RETRIEVE_DONE in the error path. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit f078c2d3bcb72bafda0dce5fe2c9418ca462bb1a Author: Anatol Pomozov <anatol.pomozov(a)gmail.com> Date: Tue May 12 15:26:38 2020 -0700 Move signature payload creation to download engine Until now callee of ALPM download functionality has been in charge of payload creation both for the main file (e.g. *.pkg) and for the accompanied *.sig file. One advantage of such solution is that all payloads are independent and can be fetched in parallel thus exploiting the maximum level of download parallelism. To build *.sig file url we've been using a simple string concatenation: $requested_url + ".sig". Unfortunately there are cases when it does not work. For example an archlinux.org "Download From Mirror" link looks like this https://www.archlinux.org/packages/core/x86_64/bash/download/ and it gets redirected to some mirror. But if we append ".sig" to the end of the link url and try to download it then archlinux.org returns 404 error. To overcome this issue we need to follow redirects for the main payload first, find the final url and only then append '.sig' suffix. This implies 2 things: - the signature payload initialization need to be moved to dload.c as it is the place where we have access to the resolved url - *.sig is downloaded serially with the main payload and this reduces level of parallelism Move *.sig payload creation to dload.c. Once the main payload is fetched successfully we check if the callee asked to download the accompanied signature. If yes - create a new payload and add it to mcurl. *.sig payload does not use server list of the main payload and thus does not support mirror failover. *.sig file comes from the same server as the main payload. Refactor event loop in curl_multi_download_internal() a bit. Instead of relying on curl_multi_check_finished_download() to return number of new payloads we simply rerun the loop iteration one more time to check if there are any active downloads left. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> Signed-off-by: Allan McRae <allan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: lib/libalpm/alpm.h | 11 ++++++ lib/libalpm/be_sync.c | 34 ++++-------------- lib/libalpm/dload.c | 95 ++++++++++++++++++++++++++++++++------------------- lib/libalpm/dload.h | 4 ++- lib/libalpm/package.c | 40 ++++++++++++++++++++++ lib/libalpm/sync.c | 67 ++++++++++++++++++++++-------------- lib/libalpm/util.c | 51 +++++++++++++++++++++++++++ lib/libalpm/util.h | 3 ++ 8 files changed, 215 insertions(+), 90 deletions(-) hooks/post-receive -- The official pacman repository

1 0