- Pacman-dev - lists.archlinux.org

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.2.1-100-g6b9c1b4d
by Allan McRae 26 Jun '20

26 Jun '20

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 6b9c1b4d54225b4c2808b5fadc2b6e779ae1916a (commit) via 84723cab5dfc9b7f4594135295974f771ceb6e5e (commit) via 16d98d657748fdbf32ab24db56d3cd4a23447673 (commit) from 59e751f72d09390067045168ac45a09a89419389 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 6b9c1b4d54225b4c2808b5fadc2b6e779ae1916a Author: Denton Liu <liu.denton(a)gmail.com> Date: Thu Jun 25 19:29:40 2020 -0400 srcinfo.sh: remove trailing newline When a .SRCINFO file is generated via `makepkg --printsrcinfo`, each section is concluded with an empty line. This means that at the end of the file, an empty line remains. This is considered a trailing whitespace error. In fact, `git diff --check` will warn about this, saying "new blank line at EOF." Instead of closing each section off with an empty line, use the empty line to separate sections, omitting the empty line at the end of the file. Signed-off-by: Denton Liu <liu.denton(a)gmail.com> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 84723cab5dfc9b7f4594135295974f771ceb6e5e Author: Anatol Pomozov <anatol.pomozov(a)gmail.com> Date: Mon Apr 20 01:42:53 2020 -0700 Cleanup the old sequential download code All users of _alpm_download() have been refactored to the new API. It is time to remove the old _alpm_download() functionality now. This change also removes obsolete SIGPIPE signal handler functionality (this is a leftover from libfetch days). Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> commit 16d98d657748fdbf32ab24db56d3cd4a23447673 Author: Anatol Pomozov <anatol.pomozov(a)gmail.com> Date: Sun Apr 19 02:12:01 2020 -0700 Convert '-U pkg1 pkg2' codepath to parallel download Installing remote packages using its URL is an interesting case for ALPM API. Unlike package sync ('pacman -S pkg1 pkg2') '-U' does not deal with server mirror list. Thus _alpm_multi_download() should be able to handle file download for payloads that either have 'fileurl' field or pair of fields ('servers' and 'filepath') set. Signature for alpm_fetch_pkgurl() has changed and it accepts an output list that is populated with filepaths to fetched packages. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> ----------------------------------------------------------------------- Summary of changes: lib/libalpm/alpm.h | 15 +- lib/libalpm/be_sync.c | 2 +- lib/libalpm/dload.c | 552 ++++++++++----------------------------- lib/libalpm/dload.h | 11 +- lib/libalpm/sync.c | 2 +- scripts/libmakepkg/srcinfo.sh.in | 5 +- src/pacman/upgrade.c | 102 ++++---- 7 files changed, 204 insertions(+), 485 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [GIT] The official pacman repository annotated tag, v5.2.2, created. v5.2.2
by Andrew Gregory 26 Jun '20

26 Jun '20

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The annotated tag, v5.2.2 has been created at c95ee2ab01a828cb55232ddffd25570610c1af39 (tag) tagging 5537881b2525a4f114fcf10b00413b4575a74968 (commit) replaces v5.2.1 tagged by Andrew Gregory on Thu Jun 25 23:46:11 2020 -0700 - Log ----------------------------------------------------------------- v5.2.2 -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEEuBUbEXA3eBCVUUynu9/8kjBrESEFAl71mbMACgkQu9/8kjBr ESFuEAgAo7SFqphsUbw0KABZb/ofD8n8qZtzqjP5WdbJmqz60HgiCHtHtnhWEj/p 2mBxOuoJexfZadi4DUBUmMmy8Hk+k2/RFneD182ARrGqvOz50TDgh+NGGvG29ZWU +0CiuT+/gbHQhiq4+wIFj7fQR2l/tXC0fDULat2cVRceNHqWoCDhiF4RdGIl2aF8 eLrrD8YuzgpX1aLXo+qt0YYwSipDK5qVmDejo4s89eCBlG3Grqcre9s0SS43rTwu djRHmy3FjnXgAx6sRDwftw+9DThst0JcnNo1xzYifWfBoAzhxfMYgnpUDPfBlxzv 2nY8SJpCDh7GfBTcNrFzcDVPTgczDw== =is9q -----END PGP SIGNATURE----- Allan McRae (11): pactest: set package tar format to GNU_FORMAT libalpm/sync.c: Do not download missing keys multiple times Handle .part files that are the size of the correct package Remove unneeded ltmain patch Increase maximum database size Fix "pacman -U <url>" operations build-aux/update-copyright 2019 2020 pacman.8: Fix typo libalpm/signing.c: Fix calculation of packet size in parse_subpacket Pull translation updates pacman-key: change signing key to RSA4096 Andrew Gregory (2): update NEWS for 5.2.2 Release 5.2.2 Daniel T. Borelli (1): Dereference double pointer before assigning NULL Earnestly (1): Use noextract with pacman-conf NoExtract Eli Schwartz (13): makepkg: fix one more file-seccomp issue makepkg: fix regression that broke extraction of file:// sources makepkg: make per-package files containing '$pkgname' consistently work pacman-conf: fix incomplete support for ILoveCandy Log invalid conf settings as an error libmakepkg/strip: don't re-add the same debug source multiple times makepkg: guard against undefined git pinned sources makepkg: correctly handle missing download clients libmakepkg: fix regression in sending plain() output to stderr makepkg/repo-add: handle GPGKEY with spaces makepkg/repo-add: do not accept public-only keys for signing doc/pacman.8: fix typo autotools: emit error message when autoconf-archive is missing Levente Polyak (1): makepkg: deterministic PKGINFO libprovides for multiple library versions morganamilo (1): libalpm: fix alpm_option_set_assumeinstalled ----------------------------------------------------------------------- hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [GIT] The official pacman repository branch, release/5.2.x, updated. v5.2.2
by Andrew Gregory 26 Jun '20

26 Jun '20

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, release/5.2.x has been updated via 5537881b2525a4f114fcf10b00413b4575a74968 (commit) via 027c87ae3f7f4728b4058adb23baf7612cc5ea77 (commit) via b2c97ad76290eab37afd3873d011745d43059585 (commit) via c834a75718949103a0d3b71f6de8bb6dbabfd74a (commit) via 82d4b98ddce0165a0d1ec5474d448b2a93167b4e (commit) from 50f5e484f278d3550da71246bfb51d3d8053bae8 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 5537881b2525a4f114fcf10b00413b4575a74968 Author: Andrew Gregory <andrew(a)archlinux.org> Date: Mon Jun 22 02:15:23 2020 -0700 Release 5.2.2 Signed-off-by: Andrew Gregory <andrew(a)archlinux.org> commit 027c87ae3f7f4728b4058adb23baf7612cc5ea77 Author: Andrew Gregory <andrew(a)archlinux.org> Date: Mon Jun 22 02:12:41 2020 -0700 update NEWS for 5.2.2 Signed-off-by: Andrew Gregory <andrew(a)archlinux.org> commit b2c97ad76290eab37afd3873d011745d43059585 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Jan 5 16:04:21 2020 -0500 autotools: emit error message when autoconf-archive is missing Forbid the AX_COMPARE_VERSION macro from being found in the output configure script. If autoconf-archive is not installed when autoreconf is run, the following error message is emitted: configure.ac:231: error: possibly undefined macro: AX_COMPARE_VERSION If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. autoreconf: /usr/bin/autoconf failed with exit status: 1 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 435f5fc20484d3728d8877eea15c7a8653da6d10) commit c834a75718949103a0d3b71f6de8bb6dbabfd74a Author: Allan McRae <allan(a)archlinux.org> Date: Mon Jun 15 19:17:41 2020 +1000 pacman-key: change signing key to RSA4096 RSA2048 may have been fine when this was written many moons ago, but time this has a bump. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 7ba8e5f376528fcef9f7613304d8f18b2f664ee8) commit 82d4b98ddce0165a0d1ec5474d448b2a93167b4e Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Jun 14 14:52:02 2020 -0400 doc/pacman.8: fix typo Fixes FS#67000 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 59e751f72d09390067045168ac45a09a89419389) ----------------------------------------------------------------------- Summary of changes: NEWS | 30 ++++++++++++++++++++++++++++++ configure.ac | 5 +++-- doc/index.asciidoc | 1 + doc/pacman.8.asciidoc | 2 +- meson.build | 4 ++-- scripts/pacman-key.sh.in | 2 +- 6 files changed, 38 insertions(+), 6 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [PATCH v2] srcinfo.sh: remove trailing newline
by Denton Liu 26 Jun '20

26 Jun '20

When a .SRCINFO file is generated via `makepkg --printsrcinfo`, each section is concluded with an empty line. This means that at the end of the file, an empty line remains. This is considered a trailing whitespace error. In fact, `git diff --check` will warn about this, saying "new blank line at EOF." Instead of closing each section off with an empty line, use the empty line to separate sections, omitting the empty line at the end of the file. Signed-off-by: Denton Liu <liu.denton(a)gmail.com> --- scripts/libmakepkg/srcinfo.sh.in | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/scripts/libmakepkg/srcinfo.sh.in b/scripts/libmakepkg/srcinfo.sh.in index 6e783279..d1e39f7d 100644 --- a/scripts/libmakepkg/srcinfo.sh.in +++ b/scripts/libmakepkg/srcinfo.sh.in @@ -30,7 +30,7 @@ srcinfo_open_section() { printf '%s = %s\n' "$1" "$2" } -srcinfo_close_section() { +srcinfo_separate_section() { echo } @@ -94,7 +94,6 @@ srcinfo_write_global() { srcinfo_open_section 'pkgbase' "${pkgbase:-$pkgname}" srcinfo_write_section_details '' - srcinfo_close_section } srcinfo_write_package() { @@ -104,7 +103,6 @@ srcinfo_write_package() { srcinfo_open_section 'pkgname' "$1" srcinfo_write_section_details "$1" - srcinfo_close_section } write_srcinfo_header() { @@ -118,6 +116,7 @@ write_srcinfo_content() { srcinfo_write_global for pkg in "${pkgname[@]}"; do + srcinfo_separate_section srcinfo_write_package "$pkg" done } -- 2.27.0.326.gff61950410

2 1

[pacman-dev] Fix srcinfo sometimes printing double newline at the end
by Martin Rys 25 Jun '20

25 Jun '20

Hi, this is a bit of an OCD thing, but `makepkg --printsrcinfo` usually generates a file with two newlines at the end. This logic happens in pacman/scripts/libmakepkg/srcinfo.sh.in (/usr/share/makepkg/srcinfo.sh) The relevant functions: srcinfo_close_section() { echo } srcinfo_write_global() { ... srcinfo_open_section 'pkgbase' "${pkgbase:-$pkgname}" srcinfo_write_section_details '' srcinfo_close_section } srcinfo_write_package() { ... srcinfo_open_section 'pkgname' "$1" srcinfo_write_section_details "$1" srcinfo_close_section } As you can see, the functions process section details, and then print out a newline no matter if details printed something or not. The write_section_details eventually leads to: srcinfo_write_attr() { ... printf "\t$attrname = %s\n" "${attrvalues[@]}" } Which seems to always print out a newline, so I believe the srcinfo_close_section function is not needed there and simply removing it and the two lines that call it would solve this issue, as per the patchfile in the attachment. Thanks for taking a look at this, Marti

4 4

[pacman-dev] Unable to join translation team
by Lukáš Kucharczyk 25 Jun '20

25 Jun '20

Hello, I have requested joining the Czech translation team more than a week ago and as far as I can see, there has been no activity since then. The organization name is "toofishes" but I cannot find a way to contact the person in charge via Transifex to ask what is the issue. What are my other options? Lukas Kucharczyk

2 1

[pacman-dev] [PATCH] Move signature payload creation to download engine
by Anatol Pomozov 23 Jun '20

23 Jun '20

Until now callee of ALPM download functionality has been in charge of payload creation both for the main file (e.g. *.pkg) and for the accompanied *.sig file. One advantage of such solution is that all payloads are independent and can be fetched in parallel thus exploiting the maximum level of download parallelism. To build *.sig file url we've been using a simple string concatenation: $requested_url + ".sig". Unfortunately there are cases when it does not work. For example an archlinux.org "Download From Mirror" link looks like this https://www.archlinux.org/packages/core/x86_64/bash/download/ and it gets redirected to some mirror. But if we append ".sig" to the end of the link url and try to download it then archlinux.org returns 404 error. To overcome this issue we need to follow redirects for the main payload first, find the final url and only then append '.sig' suffix. This implies 2 things: - the signature payload initialization need to be moved to dload.c as it is the place where we have access to the resolved url - *.sig is downloaded serially with the main payload and this reduces level of parallelism Move *.sig payload creation to dload.c. Once the main payload is fetched successfully we check if the callee asked to download the accompanied signature. If yes - create a new payload and add it to mcurl. *.sig payload does not use server list of the main payload and thus does not support mirror failover. *.sig file comes from the same server as the main payload. Refactor event loop in curl_multi_download_internal() a bit. Instead of relying on curl_multi_check_finished_download() to return number of new payloads we simply rerun the loop iteration one more time to check if there are any active downloads left. --- lib/libalpm/be_sync.c | 34 +++------------- lib/libalpm/dload.c | 91 ++++++++++++++++++++++++++----------------- lib/libalpm/dload.h | 4 +- 3 files changed, 65 insertions(+), 64 deletions(-) diff --git a/lib/libalpm/be_sync.c b/lib/libalpm/be_sync.c index 82018e15..41675d21 100644 --- a/lib/libalpm/be_sync.c +++ b/lib/libalpm/be_sync.c @@ -180,12 +180,10 @@ int SYMEXPORT alpm_db_update(alpm_handle_t *handle, alpm_list_t *dbs, int force) dbforce = 1; } - CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); + siglevel = alpm_db_get_siglevel(db); - /* set hard upper limit of 128 MiB */ - payload->max_size = 128 * 1024 * 1024; + CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); payload->servers = db->servers; - /* print server + filename into a buffer */ len = strlen(db->treename) + strlen(dbext) + 1; MALLOC(payload->filepath, len, @@ -194,31 +192,11 @@ int SYMEXPORT alpm_db_update(alpm_handle_t *handle, alpm_list_t *dbs, int force) payload->handle = handle; payload->force = dbforce; payload->unlink_on_fail = 1; - + payload->download_signature = (siglevel & ALPM_SIG_DATABASE); + payload->signature_optional = (siglevel & ALPM_SIG_DATABASE_OPTIONAL); + /* set hard upper limit of 128 MiB */ + payload->max_size = 128 * 1024 * 1024; payloads = alpm_list_add(payloads, payload); - - siglevel = alpm_db_get_siglevel(db); - if(siglevel & ALPM_SIG_DATABASE) { - struct dload_payload *sig_payload; - CALLOC(sig_payload, 1, sizeof(*sig_payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); - sig_payload->signature = 1; - - /* print filename into a buffer (leave space for separator and .sig) */ - len = strlen(db->treename) + strlen(dbext) + 5; - MALLOC(sig_payload->filepath, len, - FREE(sig_payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); - snprintf(sig_payload->filepath, len, "%s%s.sig", db->treename, dbext); - - sig_payload->handle = handle; - sig_payload->force = dbforce; - sig_payload->errors_ok = (siglevel & ALPM_SIG_DATABASE_OPTIONAL); - - /* set hard upper limit of 16 KiB */ - sig_payload->max_size = 16 * 1024; - sig_payload->servers = db->servers; - - payloads = alpm_list_add(payloads, sig_payload); - } } event.type = ALPM_EVENT_DB_RETRIEVE_START; diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 4dbb011f..b68dcf6d 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -18,6 +18,7 @@ * along with this program. If not, see <http://www.gnu.org/licenses/>. */ +#include <stdbool.h> #include <stdlib.h> #include <stdio.h> #include <errno.h> @@ -49,6 +50,10 @@ #include "handle.h" #ifdef HAVE_LIBCURL + +static int curl_multi_add_payload(alpm_handle_t *handle, CURLM *curlm, + struct dload_payload *payload, const char *localpath); + static const char *get_filename(const char *url) { char *filename = strrchr(url, '/'); @@ -476,6 +481,25 @@ static int curl_multi_check_finished_download(CURLM *curlm, CURLMsg *msg, curl_easy_getinfo(curl, CURLINFO_CONDITION_UNMET, &timecond); curl_easy_getinfo(curl, CURLINFO_EFFECTIVE_URL, &effective_url); + /* Let's check if client requested downloading accompanion *.sig file */ + if(!payload->signature && payload->download_signature && curlerr == CURLE_OK && payload->respcode < 400) { + struct dload_payload *sig = NULL; + + int len = strlen(effective_url) + 5; + CALLOC(sig, 1, sizeof(*sig), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); + MALLOC(sig->fileurl, len, FREE(sig); GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); + snprintf(sig->fileurl, len, "%s.sig", effective_url); + sig->signature = 1; + sig->handle = handle; + sig->force = payload->force; + sig->unlink_on_fail = payload->unlink_on_fail; + sig->errors_ok = payload->signature_optional; + /* set hard upper limit of 16KiB */ + sig->max_size = 16 * 1024; + + curl_multi_add_payload(handle, curlm, sig, localpath); + } + /* time condition was met and we didn't download anything. we need to * clean up the 0 byte .part file that's left behind. */ if(timecond == 1 && DOUBLE_EQ(bytes_dl, 0)) { @@ -565,6 +589,13 @@ cleanup: if(ret == -1 && payload->errors_ok) { ret = -2; } + + if(payload->signature) { + /* free signature payload memory that was allocated earlier in dload.c */ + _alpm_dload_payload_reset(payload); + FREE(payload); + } + return ret; } @@ -669,12 +700,12 @@ static int curl_multi_download_internal(alpm_handle_t *handle, int still_running = 0; int err = 0; int parallel_downloads = handle->parallel_downloads; - CURLM *curlm = handle->curlm; - CURLMsg *msg; + bool msg_done = false; - while(still_running || payloads) { - int msgs_left = -1; + while(still_running || payloads || msg_done) { + int msgs_left = 0; + CURLMcode mc; for(; still_running < parallel_downloads && payloads; still_running++) { struct dload_payload *payload = payloads->data; @@ -687,15 +718,19 @@ static int curl_multi_download_internal(alpm_handle_t *handle, payloads = payloads->next; } else { - // the payload failed to start, do not start any new downloads just wait until - // active one complete. + /* The payload failed to start. Do not start any new downloads. + * Wait until all active downloads complete. + */ _alpm_log(handle, ALPM_LOG_ERROR, _("failed to setup a download payload for %s\n"), payload->remote_name); payloads = NULL; err = -1; } } - CURLMcode mc = curl_multi_perform(curlm, &still_running); + mc = curl_multi_perform(curlm, &still_running); + if(mc == CURLM_OK) { + mc = curl_multi_wait(curlm, NULL, 0, 1000, NULL); + } if(mc != CURLM_OK) { _alpm_log(handle, ALPM_LOG_ERROR, _("curl returned error %d from transfer\n"), mc); @@ -703,28 +738,28 @@ static int curl_multi_download_internal(alpm_handle_t *handle, err = -1; } - while((msg = curl_multi_info_read(curlm, &msgs_left))) { + msg_done = false; + while(true) { + CURLMsg *msg = curl_multi_info_read(curlm, &msgs_left); + if(!msg) { + break; + } if(msg->msg == CURLMSG_DONE) { int ret = curl_multi_check_finished_download(curlm, msg, localpath); if(ret == -1) { - /* if current payload failed to download then stop adding new payloads but wait for the - * current ones - */ + /* if current payload failed to download then stop adding new payloads */ payloads = NULL; err = -1; - } else if(ret == 2) { - /* in case of a retry increase the counter of active requests - * to avoid exiting the loop early - */ - still_running++; } + /* curl_multi_check_finished_download() might add more payloads e.g. in case of a retry + * from the next mirror. We need to execute curl_multi_perform() at least one more time + * to make sure new payload requests are processed. + */ + msg_done = true; } else { _alpm_log(handle, ALPM_LOG_ERROR, _("curl transfer error: %d\n"), msg->msg); } } - if(still_running) { - curl_multi_wait(curlm, NULL, 0, 1000, NULL); - } } _alpm_log(handle, ALPM_LOG_DEBUG, "curl_multi_download_internal return code is %d\n", err); @@ -803,12 +838,10 @@ int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls, { const char *cachedir; alpm_list_t *payloads = NULL; - int download_sigs; const alpm_list_t *i; alpm_event_t event; CHECK_HANDLE(handle, return -1); - download_sigs = handle->siglevel & ALPM_SIG_PACKAGE; /* find a valid cache dir to download to */ cachedir = _alpm_filecache_setup(handle); @@ -830,21 +863,9 @@ int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls, payload->allow_resume = 1; payload->handle = handle; payload->trust_remote_name = 1; + payload->download_signature = (handle->siglevel & ALPM_SIG_PACKAGE); + payload->signature_optional = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL); payloads = alpm_list_add(payloads, payload); - - if(download_sigs) { - int len = strlen(url) + 5; - CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); - payload->signature = 1; - MALLOC(payload->fileurl, len, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); - snprintf(payload->fileurl, len, "%s.sig", url); - payload->handle = handle; - payload->trust_remote_name = 1; - payload->errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL); - /* set hard upper limit of 16KiB */ - payload->max_size = 16 * 1024; - payloads = alpm_list_add(payloads, payload); - } } } diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h index d13bc1b5..facafca2 100644 --- a/lib/libalpm/dload.h +++ b/lib/libalpm/dload.h @@ -47,11 +47,13 @@ struct dload_payload { int errors_ok; int unlink_on_fail; int trust_remote_name; - int signature; /* specifies if the payload is a signature file */ + int download_signature; /* specifies if an accompanion *.sig file need to be downloaded*/ + int signature_optional; /* *.sig file is optional */ #ifdef HAVE_LIBCURL CURL *curl; char error_buffer[CURL_ERROR_SIZE]; FILE *localf; /* temp download file */ + int signature; /* specifies if this payload is for a signature file */ #endif }; -- 2.26.2

2 3

[pacman-dev] [PATCH] Cleanup the old sequential download code
by Anatol Pomozov 23 Jun '20

23 Jun '20

All users of _alpm_download() have been refactored to the new API. It is time to remove the old _alpm_download() functionality now. This change also removes obsolete SIGPIPE signal handler functionality (this is a leftover from libfetch days). Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> --- lib/libalpm/dload.c | 323 +------------------------------------------- lib/libalpm/dload.h | 4 - 2 files changed, 3 insertions(+), 324 deletions(-) diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 43fe9847..4dbb011f 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -78,10 +78,6 @@ enum { }; static int dload_interrupted; -static void inthandler(int UNUSED signum) -{ - dload_interrupted = ABORT_SIGINT; -} static int dload_progress_cb(void *file, curl_off_t dltotal, curl_off_t dlnow, curl_off_t UNUSED ultotal, curl_off_t UNUSED ulnow) @@ -236,8 +232,7 @@ static size_t dload_parseheader_cb(void *ptr, size_t size, size_t nmemb, void *u return realsize; } -static void curl_set_handle_opts(struct dload_payload *payload, - CURL *curl, char *error_buffer) +static void curl_set_handle_opts(CURL *curl, struct dload_payload *payload) { alpm_handle_t *handle = payload->handle; const char *useragent = getenv("HTTP_USER_AGENT"); @@ -247,7 +242,7 @@ static void curl_set_handle_opts(struct dload_payload *payload, * to reset the handle's parameters for each time it's used. */ curl_easy_reset(curl); curl_easy_setopt(curl, CURLOPT_URL, payload->fileurl); - curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, error_buffer); + curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, payload->error_buffer); curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 10L); curl_easy_setopt(curl, CURLOPT_MAXREDIRS, 10L); curl_easy_setopt(curl, CURLOPT_FILETIME, 1L); @@ -301,24 +296,6 @@ static void curl_set_handle_opts(struct dload_payload *payload, } } -static void mask_signal(int signum, void (*handler)(int), - struct sigaction *origaction) -{ - struct sigaction newaction; - - newaction.sa_handler = handler; - sigemptyset(&newaction.sa_mask); - newaction.sa_flags = 0; - - sigaction(signum, NULL, origaction); - sigaction(signum, &newaction, NULL); -} - -static void unmask_signal(int signum, struct sigaction *sa) -{ - sigaction(signum, sa, NULL); -} - static FILE *create_tempfile(struct dload_payload *payload, const char *localpath) { int fd; @@ -353,259 +330,6 @@ static FILE *create_tempfile(struct dload_payload *payload, const char *localpat /* RFC1123 states applications should support this length */ #define HOSTNAME_SIZE 256 -static int curl_download_internal(struct dload_payload *payload, - const char *localpath, char **final_file, const char **final_url) -{ - int ret = -1; - FILE *localf = NULL; - char *effective_url; - char hostname[HOSTNAME_SIZE]; - char error_buffer[CURL_ERROR_SIZE] = {0}; - struct stat st; - long timecond, remote_time = -1; - double remote_size, bytes_dl; - struct sigaction orig_sig_pipe, orig_sig_int; - CURLcode curlerr; - alpm_download_event_init_t init_cb_data = {0}; - alpm_download_event_completed_t completed_cb_data = {0}; - /* shortcut to our handle within the payload */ - alpm_handle_t *handle = payload->handle; - CURL *curl = curl_easy_init(); - handle->pm_errno = ALPM_ERR_OK; - payload->curl = curl; - - /* make sure these are NULL */ - FREE(payload->tempfile_name); - FREE(payload->destfile_name); - FREE(payload->content_disp_name); - - payload->tempfile_openmode = "wb"; - if(!payload->remote_name) { - STRDUP(payload->remote_name, get_filename(payload->fileurl), - RET_ERR(handle, ALPM_ERR_MEMORY, -1)); - } - if(curl_gethost(payload->fileurl, hostname, sizeof(hostname)) != 0) { - _alpm_log(handle, ALPM_LOG_ERROR, _("url '%s' is invalid\n"), payload->fileurl); - RET_ERR(handle, ALPM_ERR_SERVER_BAD_URL, -1); - } - - if(payload->remote_name && strlen(payload->remote_name) > 0 && - strcmp(payload->remote_name, ".sig") != 0) { - payload->destfile_name = get_fullpath(localpath, payload->remote_name, ""); - payload->tempfile_name = get_fullpath(localpath, payload->remote_name, ".part"); - if(!payload->destfile_name || !payload->tempfile_name) { - goto cleanup; - } - } else { - /* URL doesn't contain a filename, so make a tempfile. We can't support - * resuming this kind of download; partial transfers will be destroyed */ - payload->unlink_on_fail = 1; - - localf = create_tempfile(payload, localpath); - if(localf == NULL) { - goto cleanup; - } - } - - curl_set_handle_opts(payload, curl, error_buffer); - - if(payload->max_size == payload->initial_size && payload->max_size != 0) { - /* .part file is complete */ - ret = 0; - goto cleanup; - } - - if(localf == NULL) { - localf = fopen(payload->tempfile_name, payload->tempfile_openmode); - if(localf == NULL) { - _alpm_log(handle, ALPM_LOG_ERROR, - _("could not open file %s: %s\n"), - payload->tempfile_name, strerror(errno)); - GOTO_ERR(handle, ALPM_ERR_RETRIEVE, cleanup); - } - } - - _alpm_log(handle, ALPM_LOG_DEBUG, - "opened tempfile for download: %s (%s)\n", payload->tempfile_name, - payload->tempfile_openmode); - - curl_easy_setopt(curl, CURLOPT_WRITEDATA, localf); - - /* Ignore any SIGPIPE signals. With libcurl, these shouldn't be happening, - * but better safe than sorry. Store the old signal handler first. */ - mask_signal(SIGPIPE, SIG_IGN, &orig_sig_pipe); - dload_interrupted = 0; - mask_signal(SIGINT, &inthandler, &orig_sig_int); - - handle->dlcb(payload->remote_name, ALPM_DOWNLOAD_INIT, &init_cb_data); - - /* perform transfer */ - curlerr = curl_easy_perform(curl); - _alpm_log(handle, ALPM_LOG_DEBUG, "curl returned error %d from transfer\n", - curlerr); - - /* disconnect relationships from the curl handle for things that might go out - * of scope, but could still be touched on connection teardown. This really - * only applies to FTP transfers. */ - curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 1L); - curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, (char *)NULL); - - /* was it a success? */ - switch(curlerr) { - case CURLE_OK: - /* get http/ftp response code */ - _alpm_log(handle, ALPM_LOG_DEBUG, "response code: %ld\n", payload->respcode); - if(payload->respcode >= 400) { - payload->unlink_on_fail = 1; - if(!payload->errors_ok) { - handle->pm_errno = ALPM_ERR_RETRIEVE; - /* non-translated message is same as libcurl */ - snprintf(error_buffer, sizeof(error_buffer), - "The requested URL returned error: %ld", payload->respcode); - _alpm_log(handle, ALPM_LOG_ERROR, - _("failed retrieving file '%s' from %s : %s\n"), - payload->remote_name, hostname, error_buffer); - } - goto cleanup; - } - break; - case CURLE_ABORTED_BY_CALLBACK: - /* handle the interrupt accordingly */ - if(dload_interrupted == ABORT_OVER_MAXFILESIZE) { - curlerr = CURLE_FILESIZE_EXCEEDED; - payload->unlink_on_fail = 1; - handle->pm_errno = ALPM_ERR_LIBCURL; - _alpm_log(handle, ALPM_LOG_ERROR, - _("failed retrieving file '%s' from %s : expected download size exceeded\n"), - payload->remote_name, hostname); - } - goto cleanup; - case CURLE_COULDNT_RESOLVE_HOST: - payload->unlink_on_fail = 1; - handle->pm_errno = ALPM_ERR_SERVER_BAD_URL; - _alpm_log(handle, ALPM_LOG_ERROR, - _("failed retrieving file '%s' from %s : %s\n"), - payload->remote_name, hostname, error_buffer); - goto cleanup; - default: - /* delete zero length downloads */ - if(fstat(fileno(localf), &st) == 0 && st.st_size == 0) { - payload->unlink_on_fail = 1; - } - if(!payload->errors_ok) { - handle->pm_errno = ALPM_ERR_LIBCURL; - _alpm_log(handle, ALPM_LOG_ERROR, - _("failed retrieving file '%s' from %s : %s\n"), - payload->remote_name, hostname, error_buffer); - } else { - _alpm_log(handle, ALPM_LOG_DEBUG, - "failed retrieving file '%s' from %s : %s\n", - payload->remote_name, hostname, error_buffer); - } - goto cleanup; - } - - /* retrieve info about the state of the transfer */ - curl_easy_getinfo(curl, CURLINFO_FILETIME, &remote_time); - curl_easy_getinfo(curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &remote_size); - curl_easy_getinfo(curl, CURLINFO_SIZE_DOWNLOAD, &bytes_dl); - curl_easy_getinfo(curl, CURLINFO_CONDITION_UNMET, &timecond); - curl_easy_getinfo(curl, CURLINFO_EFFECTIVE_URL, &effective_url); - - if(final_url != NULL) { - *final_url = effective_url; - } - - /* time condition was met and we didn't download anything. we need to - * clean up the 0 byte .part file that's left behind. */ - if(timecond == 1 && DOUBLE_EQ(bytes_dl, 0)) { - _alpm_log(handle, ALPM_LOG_DEBUG, "file met time condition\n"); - ret = 1; - unlink(payload->tempfile_name); - goto cleanup; - } - - /* remote_size isn't necessarily the full size of the file, just what the - * server reported as remaining to download. compare it to what curl reported - * as actually being transferred during curl_easy_perform() */ - if(!DOUBLE_EQ(remote_size, -1) && !DOUBLE_EQ(bytes_dl, -1) && - !DOUBLE_EQ(bytes_dl, remote_size)) { - _alpm_log(handle, ALPM_LOG_ERROR, _("%s appears to be truncated: %jd/%jd bytes\n"), - payload->remote_name, (intmax_t)bytes_dl, (intmax_t)remote_size); - GOTO_ERR(handle, ALPM_ERR_RETRIEVE, cleanup); - } - - if(payload->trust_remote_name) { - if(payload->content_disp_name) { - /* content-disposition header has a better name for our file */ - free(payload->destfile_name); - payload->destfile_name = get_fullpath(localpath, - get_filename(payload->content_disp_name), ""); - } else { - const char *effective_filename = strrchr(effective_url, '/'); - if(effective_filename && strlen(effective_filename) > 2) { - effective_filename++; - - /* if destfile was never set, we wrote to a tempfile. even if destfile is - * set, we may have followed some redirects and the effective url may - * have a better suggestion as to what to name our file. in either case, - * refactor destfile to this newly derived name. */ - if(!payload->destfile_name || strcmp(effective_filename, - strrchr(payload->destfile_name, '/') + 1) != 0) { - free(payload->destfile_name); - payload->destfile_name = get_fullpath(localpath, effective_filename, ""); - } - } - } - } - - ret = 0; - -cleanup: - if(localf != NULL) { - fclose(localf); - utimes_long(payload->tempfile_name, remote_time); - } - - if(ret == 0) { - const char *realname = payload->tempfile_name; - if(payload->destfile_name) { - realname = payload->destfile_name; - if(rename(payload->tempfile_name, payload->destfile_name)) { - _alpm_log(handle, ALPM_LOG_ERROR, _("could not rename %s to %s (%s)\n"), - payload->tempfile_name, payload->destfile_name, strerror(errno)); - ret = -1; - } - } - if(ret != -1 && final_file) { - STRDUP(*final_file, strrchr(realname, '/') + 1, - RET_ERR(handle, ALPM_ERR_MEMORY, -1)); - } - } - - if((ret == -1 || dload_interrupted) && payload->unlink_on_fail && - payload->tempfile_name) { - unlink(payload->tempfile_name); - } - - curl_easy_cleanup(curl); - payload->curl = NULL; - - /* restore the old signal handlers */ - unmask_signal(SIGINT, &orig_sig_int); - unmask_signal(SIGPIPE, &orig_sig_pipe); - /* if we were interrupted, trip the old handler */ - if(dload_interrupted == ABORT_SIGINT) { - raise(SIGINT); - } - - completed_cb_data.total = bytes_dl; - completed_cb_data.result = ret; - handle->dlcb(payload->remote_name, ALPM_DOWNLOAD_COMPLETED, &completed_cb_data); - - return ret; -} - /* Return 0 if retry was successful, -1 otherwise */ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_payload *payload) { @@ -901,7 +625,7 @@ static int curl_multi_add_payload(alpm_handle_t *handle, CURLM *curlm, } } - curl_set_handle_opts(payload, curl, payload->error_buffer); + curl_set_handle_opts(curl, payload); if(payload->max_size == payload->initial_size && payload->max_size != 0) { /* .part file is complete */ @@ -1009,36 +733,6 @@ static int curl_multi_download_internal(alpm_handle_t *handle, #endif -/** Download a file given by a URL to a local directory. - * Does not overwrite an existing file if the download fails. - * @param payload the payload context - * @param localpath the directory to save the file in - * @param final_file the real name of the downloaded file (may be NULL) - * @return 0 on success, -1 on error (pm_errno is set accordingly if errors_ok == 0) - */ -int _alpm_download(struct dload_payload *payload, const char *localpath, - char **final_file, const char **final_url) -{ - alpm_handle_t *handle = payload->handle; - - if(handle->fetchcb == NULL) { -#ifdef HAVE_LIBCURL - return curl_download_internal(payload, localpath, final_file, final_url); -#else - /* work around unused warnings when building without libcurl */ - (void)final_file; - (void)final_url; - RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1); -#endif - } else { - int ret = handle->fetchcb(payload->fileurl, localpath, payload->force); - if(ret == -1 && !payload->errors_ok) { - RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1); - } - return ret; - } -} - int _alpm_multi_download(alpm_handle_t *handle, alpm_list_t *payloads /* struct dload_payload */, const char *localpath) @@ -1213,14 +907,3 @@ void _alpm_dload_payload_reset(struct dload_payload *payload) FREE(payload->filepath); *payload = (struct dload_payload){0}; } - -void _alpm_dload_payload_reset_for_retry(struct dload_payload *payload) -{ - ASSERT(payload, return); - - FREE(payload->fileurl); - FREE(payload->filepath); - payload->initial_size += payload->prevprogress; - payload->prevprogress = 0; - payload->unlink_on_fail = 0; -} diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h index f119fc2e..d13bc1b5 100644 --- a/lib/libalpm/dload.h +++ b/lib/libalpm/dload.h @@ -56,10 +56,6 @@ struct dload_payload { }; void _alpm_dload_payload_reset(struct dload_payload *payload); -void _alpm_dload_payload_reset_for_retry(struct dload_payload *payload); - -int _alpm_download(struct dload_payload *payload, const char *localpath, - char **final_file, const char **final_url); int _alpm_multi_download(alpm_handle_t *handle, alpm_list_t *payloads /* struct dload_payload */, -- 2.26.2

2 4

[pacman-dev] [PATCH v2] Convert '-U pkg1 pkg2' codepath to parallel download
by Anatol Pomozov 23 Jun '20

23 Jun '20

Installing remote packages using its URL is an interesting case for ALPM API. Unlike package sync ('pacman -S pkg1 pkg2') '-U' does not deal with server mirror list. Thus _alpm_multi_download() should be able to handle file download for payloads that either have 'fileurl' field or pair of fields ('servers' and 'filepath') set. Signature for alpm_fetch_pkgurl() has changed and it accepts an output list that is populated with filepaths to fetched packages. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> --- lib/libalpm/alpm.h | 15 ++-- lib/libalpm/dload.c | 205 +++++++++++++++++++++++++------------------ lib/libalpm/dload.h | 5 ++ src/pacman/upgrade.c | 102 ++++++++++----------- 4 files changed, 184 insertions(+), 143 deletions(-) diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index 3ea66ccc..977c292c 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -755,12 +755,17 @@ typedef void (*alpm_cb_totaldl)(off_t total); typedef int (*alpm_cb_fetch)(const char *url, const char *localpath, int force); -/** Fetch a remote pkg. +/** Fetch a list of remote packages. * @param handle the context handle - * @param url URL of the package to download - * @return the downloaded filepath on success, NULL on error - */ -char *alpm_fetch_pkgurl(alpm_handle_t *handle, const char *url); + * @param urls list of package URLs to download + * @param fetched list of filepaths to the fetched packages, each item + * corresponds to one in `urls` list. This is an output parameter, + * the caller should provide a pointer to an empty list + * (*fetched === NULL) and the callee fills the list with data. + * @return 0 on success or -1 on failure + */ +int alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls, + alpm_list_t **fetched); /** @addtogroup alpm_api_options Options * Libalpm option getters and setters diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 13aa4086..c1ec1e70 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -613,7 +613,9 @@ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_p size_t len; alpm_handle_t *handle = payload->handle; - payload->servers = payload->servers->next; + if(payload->servers) { + payload->servers = payload->servers->next; + } if(!payload->servers) { _alpm_log(payload->handle, ALPM_LOG_DEBUG, "%s: no more servers to retry\n", payload->remote_name); @@ -622,7 +624,7 @@ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_p server = payload->servers->data; /* regenerate a new fileurl */ - free(payload->fileurl); + FREE(payload->fileurl); len = strlen(server) + strlen(payload->filepath) + 2; MALLOC(payload->fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1)); snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath); @@ -849,20 +851,28 @@ static int curl_multi_add_payload(alpm_handle_t *handle, CURLM *curlm, struct dload_payload *payload, const char *localpath) { size_t len; - const char *server; CURL *curl = NULL; char hostname[HOSTNAME_SIZE]; int ret = -1; - ASSERT(payload->servers, RET_ERR(handle, ALPM_ERR_SERVER_NONE, -1)); - server = payload->servers->data; - curl = curl_easy_init(); payload->curl = curl; - len = strlen(server) + strlen(payload->filepath) + 2; - MALLOC(payload->fileurl, len, GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); - snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath); + if(payload->fileurl) { + ASSERT(!payload->servers, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup)); + ASSERT(!payload->filepath, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup)); + } else { + const char *server; + + ASSERT(payload->servers, GOTO_ERR(handle, ALPM_ERR_SERVER_NONE, cleanup)); + ASSERT(payload->filepath, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup)); + + server = payload->servers->data; + + len = strlen(server) + strlen(payload->filepath) + 2; + MALLOC(payload->fileurl, len, GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); + snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath); + } payload->tempfile_openmode = "wb"; if(!payload->remote_name) { @@ -1046,22 +1056,29 @@ int _alpm_multi_download(alpm_handle_t *handle, alpm_list_t *s; int success = 0; - for(s = payload->servers; s; s = s->next) { - const char *server = s->data; - char *fileurl; - int ret; - - size_t len = strlen(server) + strlen(payload->filepath) + 2; - MALLOC(fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1)); - snprintf(fileurl, len, "%s/%s", server, payload->filepath); - - ret = handle->fetchcb(fileurl, localpath, payload->force); - free(fileurl); - - if (ret != -1) { + if(payload->fileurl) { + if (handle->fetchcb(payload->fileurl, localpath, payload->force) != -1) { success = 1; break; } + } else { + for(s = payload->servers; s; s = s->next) { + const char *server = s->data; + char *fileurl; + int ret; + + size_t len = strlen(server) + strlen(payload->filepath) + 2; + MALLOC(fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1)); + snprintf(fileurl, len, "%s/%s", server, payload->filepath); + + ret = handle->fetchcb(fileurl, localpath, payload->force); + free(fileurl); + + if (ret != -1) { + success = 1; + break; + } + } } if(!success && !payload->errors_ok) { RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1); @@ -1087,82 +1104,102 @@ static char *filecache_find_url(alpm_handle_t *handle, const char *url) return _alpm_filecache_find(handle, filebase); } -char SYMEXPORT *alpm_fetch_pkgurl(alpm_handle_t *handle, const char *url) +int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls, + alpm_list_t **fetched) { - char *filepath; - const char *cachedir, *final_pkg_url = NULL; - char *final_file = NULL; - struct dload_payload payload = {0}; - int ret = 0; + const char *cachedir; + alpm_list_t *payloads = NULL; + int download_sigs; + const alpm_list_t *i; + alpm_event_t event; - CHECK_HANDLE(handle, return NULL); - ASSERT(url, RET_ERR(handle, ALPM_ERR_WRONG_ARGS, NULL)); + CHECK_HANDLE(handle, return -1); + ASSERT(*fetched == NULL, RET_ERR(handle, ALPM_ERR_WRONG_ARGS, -1)); + download_sigs = handle->siglevel & ALPM_SIG_PACKAGE; /* find a valid cache dir to download to */ cachedir = _alpm_filecache_setup(handle); - /* attempt to find the file in our pkgcache */ - filepath = filecache_find_url(handle, url); - if(filepath == NULL) { - STRDUP(payload.fileurl, url, RET_ERR(handle, ALPM_ERR_MEMORY, NULL)); - payload.allow_resume = 1; - payload.handle = handle; - payload.trust_remote_name = 1; - - /* download the file */ - ret = _alpm_download(&payload, cachedir, &final_file, &final_pkg_url); - _alpm_dload_payload_reset(&payload); - if(ret == -1) { - _alpm_log(handle, ALPM_LOG_WARNING, _("failed to download %s\n"), url); - free(final_file); - return NULL; + for(i = urls; i; i = i->next) { + char *url = i->data; + + /* attempt to find the file in our pkgcache */ + char *filepath = filecache_find_url(handle, url); + if(filepath) { + /* the file is locally cached so add it to the output right away */ + alpm_list_append(fetched, filepath); + } else { + struct dload_payload *payload = NULL; + + ASSERT(url, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, err)); + CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + STRDUP(payload->fileurl, url, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + payload->allow_resume = 1; + payload->handle = handle; + payload->trust_remote_name = 1; + payloads = alpm_list_add(payloads, payload); + + if(download_sigs) { + int len = strlen(url) + 5; + CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + payload->signature = 1; + MALLOC(payload->fileurl, len, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + snprintf(payload->fileurl, len, "%s.sig", url); + payload->handle = handle; + payload->trust_remote_name = 1; + payload->errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL); + /* set hard upper limit of 16KiB */ + payload->max_size = 16 * 1024; + payloads = alpm_list_add(payloads, payload); + } } - _alpm_log(handle, ALPM_LOG_DEBUG, "successfully downloaded %s\n", url); } - /* attempt to download the signature */ - if(ret == 0 && final_pkg_url && (handle->siglevel & ALPM_SIG_PACKAGE)) { - char *sig_filepath, *sig_final_file = NULL; - size_t len; - - len = strlen(final_pkg_url) + 5; - MALLOC(payload.fileurl, len, free(final_file); RET_ERR(handle, ALPM_ERR_MEMORY, NULL)); - snprintf(payload.fileurl, len, "%s.sig", final_pkg_url); - - sig_filepath = filecache_find_url(handle, payload.fileurl); - if(sig_filepath == NULL) { - payload.signature = 1; - payload.handle = handle; - payload.trust_remote_name = 1; - payload.force = 1; - payload.errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL); - - /* set hard upper limit of 16KiB */ - payload.max_size = 16 * 1024; - - ret = _alpm_download(&payload, cachedir, &sig_final_file, NULL); - if(ret == -1 && !payload.errors_ok) { - _alpm_log(handle, ALPM_LOG_WARNING, - _("failed to download %s\n"), payload.fileurl); - /* Warn now, but don't return NULL. We will fail later during package - * load time. */ - } else if(ret == 0) { - _alpm_log(handle, ALPM_LOG_DEBUG, - "successfully downloaded %s\n", payload.fileurl); + if(payloads) { + event.type = ALPM_EVENT_PKG_RETRIEVE_START; + EVENT(handle, &event); + if(_alpm_multi_download(handle, payloads, cachedir) == -1) { + _alpm_log(handle, ALPM_LOG_WARNING, _("failed to retrieve some files\n")); + event.type = ALPM_EVENT_PKG_RETRIEVE_FAILED; + EVENT(handle, &event); + + GOTO_ERR(handle, ALPM_ERR_RETRIEVE, err); + } else { + event.type = ALPM_EVENT_PKG_RETRIEVE_DONE; + EVENT(handle, &event); + } + + for(i = payloads; i; i = i->next) { + struct dload_payload *payload = i->data; + const char *filename; + char *filepath; + + if(payload->signature) { + continue; + } + + filename = mbasename(payload->destfile_name); + filepath = _alpm_filecache_find(handle, filename); + if(filepath) { + alpm_list_append(fetched, filepath); + } else { + _alpm_log(handle, ALPM_LOG_WARNING, _("download completed successfully but no file in the cache\n")); + GOTO_ERR(handle, ALPM_ERR_RETRIEVE, err); } - FREE(sig_final_file); } - free(sig_filepath); - _alpm_dload_payload_reset(&payload); - } - /* we should be able to find the file the second time around */ - if(filepath == NULL) { - filepath = _alpm_filecache_find(handle, final_file); + alpm_list_free_inner(payloads, (alpm_list_fn_free)_alpm_dload_payload_reset); + FREELIST(payloads); } - free(final_file); - return filepath; + return 0; + +err: + alpm_list_free_inner(payloads, (alpm_list_fn_free)_alpm_dload_payload_reset); + FREELIST(payloads); + FREELIST(*fetched); + + return -1; } void _alpm_dload_payload_reset(struct dload_payload *payload) diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h index 1e4af755..b9ef9c99 100644 --- a/lib/libalpm/dload.h +++ b/lib/libalpm/dload.h @@ -30,6 +30,11 @@ struct dload_payload { char *tempfile_name; char *destfile_name; char *content_disp_name; + /* client has to provide either + * 1) fileurl - full URL to the file + * 2) pair of (servers, filepath), in this case ALPM iterates over the + * server list and tries to download "$server/$filepath" + */ char *fileurl; char *filepath; /* download URL path */ alpm_list_t *servers; diff --git a/src/pacman/upgrade.c b/src/pacman/upgrade.c index 5f984e44..840e3a31 100644 --- a/src/pacman/upgrade.c +++ b/src/pacman/upgrade.c @@ -30,6 +30,34 @@ #include "conf.h" #include "util.h" +/* add targets to the created transaction */ +static int load_packages(alpm_list_t *targets, int siglevel) +{ + alpm_list_t *i; + int retval = 0; + + for(i = targets; i; i = alpm_list_next(i)) { + const char *targ = i->data; + alpm_pkg_t *pkg; + + if(alpm_pkg_load(config->handle, targ, 1, siglevel, &pkg) != 0) { + pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", + targ, alpm_strerror(alpm_errno(config->handle))); + retval = 1; + continue; + } + if(alpm_add_pkg(config->handle, pkg) == -1) { + pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", + targ, alpm_strerror(alpm_errno(config->handle))); + alpm_pkg_free(pkg); + retval = 1; + continue; + } + config->explicit_adds = alpm_list_add(config->explicit_adds, pkg); + } + return retval; +} + /** * @brief Upgrade a specified list of packages. * @@ -39,43 +67,30 @@ */ int pacman_upgrade(alpm_list_t *targets) { - int retval = 0, *file_is_remote; + int retval = 0; + alpm_list_t *remote_targets = NULL, *fetched_files = NULL; + alpm_list_t *local_targets = NULL; alpm_list_t *i; - unsigned int n, num_targets; if(targets == NULL) { pm_printf(ALPM_LOG_ERROR, _("no targets specified (use -h for help)\n")); return 1; } - num_targets = alpm_list_count(targets); - - /* Check for URL targets and process them */ - file_is_remote = malloc(num_targets * sizeof(int)); - if(file_is_remote == NULL) { - pm_printf(ALPM_LOG_ERROR, _("memory exhausted\n")); - return 1; - } - - for(i = targets, n = 0; i; i = alpm_list_next(i), n++) { + /* carve out remote targets and move it into a separate list */ + for(i = targets; i; i = alpm_list_next(i)) { if(strstr(i->data, "://")) { - char *str = alpm_fetch_pkgurl(config->handle, i->data); - if(str == NULL) { - pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", - (char *)i->data, alpm_strerror(alpm_errno(config->handle))); - retval = 1; - } else { - free(i->data); - i->data = str; - file_is_remote[n] = 1; - } + remote_targets = alpm_list_add(remote_targets, i->data); } else { - file_is_remote[n] = 0; + local_targets = alpm_list_add(local_targets, i->data); } } - if(retval) { - goto fail_free; + if(remote_targets) { + retval = alpm_fetch_pkgurl(config->handle, remote_targets, &fetched_files); + if(retval) { + goto fail_free; + } } /* Step 1: create a new transaction */ @@ -85,39 +100,16 @@ int pacman_upgrade(alpm_list_t *targets) } printf(_("loading packages...\n")); - /* add targets to the created transaction */ - for(i = targets, n = 0; i; i = alpm_list_next(i), n++) { - const char *targ = i->data; - alpm_pkg_t *pkg; - int siglevel; - - if(file_is_remote[n]) { - siglevel = alpm_option_get_remote_file_siglevel(config->handle); - } else { - siglevel = alpm_option_get_local_file_siglevel(config->handle); - } - - if(alpm_pkg_load(config->handle, targ, 1, siglevel, &pkg) != 0) { - pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", - targ, alpm_strerror(alpm_errno(config->handle))); - retval = 1; - continue; - } - if(alpm_add_pkg(config->handle, pkg) == -1) { - pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", - targ, alpm_strerror(alpm_errno(config->handle))); - alpm_pkg_free(pkg); - retval = 1; - continue; - } - config->explicit_adds = alpm_list_add(config->explicit_adds, pkg); - } + retval |= load_packages(local_targets, alpm_option_get_local_file_siglevel(config->handle)); + retval |= load_packages(fetched_files, alpm_option_get_remote_file_siglevel(config->handle)); if(retval) { goto fail_release; } - free(file_is_remote); + alpm_list_free(remote_targets); + alpm_list_free(local_targets); + FREELIST(fetched_files); /* now that targets are resolved, we can hand it all off to the sync code */ return sync_prepare_execute(); @@ -125,7 +117,9 @@ int pacman_upgrade(alpm_list_t *targets) fail_release: trans_release(); fail_free: - free(file_is_remote); + alpm_list_free(remote_targets); + alpm_list_free(local_targets); + FREELIST(fetched_files); return retval; } -- 2.27.0

1 0

[pacman-dev] [PATCH] Convert '-U pkg1 pkg2' codepath to parallel download
by Anatol Pomozov 23 Jun '20

23 Jun '20

Installing remote packages using its URL is an interesting case for ALPM API. Unlike package sync ('pacman -S pkg1 pkg2') '-U' does not deal with server mirror list. Thus _alpm_multi_download() should be able to handle file download for payloads that either have 'fileurl' field or pair of fields ('servers' and 'filepath') set. Signature for alpm_fetch_pkgurl() has changed and it accepts an output list that is populated with filepaths to fetched packages. Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com> --- lib/libalpm/alpm.h | 11 ++- lib/libalpm/dload.c | 204 +++++++++++++++++++++++++------------------ lib/libalpm/dload.h | 7 +- src/pacman/upgrade.c | 102 ++++++++++------------ 4 files changed, 181 insertions(+), 143 deletions(-) diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index 3ea66ccc..c6a13273 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -755,12 +755,15 @@ typedef void (*alpm_cb_totaldl)(off_t total); typedef int (*alpm_cb_fetch)(const char *url, const char *localpath, int force); -/** Fetch a remote pkg. +/** Fetch a list of remote packages. * @param handle the context handle - * @param url URL of the package to download - * @return the downloaded filepath on success, NULL on error + * @param urls list of package URLs to download + * @param fetched list of filepaths to the fetched packages, each item + * corresponds to one in `urls` list + * @return 0 on success or -1 on failure */ -char *alpm_fetch_pkgurl(alpm_handle_t *handle, const char *url); +int alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls, + alpm_list_t **fetched); /** @addtogroup alpm_api_options Options * Libalpm option getters and setters diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 13aa4086..43fe9847 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -613,7 +613,9 @@ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_p size_t len; alpm_handle_t *handle = payload->handle; - payload->servers = payload->servers->next; + if(payload->servers) { + payload->servers = payload->servers->next; + } if(!payload->servers) { _alpm_log(payload->handle, ALPM_LOG_DEBUG, "%s: no more servers to retry\n", payload->remote_name); @@ -622,7 +624,7 @@ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_p server = payload->servers->data; /* regenerate a new fileurl */ - free(payload->fileurl); + FREE(payload->fileurl); len = strlen(server) + strlen(payload->filepath) + 2; MALLOC(payload->fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1)); snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath); @@ -849,20 +851,28 @@ static int curl_multi_add_payload(alpm_handle_t *handle, CURLM *curlm, struct dload_payload *payload, const char *localpath) { size_t len; - const char *server; CURL *curl = NULL; char hostname[HOSTNAME_SIZE]; int ret = -1; - ASSERT(payload->servers, RET_ERR(handle, ALPM_ERR_SERVER_NONE, -1)); - server = payload->servers->data; - curl = curl_easy_init(); payload->curl = curl; - len = strlen(server) + strlen(payload->filepath) + 2; - MALLOC(payload->fileurl, len, GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); - snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath); + if(payload->fileurl) { + ASSERT(!payload->servers, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup)); + ASSERT(!payload->filepath, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup)); + } else { + const char *server; + + ASSERT(payload->servers, GOTO_ERR(handle, ALPM_ERR_SERVER_NONE, cleanup)); + ASSERT(payload->filepath, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup)); + + server = payload->servers->data; + + len = strlen(server) + strlen(payload->filepath) + 2; + MALLOC(payload->fileurl, len, GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup)); + snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath); + } payload->tempfile_openmode = "wb"; if(!payload->remote_name) { @@ -1046,22 +1056,29 @@ int _alpm_multi_download(alpm_handle_t *handle, alpm_list_t *s; int success = 0; - for(s = payload->servers; s; s = s->next) { - const char *server = s->data; - char *fileurl; - int ret; - - size_t len = strlen(server) + strlen(payload->filepath) + 2; - MALLOC(fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1)); - snprintf(fileurl, len, "%s/%s", server, payload->filepath); - - ret = handle->fetchcb(fileurl, localpath, payload->force); - free(fileurl); - - if (ret != -1) { + if(payload->fileurl) { + if (handle->fetchcb(payload->fileurl, localpath, payload->force) != -1) { success = 1; break; } + } else { + for(s = payload->servers; s; s = s->next) { + const char *server = s->data; + char *fileurl; + int ret; + + size_t len = strlen(server) + strlen(payload->filepath) + 2; + MALLOC(fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1)); + snprintf(fileurl, len, "%s/%s", server, payload->filepath); + + ret = handle->fetchcb(fileurl, localpath, payload->force); + free(fileurl); + + if (ret != -1) { + success = 1; + break; + } + } } if(!success && !payload->errors_ok) { RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1); @@ -1087,82 +1104,101 @@ static char *filecache_find_url(alpm_handle_t *handle, const char *url) return _alpm_filecache_find(handle, filebase); } -char SYMEXPORT *alpm_fetch_pkgurl(alpm_handle_t *handle, const char *url) +int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls, + alpm_list_t **fetched) { - char *filepath; - const char *cachedir, *final_pkg_url = NULL; - char *final_file = NULL; - struct dload_payload payload = {0}; - int ret = 0; + const char *cachedir; + alpm_list_t *payloads = NULL; + int download_sigs; + const alpm_list_t *i; + alpm_event_t event; - CHECK_HANDLE(handle, return NULL); - ASSERT(url, RET_ERR(handle, ALPM_ERR_WRONG_ARGS, NULL)); + CHECK_HANDLE(handle, return -1); + download_sigs = handle->siglevel & ALPM_SIG_PACKAGE; /* find a valid cache dir to download to */ cachedir = _alpm_filecache_setup(handle); - /* attempt to find the file in our pkgcache */ - filepath = filecache_find_url(handle, url); - if(filepath == NULL) { - STRDUP(payload.fileurl, url, RET_ERR(handle, ALPM_ERR_MEMORY, NULL)); - payload.allow_resume = 1; - payload.handle = handle; - payload.trust_remote_name = 1; - - /* download the file */ - ret = _alpm_download(&payload, cachedir, &final_file, &final_pkg_url); - _alpm_dload_payload_reset(&payload); - if(ret == -1) { - _alpm_log(handle, ALPM_LOG_WARNING, _("failed to download %s\n"), url); - free(final_file); - return NULL; + for(i = urls; i; i = i->next) { + char *url = i->data; + + /* attempt to find the file in our pkgcache */ + char *filepath = filecache_find_url(handle, url); + if(filepath) { + /* the file is locally cached so add it to the output right away */ + alpm_list_append(fetched, filepath); + } else { + struct dload_payload *payload = NULL; + + ASSERT(url, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, err)); + CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + STRDUP(payload->fileurl, url, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + payload->allow_resume = 1; + payload->handle = handle; + payload->trust_remote_name = 1; + payloads = alpm_list_add(payloads, payload); + + if(download_sigs) { + int len = strlen(url) + 5; + CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + payload->signature = 1; + MALLOC(payload->fileurl, len, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err)); + snprintf(payload->fileurl, len, "%s.sig", url); + payload->handle = handle; + payload->trust_remote_name = 1; + payload->errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL); + /* set hard upper limit of 16KiB */ + payload->max_size = 16 * 1024; + payloads = alpm_list_add(payloads, payload); + } } - _alpm_log(handle, ALPM_LOG_DEBUG, "successfully downloaded %s\n", url); } - /* attempt to download the signature */ - if(ret == 0 && final_pkg_url && (handle->siglevel & ALPM_SIG_PACKAGE)) { - char *sig_filepath, *sig_final_file = NULL; - size_t len; - - len = strlen(final_pkg_url) + 5; - MALLOC(payload.fileurl, len, free(final_file); RET_ERR(handle, ALPM_ERR_MEMORY, NULL)); - snprintf(payload.fileurl, len, "%s.sig", final_pkg_url); - - sig_filepath = filecache_find_url(handle, payload.fileurl); - if(sig_filepath == NULL) { - payload.signature = 1; - payload.handle = handle; - payload.trust_remote_name = 1; - payload.force = 1; - payload.errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL); - - /* set hard upper limit of 16KiB */ - payload.max_size = 16 * 1024; - - ret = _alpm_download(&payload, cachedir, &sig_final_file, NULL); - if(ret == -1 && !payload.errors_ok) { - _alpm_log(handle, ALPM_LOG_WARNING, - _("failed to download %s\n"), payload.fileurl); - /* Warn now, but don't return NULL. We will fail later during package - * load time. */ - } else if(ret == 0) { - _alpm_log(handle, ALPM_LOG_DEBUG, - "successfully downloaded %s\n", payload.fileurl); + if(payloads) { + event.type = ALPM_EVENT_PKG_RETRIEVE_START; + EVENT(handle, &event); + if(_alpm_multi_download(handle, payloads, cachedir) == -1) { + _alpm_log(handle, ALPM_LOG_WARNING, _("failed to retrieve some files\n")); + event.type = ALPM_EVENT_PKG_RETRIEVE_FAILED; + EVENT(handle, &event); + + GOTO_ERR(handle, ALPM_ERR_RETRIEVE, err); + } else { + event.type = ALPM_EVENT_PKG_RETRIEVE_DONE; + EVENT(handle, &event); + } + + for(i = payloads; i; i = i->next) { + struct dload_payload *payload = i->data; + const char *filename; + char *filepath; + + if(payload->signature) { + continue; + } + + filename = mbasename(payload->destfile_name); + filepath = _alpm_filecache_find(handle, filename); + if(filepath) { + alpm_list_append(fetched, filepath); + } else { + _alpm_log(handle, ALPM_LOG_WARNING, _("download completed successfully but no file in the cache\n")); + GOTO_ERR(handle, ALPM_ERR_RETRIEVE, err); } - FREE(sig_final_file); } - free(sig_filepath); - _alpm_dload_payload_reset(&payload); - } - /* we should be able to find the file the second time around */ - if(filepath == NULL) { - filepath = _alpm_filecache_find(handle, final_file); + alpm_list_free_inner(payloads, (alpm_list_fn_free)_alpm_dload_payload_reset); + FREELIST(payloads); } - free(final_file); - return filepath; + return 0; + +err: + alpm_list_free_inner(payloads, (alpm_list_fn_free)_alpm_dload_payload_reset); + FREELIST(payloads); + FREELIST(*fetched); + + return -1; } void _alpm_dload_payload_reset(struct dload_payload *payload) diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h index 1e4af755..f119fc2e 100644 --- a/lib/libalpm/dload.h +++ b/lib/libalpm/dload.h @@ -30,9 +30,14 @@ struct dload_payload { char *tempfile_name; char *destfile_name; char *content_disp_name; + /* client has to provide either + * 1) fileurl - full URL to the file + * 2) pair of (servers, filepath), in this case ALPM iterates over the + * server list and tries to download "$server/$filepath" + */ char *fileurl; - char *filepath; /* download URL path */ alpm_list_t *servers; + char *filepath; /* download URL path */ long respcode; off_t initial_size; off_t max_size; diff --git a/src/pacman/upgrade.c b/src/pacman/upgrade.c index 5f984e44..840e3a31 100644 --- a/src/pacman/upgrade.c +++ b/src/pacman/upgrade.c @@ -30,6 +30,34 @@ #include "conf.h" #include "util.h" +/* add targets to the created transaction */ +static int load_packages(alpm_list_t *targets, int siglevel) +{ + alpm_list_t *i; + int retval = 0; + + for(i = targets; i; i = alpm_list_next(i)) { + const char *targ = i->data; + alpm_pkg_t *pkg; + + if(alpm_pkg_load(config->handle, targ, 1, siglevel, &pkg) != 0) { + pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", + targ, alpm_strerror(alpm_errno(config->handle))); + retval = 1; + continue; + } + if(alpm_add_pkg(config->handle, pkg) == -1) { + pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", + targ, alpm_strerror(alpm_errno(config->handle))); + alpm_pkg_free(pkg); + retval = 1; + continue; + } + config->explicit_adds = alpm_list_add(config->explicit_adds, pkg); + } + return retval; +} + /** * @brief Upgrade a specified list of packages. * @@ -39,43 +67,30 @@ */ int pacman_upgrade(alpm_list_t *targets) { - int retval = 0, *file_is_remote; + int retval = 0; + alpm_list_t *remote_targets = NULL, *fetched_files = NULL; + alpm_list_t *local_targets = NULL; alpm_list_t *i; - unsigned int n, num_targets; if(targets == NULL) { pm_printf(ALPM_LOG_ERROR, _("no targets specified (use -h for help)\n")); return 1; } - num_targets = alpm_list_count(targets); - - /* Check for URL targets and process them */ - file_is_remote = malloc(num_targets * sizeof(int)); - if(file_is_remote == NULL) { - pm_printf(ALPM_LOG_ERROR, _("memory exhausted\n")); - return 1; - } - - for(i = targets, n = 0; i; i = alpm_list_next(i), n++) { + /* carve out remote targets and move it into a separate list */ + for(i = targets; i; i = alpm_list_next(i)) { if(strstr(i->data, "://")) { - char *str = alpm_fetch_pkgurl(config->handle, i->data); - if(str == NULL) { - pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", - (char *)i->data, alpm_strerror(alpm_errno(config->handle))); - retval = 1; - } else { - free(i->data); - i->data = str; - file_is_remote[n] = 1; - } + remote_targets = alpm_list_add(remote_targets, i->data); } else { - file_is_remote[n] = 0; + local_targets = alpm_list_add(local_targets, i->data); } } - if(retval) { - goto fail_free; + if(remote_targets) { + retval = alpm_fetch_pkgurl(config->handle, remote_targets, &fetched_files); + if(retval) { + goto fail_free; + } } /* Step 1: create a new transaction */ @@ -85,39 +100,16 @@ int pacman_upgrade(alpm_list_t *targets) } printf(_("loading packages...\n")); - /* add targets to the created transaction */ - for(i = targets, n = 0; i; i = alpm_list_next(i), n++) { - const char *targ = i->data; - alpm_pkg_t *pkg; - int siglevel; - - if(file_is_remote[n]) { - siglevel = alpm_option_get_remote_file_siglevel(config->handle); - } else { - siglevel = alpm_option_get_local_file_siglevel(config->handle); - } - - if(alpm_pkg_load(config->handle, targ, 1, siglevel, &pkg) != 0) { - pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", - targ, alpm_strerror(alpm_errno(config->handle))); - retval = 1; - continue; - } - if(alpm_add_pkg(config->handle, pkg) == -1) { - pm_printf(ALPM_LOG_ERROR, "'%s': %s\n", - targ, alpm_strerror(alpm_errno(config->handle))); - alpm_pkg_free(pkg); - retval = 1; - continue; - } - config->explicit_adds = alpm_list_add(config->explicit_adds, pkg); - } + retval |= load_packages(local_targets, alpm_option_get_local_file_siglevel(config->handle)); + retval |= load_packages(fetched_files, alpm_option_get_remote_file_siglevel(config->handle)); if(retval) { goto fail_release; } - free(file_is_remote); + alpm_list_free(remote_targets); + alpm_list_free(local_targets); + FREELIST(fetched_files); /* now that targets are resolved, we can hand it all off to the sync code */ return sync_prepare_execute(); @@ -125,7 +117,9 @@ int pacman_upgrade(alpm_list_t *targets) fail_release: trans_release(); fail_free: - free(file_is_remote); + alpm_list_free(remote_targets); + alpm_list_free(local_targets); + FREELIST(fetched_files); return retval; } -- 2.26.2

2 2

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.2.1-97-g59e751f7
by Allan McRae 19 Jun '20

19 Jun '20

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 59e751f72d09390067045168ac45a09a89419389 (commit) via 7ba8e5f376528fcef9f7613304d8f18b2f664ee8 (commit) from 02ae97b0da220d9079c6c2c1ac3e3ab0d12c1ac7 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 59e751f72d09390067045168ac45a09a89419389 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun Jun 14 14:52:02 2020 -0400 doc/pacman.8: fix typo Fixes FS#67000 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 7ba8e5f376528fcef9f7613304d8f18b2f664ee8 Author: Allan McRae <allan(a)archlinux.org> Date: Mon Jun 15 19:17:41 2020 +1000 pacman-key: change signing key to RSA4096 RSA2048 may have been fine when this was written many moons ago, but time this has a bump. Signed-off-by: Allan McRae <allan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: doc/pacman.8.asciidoc | 2 +- scripts/pacman-key.sh.in | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [GIT] The official pacman repository branch, release/5.2.x, updated. v5.2.1-25-g50f5e484
by Allan McRae 18 Jun '20

18 Jun '20

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, release/5.2.x has been updated via 50f5e484f278d3550da71246bfb51d3d8053bae8 (commit) from 1e9cd30e488cd133d24eac7ed9cac7806db2c406 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 50f5e484f278d3550da71246bfb51d3d8053bae8 Author: Allan McRae <allan(a)archlinux.org> Date: Fri Jun 19 09:49:28 2020 +1000 Pull translation updates Signed-off-by: Allan McRae <allan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: lib/libalpm/po/ar.po | 28 ++--- lib/libalpm/po/ast.po | 34 +++--- lib/libalpm/po/bg.po | 28 ++--- lib/libalpm/po/br.po | 28 ++--- lib/libalpm/po/ca.po | 28 ++--- lib/libalpm/po/cs.po | 28 ++--- lib/libalpm/po/da.po | 28 ++--- lib/libalpm/po/de.po | 45 ++++---- lib/libalpm/po/el.po | 56 ++++----- lib/libalpm/po/en_GB.po | 28 ++--- lib/libalpm/po/eo.po | 262 +++++++++++++++++++++--------------------- lib/libalpm/po/es.po | 28 ++--- lib/libalpm/po/es_419.po | 32 +++--- lib/libalpm/po/eu.po | 28 ++--- lib/libalpm/po/eu_ES.po | 28 ++--- lib/libalpm/po/fi.po | 28 ++--- lib/libalpm/po/fr.po | 28 ++--- lib/libalpm/po/gl.po | 28 ++--- lib/libalpm/po/hr.po | 28 ++--- lib/libalpm/po/hu.po | 28 ++--- lib/libalpm/po/id.po | 28 ++--- lib/libalpm/po/it.po | 34 +++--- lib/libalpm/po/ja.po | 28 ++--- lib/libalpm/po/kk.po | 28 ++--- lib/libalpm/po/ko.po | 30 ++--- lib/libalpm/po/libalpm.pot | 28 ++--- lib/libalpm/po/lt.po | 28 ++--- lib/libalpm/po/nb.po | 28 ++--- lib/libalpm/po/nl.po | 28 ++--- lib/libalpm/po/pl.po | 41 +++---- lib/libalpm/po/pt.po | 28 ++--- lib/libalpm/po/pt_BR.po | 28 ++--- lib/libalpm/po/ro.po | 32 +++--- lib/libalpm/po/ru.po | 28 ++--- lib/libalpm/po/sk.po | 32 +++--- lib/libalpm/po/sl.po | 30 ++--- lib/libalpm/po/sr.po | 28 ++--- lib/libalpm/po/sr(a)latin.po | 28 ++--- lib/libalpm/po/sv.po | 55 ++++----- lib/libalpm/po/tr.po | 36 +++--- lib/libalpm/po/uk.po | 28 ++--- lib/libalpm/po/zh_CN.po | 47 ++++---- lib/libalpm/po/zh_TW.po | 28 ++--- scripts/po/bg.po | 176 ++++++++++++++-------------- scripts/po/br.po | 176 ++++++++++++++-------------- scripts/po/ca.po | 176 ++++++++++++++-------------- scripts/po/cs.po | 176 ++++++++++++++-------------- scripts/po/da.po | 176 ++++++++++++++-------------- scripts/po/de.po | 208 +++++++++++++++++---------------- scripts/po/el.po | 216 +++++++++++++++++----------------- scripts/po/en_GB.po | 176 ++++++++++++++-------------- scripts/po/eo.po | 203 ++++++++++++++++---------------- scripts/po/es.po | 176 ++++++++++++++-------------- scripts/po/es_419.po | 180 ++++++++++++++--------------- scripts/po/eu.po | 176 ++++++++++++++-------------- scripts/po/eu_ES.po | 176 ++++++++++++++-------------- scripts/po/fi.po | 176 ++++++++++++++-------------- scripts/po/fr.po | 178 ++++++++++++++-------------- scripts/po/gl.po | 176 ++++++++++++++-------------- scripts/po/hu.po | 176 ++++++++++++++-------------- scripts/po/id.po | 176 ++++++++++++++-------------- scripts/po/it.po | 180 ++++++++++++++--------------- scripts/po/ja.po | 176 ++++++++++++++-------------- scripts/po/ko.po | 176 ++++++++++++++-------------- scripts/po/lt.po | 176 ++++++++++++++-------------- scripts/po/nb.po | 182 ++++++++++++++--------------- scripts/po/nl.po | 176 ++++++++++++++-------------- scripts/po/pacman-scripts.pot | 176 ++++++++++++++-------------- scripts/po/pl.po | 215 +++++++++++++++++----------------- scripts/po/pt.po | 176 ++++++++++++++-------------- scripts/po/pt_BR.po | 176 ++++++++++++++-------------- scripts/po/ro.po | 176 ++++++++++++++-------------- scripts/po/ru.po | 176 ++++++++++++++-------------- scripts/po/sk.po | 180 ++++++++++++++--------------- scripts/po/sl.po | 203 ++++++++++++++++---------------- scripts/po/sr.po | 176 ++++++++++++++-------------- scripts/po/sr(a)latin.po | 176 ++++++++++++++-------------- scripts/po/sv.po | 229 ++++++++++++++++++------------------ scripts/po/tr.po | 186 +++++++++++++++--------------- scripts/po/uk.po | 176 ++++++++++++++-------------- scripts/po/zh_CN.po | 247 +++++++++++++++++++-------------------- scripts/po/zh_TW.po | 200 ++++++++++++++++---------------- src/pacman/po/ar.po | 8 +- src/pacman/po/ast.po | 10 +- src/pacman/po/bg.po | 8 +- src/pacman/po/br.po | 8 +- src/pacman/po/ca.po | 8 +- src/pacman/po/cs.po | 8 +- src/pacman/po/da.po | 8 +- src/pacman/po/de.po | 27 ++--- src/pacman/po/el.po | 75 ++++++------ src/pacman/po/en_GB.po | 8 +- src/pacman/po/eo.po | 42 +++---- src/pacman/po/es.po | 16 +-- src/pacman/po/es_419.po | 12 +- src/pacman/po/eu.po | 8 +- src/pacman/po/eu_ES.po | 8 +- src/pacman/po/fi.po | 8 +- src/pacman/po/fr.po | 21 ++-- src/pacman/po/gl.po | 8 +- src/pacman/po/hr.po | 16 +-- src/pacman/po/hu.po | 8 +- src/pacman/po/id.po | 8 +- src/pacman/po/it.po | 14 +-- src/pacman/po/ja.po | 8 +- src/pacman/po/ko.po | 16 +-- src/pacman/po/lt.po | 20 ++-- src/pacman/po/nb.po | 40 +++---- src/pacman/po/nl.po | 12 +- src/pacman/po/pacman.pot | 8 +- src/pacman/po/pl.po | 67 ++++++----- src/pacman/po/pt.po | 8 +- src/pacman/po/pt_BR.po | 8 +- src/pacman/po/ro.po | 12 +- src/pacman/po/ru.po | 8 +- src/pacman/po/sk.po | 12 +- src/pacman/po/sl.po | 10 +- src/pacman/po/sr.po | 8 +- src/pacman/po/sr(a)latin.po | 8 +- src/pacman/po/sv.po | 47 ++++---- src/pacman/po/tr.po | 16 +-- src/pacman/po/uk.po | 14 +-- src/pacman/po/zh_CN.po | 68 +++++------ src/pacman/po/zh_TW.po | 8 +- 124 files changed, 4782 insertions(+), 4738 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [GIT] The official pacman repository branch, release/5.2.x, updated. v5.2.1-24-g1e9cd30e
by Andrew Gregory 18 Jun '20

18 Jun '20

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, release/5.2.x has been updated via 1e9cd30e488cd133d24eac7ed9cac7806db2c406 (commit) via 2a345604cd8bd6a8fe5041ea7ec046c8d20aa30b (commit) via f4da297de252dcb00e6c83df00fea11c86606348 (commit) via 22e6daa79455b1c59ab73af9438ca0c1d2a884b9 (commit) via d69da08abef42a52d2fa81daf4c045c3305598cf (commit) via 12503767c0be8f3eb28433fd58bcab0ec6f44a26 (commit) via 03cfe9e21c6aaecaac21e932229a55f5f1041a3c (commit) via b3be0ce99b29e5384969c7b3e5ff6f4b0f581c4b (commit) via 68418c54427b2ecbd5f882ffad261e0ba6e67c42 (commit) via bb50e8d73afda7f17777c464bd7bfa127a749b8d (commit) via b9d397c7317b9b495da73051fc670b0e9f472a06 (commit) via 39ce2b766303f046bb75ad900cc8a2695e947f83 (commit) via 01d5a68c1ae07ad43b5255721d2429710794ea04 (commit) via 08aa3d97ea3823f2919e7d66041438db9e7488a0 (commit) via 0bf4779cda35cfb8dc049ec5f27017c65c8ce716 (commit) via d61c398b2cb3ec41e62a6e258b5791d0ce9de720 (commit) via 7faa79526874c09fc89fb769744d103bc4918e17 (commit) via 0d0a4bd68099596ba17b8c314664484c3819fbf3 (commit) via 76c50e34393d68a740f825a6af9df8b7ac677303 (commit) via c2fa9f85cccc4b576816330762fb55167a1ba78c (commit) via 6f1a9e6ea85a49b8aeb3b7bd2454215e9c34f565 (commit) via 41c3b1d78c3807beb4246624b20a31e374fb5af1 (commit) via d6dcc936457815cdda87a8507cf82169b437e4d2 (commit) via 019f9386ef2c85b4c3c9f9293f462cfb5ddcaa32 (commit) from f6564377a2b0a0dd6294fb366a3f91a31142e124 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 1e9cd30e488cd133d24eac7ed9cac7806db2c406 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon Jun 8 22:03:18 2020 -0400 makepkg/repo-add: do not accept public-only keys for signing If it's not listed by --list-secret-key we don't care if it has been imported into your keyring, it's unusable. And you might not have a private key at all in the no-keyid-specified case. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 02ae97b0da220d9079c6c2c1ac3e3ab0d12c1ac7) commit 2a345604cd8bd6a8fe5041ea7ec046c8d20aa30b Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon Jun 8 21:59:18 2020 -0400 makepkg/repo-add: handle GPGKEY with spaces We pass this to gpg -u and this gpg option can accept a number of different formats, not just the historical hexadecimal fingerprint we assumed. We should not barf hard if a format is used which happens to contain spaces. This also fixes a validation bug. When we initially check if the desired key is available, we don't quote spaces, so gpg goes ahead and treats each space-separated string as a *different key* to search for, returning partial matches, and returning success if at least one key is found. But gpg --detach-sign -u will certainly not accept multiple keys! Fixes FS#66949 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 899d39b635d46f9e2daff1aada75ea07f08fef64) commit f4da297de252dcb00e6c83df00fea11c86606348 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Tue Jun 2 17:50:24 2020 -0400 libmakepkg: fix regression in sending plain() output to stderr In commit 882e707e40bbade0111cf3bdedbdac4d4b70453b we changed message output to go to stdout by default, unless it was an error. The plain() function doesn't *look* like an error function, but in practice it was -- it's used to continue multiline messages, and all in-tree uses were for warning/error. This is a problem both because we're sending output to the wrong place, and because in some cases, we were performing error logging from a function which would otherwise return a value to be captured in a variable using command substution. Fix this and straighten out the API by providing two functions: one for continuing msg output, and one which wraps this by sending output to stderr, for continuing error output. Change all callers to use the second function. (cherry picked from commit bf458cced7c0845f7b6fabb887d3878ae4cd51b2) commit 22e6daa79455b1c59ab73af9438ca0c1d2a884b9 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Tue Jun 2 18:16:48 2020 -0400 makepkg: correctly handle missing download clients This was broken in commit 882e707e40bbade0111cf3bdedbdac4d4b70453b, which changed 'plain()' messages to go to stdout, which was then captured as the download client in question: cmdline=("Aborting..."). The result was a very confusing error message e.g. /usr/share/makepkg/source/file.sh: line 72: $'\E[1m': command not found or with makepkg --nocolor: /usr/share/makepkg/source/file.sh: line 72: Aborting...: command not found The problem here is that we checked to see if an asynchronous subshell, in our case <(...), failed, by checking if its captured stdout is non-empty. Which is terrible, and also a limitation of old bash. But bash 4.4 can use wait $! to retrieve the return value of an asynchronous subshell. Now we target that as our minimum, we can sanely handle errors in such functions. Losing error messages on stdout by capturing them in a variable instead of printing them, continues to be a problem, but this will be fixed systematically in a later commit. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 381e11375569fa7588b1297e0e744749bdafe8f5) commit d69da08abef42a52d2fa81daf4c045c3305598cf Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon May 25 23:52:16 2020 -0400 makepkg: guard against undefined git pinned sources If something like source=(..."#commit=") is used, e.g. due to failed variable expansion, we try to check out an empty refspec as nothing at all, and end up just running "git checkout". This happens because we fail at variable expansion too -- so let's quote our variables properly and make sure git sees this as an empty refspec, so it can error out. Also make sure it is interpreted as a ref instead of a path. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 817f9fb715b4e90d28bc141dfccfc530b9e951dc) commit 12503767c0be8f3eb28433fd58bcab0ec6f44a26 Author: Allan McRae <allan(a)archlinux.org> Date: Wed May 20 14:17:11 2020 +1000 libalpm/signing.c: Fix calculation of packet size in parse_subpacket Given RFC 4880 provides the code to do this calculation, I am not sure how I managed to stuff that up! This bug was only exposed when a signature made with "include-key-block" was added to the Arch repos, which provided a subpacket with the required size to hit this issue. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 5f6ef895b1dac04c7fb1b63acab2d42c19f91922) commit 03cfe9e21c6aaecaac21e932229a55f5f1041a3c Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon May 11 00:16:30 2020 -0400 libmakepkg/strip: don't re-add the same debug source multiple times It's either a waste of work, or triggers edge cases in some packages (like coreutils-8.31) where the source file is readonly and cp gets a permission denied error trying to overwrite it with an identical copy of itself. Also while we are at it, make the variable names be something readable, because I could barely tell what this was doing while editing it. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 3674144a74cfe897ec3ff46c18681df293290caa) commit b3be0ce99b29e5384969c7b3e5ff6f4b0f581c4b Author: Levente Polyak <anthraxx(a)archlinux.org> Date: Mon May 11 00:45:49 2020 +0200 makepkg: deterministic PKGINFO libprovides for multiple library versions While iterating over the provides array, the find call for locating a shared library may result in listing multiple entries which by itself does not produce a stable deterministic order and may vary depending on the underlying filesystem. To provide a stable listing and a reproducible .PKGINFO file the result of find is piped to sort with a static LC_ALL=C localisation. Signed-off-by: Levente Polyak <anthraxx(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 8ce142a2552418f64a74e773f659d92b065d6209) commit 68418c54427b2ecbd5f882ffad261e0ba6e67c42 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun May 10 00:32:39 2020 -0400 Log invalid conf settings as an error This is not a warning, _parse_options() returns failure without even parsing further lines and the attempted pacman/pacman-conf program execution immediately aborts. Warnings are for when e.g. later on if we don't recognize a setting at all, we skip over it and have enough confidence in this to continue executing the program. The current implementation results in pacman-conf aborting with: warning: config file /etc/pacman.conf, line 60: invalid value for 'ParallelDownloads' : '2.5' error parsing '/etc/pacman.conf' or pacman -Syu aborting with the entirely more cryptic: warning: config file /etc/pacman.conf, line 59: invalid value for 'ParallelDownloads' : '2.5' and this isn't just a problem for the newly added ParallelDownloads setting, either, you could get the same problem if you specified a broken XferCommand, but that's harder as it's more accepting of input and you probably don't hit this except with unbalanced quotes. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 8e769ddb8a59a9fbacf4614283d2fb519f022386) commit bb50e8d73afda7f17777c464bd7bfa127a749b8d Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Sun May 10 00:32:40 2020 -0400 pacman-conf: fix incomplete support for ILoveCandy This was only partially implemented in the original implementation. `pacman-conf | grep ILoveCandy` would tell you if it was set, but querying directly by name would not. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 7423b166047ff168ef337818958bf86ed1d42a0a) commit b9d397c7317b9b495da73051fc670b0e9f472a06 Author: Allan McRae <allan(a)archlinux.org> Date: Tue Apr 14 23:45:24 2020 +1000 pacman.8: Fix typo Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 3ea7fdd96c874e111588a6e64ee30b75b55db785) commit 39ce2b766303f046bb75ad900cc8a2695e947f83 Author: Earnestly <zibeon(a)googlemail.com> Date: Wed Apr 1 11:29:57 2020 -0400 Use noextract with pacman-conf NoExtract Current code accidently uses noupgrade for the NoExtract directive. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 1d39557aa070d2260cfda650e59c8d190397ba01) commit 01d5a68c1ae07ad43b5255721d2429710794ea04 Author: Allan McRae <allan(a)archlinux.org> Date: Mon Feb 10 10:46:03 2020 +1000 build-aux/update-copyright 2019 2020 Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit e76ec94083235ddc5510ab57b7c2bc12a1d34e8a) commit 08aa3d97ea3823f2919e7d66041438db9e7488a0 Author: morganamilo <morganamilo(a)archlinux.org> Date: Thu Jan 23 06:38:35 2020 +0000 libalpm: fix alpm_option_set_assumeinstalled It looks like this function has never actually worked. The current list is never set to NULL after being freed. So the new deps were just appended to the already freed list, leading to a segfault. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 0a25548cd0910f66dea2dfab21f75a6d15366d64) commit 0bf4779cda35cfb8dc049ec5f27017c65c8ce716 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Jan 16 12:27:34 2020 -0500 makepkg: make per-package files containing '$pkgname' consistently work Extracting function variables containing arbitrarily scoped variables of arbitrary nature is a disaster, but let's at least cover the common case of using the actual '$pkgname' in an install/changelog file. It's the odd case of actually being basically justified use of disambiguating between the same variable used in multiple different split packages... and also, --printsrcinfo already uses and overwrites the variable 'pkgname' in pkgbuild_extract_to_srcinfo, so this "works" in .SRCINFO but doesn't work in .src.tar.gz It doesn't work in lint_pkgbuild either, but in that case the problem is being too permissive, not too restrictive -- we might end up checking the same file twice, and printing that it is missing twice. Fixes FS#64932 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit d626a17ef94d79975ff17d04250152ea46aaade3) commit d61c398b2cb3ec41e62a6e258b5791d0ce9de720 Author: Allan McRae <allan(a)archlinux.org> Date: Thu Jan 23 10:47:02 2020 +1000 Fix "pacman -U <url>" operations Commit e6a6d307 detected complete part files by comparing a payload's max_size to initial_size. However, these values are also equal when we use pacman -U on a URL as max_size is set to 0 in that case. Add a further condition to avoid that. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit e54617c7d554e0c14c039432b5f7bef66e43769c) commit 7faa79526874c09fc89fb769744d103bc4918e17 Author: Allan McRae <allan(a)archlinux.org> Date: Sun Jan 19 09:30:54 2020 +1000 Increase maximum database size We previously has the maximum database size as 25MB. This was set in the days before repos had as many packages as they do now, and before we started distributing files databases. Increase this limit to 128MB. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 2856a7dea3c0d4584e126b5ca5957e13e23f83d1) commit 0d0a4bd68099596ba17b8c314664484c3819fbf3 Author: Allan McRae <allan(a)archlinux.org> Date: Thu Jan 16 21:20:19 2020 +1000 Remove unneeded ltmain patch Many moons ago, libtool was bad - I mean worse than today! It gobbled all --as-needed and we ended up with an overlinked libalpm. This was annoying, particularly when dealing with soname bumps in libraries pacman/libalpm had no business linking to. Luckily we had a fix, stolen from GNOME I believe. And with that fix, we lived in harmony with libtool for many years. Until one day, unbeknownst to us, libtool was "fixed". We kept applying our patch, because it still applied, but it did worse than nothing. It gobbled up our other LDFLAGS, and our libalpm started missing out on RELRO and BIND_NOW. This made the Arch Security Team unhappy. We will make them happy again by stopping the patch. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 2e32d51a262fef398a94a17c0a0d4ff3233990b5) [Andrew: restore autogen.sh] commit 76c50e34393d68a740f825a6af9df8b7ac677303 Author: Allan McRae <allan(a)archlinux.org> Date: Fri Nov 15 23:29:20 2019 +1000 Handle .part files that are the size of the correct package In rare cases, likely due to a well timed Ctrl+C, but possibly due to a broken mirror, a ".part" file may have size at least that of the correct package size. When encountering this issue, currently pacman fails in different ways depending on where the package falls in the list to download. If last, "wrong or NULL argument passed" error is reported, or a "invalid or corrupt package" issue if not. Capture these .part files, and remove the extension. This lets pacman either use the package if valid, or offer to remove it if it fails checksum or signature verification. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit e6a6d3079315ce722fec39604fddbab1c7ac79b3) commit c2fa9f85cccc4b576816330762fb55167a1ba78c Author: Allan McRae <allan(a)archlinux.org> Date: Tue Nov 12 07:29:52 2019 +1000 libalpm/sync.c: Do not download missing keys multiple times We now store key structs of our missing key info, so can not search the list for string matches. This caused missing keys to be downloaded once for every package they signed. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 540b19164b1ab3a4950b4a828fb90d047f4a591d) commit 6f1a9e6ea85a49b8aeb3b7bd2454215e9c34f565 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon Nov 25 19:07:56 2019 -0500 makepkg: fix regression that broke extraction of file:// sources In commit 9c817b654996249b8022e189ee7e2692f4668431 we made these sources extendable, and heuristically determined the correct extraction functions to use. But our fallback for protos that didn't have an exact extract_* function didn't take into account that 'extract_file' matches an actual proto... so we passed the netfile in while the function expected a file. Solution: the function should expect a netfile too, thereby allowing us to delay an attempted resolution of netfile -> file, to the one case where it is actually used. This makes us slightly more efficient in the non-file case, makes our functions a bit more consistent, and makes file:// extraction work again. Fixes FS#64648 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit 349c22d043290ccd0cce9f30981f5415e295442a) commit 41c3b1d78c3807beb4246624b20a31e374fb5af1 Author: Daniel T. Borelli <danieltborelli(a)gmail.com> Date: Sat Nov 16 15:14:30 2019 +1000 Dereference double pointer before assigning NULL Daniel T. Borelli <danieltborelli(a)gmail.com> Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit eaa2770c80f8f3b168b99021e4258d696523d38a) commit d6dcc936457815cdda87a8507cf82169b437e4d2 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Thu Jan 9 20:59:35 2020 -0500 makepkg: fix one more file-seccomp issue When file is called via fakeroot, it doesn't matter whether you use -z or not, it is still incompatible with seccomp. Fix by configuring it with FILECMD when used in the fakeroot 'tidy' run. Fixes FS#65100 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> commit 019f9386ef2c85b4c3c9f9293f462cfb5ddcaa32 Author: Allan McRae <allan(a)archlinux.org> Date: Tue Nov 12 16:14:57 2019 +1000 pactest: set package tar format to GNU_FORMAT python-3.8 changed the default tar format to PAX_FORMAT. This caused issues in our testsuite with package extraction of files with UTF-8 characters as we run the tests under the C locale. sycn600.py: error: error while reading package /tmp/pactest-xuhri4xa/var/cache/pacman/pkg/unicodechars-2.0-1.pkg.tar.gz: Pathname can't be converted from UTF-8 to current locale. Set format back to GNU_FORMAT. Signed-off-by: Allan McRae <allan(a)archlinux.org> (cherry picked from commit b9faf652735c603d1bdf849a570185eb721f11c1) ----------------------------------------------------------------------- Summary of changes: autogen.sh | 1 - build-aux/ltmain-asneeded.patch | 33 --------------------- doc/index.asciidoc | 2 +- doc/pacman.8.asciidoc | 2 +- lib/libalpm/add.c | 2 +- lib/libalpm/add.h | 2 +- lib/libalpm/alpm.c | 2 +- lib/libalpm/alpm.h | 2 +- lib/libalpm/alpm_list.c | 2 +- lib/libalpm/alpm_list.h | 2 +- lib/libalpm/backup.c | 2 +- lib/libalpm/backup.h | 2 +- lib/libalpm/be_local.c | 2 +- lib/libalpm/be_package.c | 2 +- lib/libalpm/be_sync.c | 6 ++-- lib/libalpm/conflict.c | 2 +- lib/libalpm/conflict.h | 2 +- lib/libalpm/db.c | 2 +- lib/libalpm/db.h | 2 +- lib/libalpm/deps.c | 2 +- lib/libalpm/deps.h | 2 +- lib/libalpm/diskspace.c | 2 +- lib/libalpm/diskspace.h | 2 +- lib/libalpm/dload.c | 8 ++++- lib/libalpm/dload.h | 2 +- lib/libalpm/error.c | 2 +- lib/libalpm/filelist.c | 2 +- lib/libalpm/filelist.h | 2 +- lib/libalpm/graph.c | 2 +- lib/libalpm/graph.h | 2 +- lib/libalpm/group.c | 2 +- lib/libalpm/group.h | 2 +- lib/libalpm/handle.c | 3 +- lib/libalpm/handle.h | 2 +- lib/libalpm/hook.c | 2 +- lib/libalpm/hook.h | 2 +- lib/libalpm/libarchive-compat.h | 2 +- lib/libalpm/log.c | 2 +- lib/libalpm/log.h | 2 +- lib/libalpm/package.c | 2 +- lib/libalpm/package.h | 2 +- lib/libalpm/pkghash.c | 2 +- lib/libalpm/pkghash.h | 2 +- lib/libalpm/remove.c | 2 +- lib/libalpm/remove.h | 2 +- lib/libalpm/signing.c | 8 ++--- lib/libalpm/signing.h | 2 +- lib/libalpm/sync.c | 28 ++++++++++++++---- lib/libalpm/sync.h | 2 +- lib/libalpm/trans.c | 2 +- lib/libalpm/trans.h | 2 +- lib/libalpm/util.c | 2 +- lib/libalpm/util.h | 2 +- lib/libalpm/version.c | 2 +- scripts/libmakepkg/buildenv.sh.in | 2 +- scripts/libmakepkg/buildenv/buildflags.sh.in | 2 +- scripts/libmakepkg/buildenv/compiler.sh.in | 2 +- scripts/libmakepkg/buildenv/debugflags.sh.in | 2 +- scripts/libmakepkg/buildenv/makeflags.sh.in | 2 +- scripts/libmakepkg/executable.sh.in | 2 +- scripts/libmakepkg/executable/ccache.sh.in | 2 +- scripts/libmakepkg/executable/checksum.sh.in | 2 +- scripts/libmakepkg/executable/distcc.sh.in | 2 +- scripts/libmakepkg/executable/fakeroot.sh.in | 2 +- scripts/libmakepkg/executable/gpg.sh.in | 2 +- scripts/libmakepkg/executable/gzip.sh.in | 2 +- scripts/libmakepkg/executable/pacman.sh.in | 2 +- scripts/libmakepkg/executable/strip.sh.in | 2 +- scripts/libmakepkg/executable/sudo.sh.in | 2 +- scripts/libmakepkg/executable/vcs.sh.in | 4 +-- scripts/libmakepkg/integrity.sh.in | 2 +- .../libmakepkg/integrity/generate_checksum.sh.in | 2 +- .../libmakepkg/integrity/generate_signature.sh.in | 8 ++--- scripts/libmakepkg/integrity/verify_checksum.sh.in | 2 +- .../libmakepkg/integrity/verify_signature.sh.in | 4 +-- scripts/libmakepkg/lint_config.sh.in | 2 +- scripts/libmakepkg/lint_config/ext.sh.in | 2 +- scripts/libmakepkg/lint_config/paths.sh.in | 2 +- .../libmakepkg/lint_config/source_date_epoch.sh.in | 2 +- scripts/libmakepkg/lint_config/variable.sh.in | 2 +- scripts/libmakepkg/lint_package.sh.in | 2 +- .../libmakepkg/lint_package/build_references.sh.in | 2 +- scripts/libmakepkg/lint_package/dotfiles.sh.in | 2 +- scripts/libmakepkg/lint_package/file_names.sh.in | 2 +- .../libmakepkg/lint_package/missing_backup.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/arch.sh.in | 2 +- .../libmakepkg/lint_pkgbuild/arch_specific.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/backup.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/changelog.sh.in | 12 +++++--- .../libmakepkg/lint_pkgbuild/checkdepends.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/conflicts.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/depends.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/epoch.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/fullpkgver.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/install.sh.in | 12 +++++--- scripts/libmakepkg/lint_pkgbuild/makedepends.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/optdepends.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/options.sh.in | 2 +- .../lint_pkgbuild/package_function.sh.in | 2 +- .../lint_pkgbuild/package_function_variable.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/pkgbase.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/pkglist.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/pkgname.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/pkgrel.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/pkgver.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/provides.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/source.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/util.sh.in | 2 +- scripts/libmakepkg/lint_pkgbuild/variable.sh.in | 2 +- scripts/libmakepkg/source.sh.in | 5 ++-- scripts/libmakepkg/source/bzr.sh.in | 10 +++---- scripts/libmakepkg/source/file.sh.in | 11 +++---- scripts/libmakepkg/source/git.sh.in | 18 ++++++------ scripts/libmakepkg/source/hg.sh.in | 10 +++---- scripts/libmakepkg/source/local.sh.in | 2 +- scripts/libmakepkg/source/svn.sh.in | 6 ++-- scripts/libmakepkg/srcinfo.sh.in | 2 +- scripts/libmakepkg/tidy.sh.in | 2 +- scripts/libmakepkg/tidy/docs.sh.in | 2 +- scripts/libmakepkg/tidy/emptydirs.sh.in | 2 +- scripts/libmakepkg/tidy/libtool.sh.in | 2 +- scripts/libmakepkg/tidy/purge.sh.in | 2 +- scripts/libmakepkg/tidy/staticlibs.sh.in | 2 +- scripts/libmakepkg/tidy/strip.sh.in | 15 ++++++---- scripts/libmakepkg/tidy/zipman.sh.in | 2 +- scripts/libmakepkg/util.sh.in | 2 +- scripts/libmakepkg/util/compress.sh.in | 2 +- scripts/libmakepkg/util/config.sh.in | 4 +-- scripts/libmakepkg/util/dirsize.sh.in | 2 +- scripts/libmakepkg/util/error.sh.in | 2 +- scripts/libmakepkg/util/message.sh.in | 10 ++++++- scripts/libmakepkg/util/option.sh.in | 2 +- scripts/libmakepkg/util/parseopts.sh.in | 2 +- scripts/libmakepkg/util/pkgbuild.sh.in | 2 +- scripts/libmakepkg/util/schema.sh.in | 2 +- scripts/libmakepkg/util/source.sh.in | 6 ++-- scripts/libmakepkg/util/util.sh.in | 4 +-- scripts/makepkg-template.pl.in | 4 +-- scripts/makepkg.sh.in | 34 ++++++++++++---------- scripts/pacman-db-upgrade.sh.in | 4 +-- scripts/pacman-key.sh.in | 4 +-- scripts/repo-add.sh.in | 12 ++++---- scripts/wrapper.sh.in | 2 +- src/common/ini.c | 2 +- src/common/ini.h | 2 +- src/common/util-common.c | 2 +- src/common/util-common.h | 2 +- src/pacman/callback.c | 2 +- src/pacman/callback.h | 2 +- src/pacman/check.c | 2 +- src/pacman/check.h | 2 +- src/pacman/conf.c | 4 +-- src/pacman/conf.h | 2 +- src/pacman/database.c | 2 +- src/pacman/deptest.c | 2 +- src/pacman/files.c | 2 +- src/pacman/package.c | 2 +- src/pacman/package.h | 2 +- src/pacman/pacman-conf.c | 6 ++-- src/pacman/pacman.c | 4 +-- src/pacman/pacman.h | 2 +- src/pacman/query.c | 2 +- src/pacman/remove.c | 2 +- src/pacman/sighandler.c | 2 +- src/pacman/sighandler.h | 2 +- src/pacman/sync.c | 2 +- src/pacman/upgrade.c | 2 +- src/pacman/util.c | 2 +- src/pacman/util.h | 2 +- src/util/testpkg.c | 2 +- src/util/vercmp.c | 2 +- test/pacman/pactest.py | 2 +- test/pacman/pmdb.py | 2 +- test/pacman/pmenv.py | 2 +- test/pacman/pmfile.py | 2 +- test/pacman/pmpkg.py | 4 +-- test/pacman/pmrule.py | 2 +- test/pacman/pmtest.py | 2 +- test/pacman/tap.py | 2 +- test/pacman/util.py | 2 +- test/util/vercmptest.sh | 2 +- 182 files changed, 309 insertions(+), 293 deletions(-) delete mode 100644 build-aux/ltmain-asneeded.patch hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [PATCH] Adds a check to -Qo, so that pacman warns the user if the file does not exits
by Pedro Victor 18 Jun '20

18 Jun '20

From: Pedro Victor <pedrocga1(a)gmail.com> --- src/pacman/query.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/pacman/query.c b/src/pacman/query.c index 513ddb7a..784936b9 100644 --- a/src/pacman/query.c +++ b/src/pacman/query.c @@ -209,6 +209,11 @@ static int query_fileowner(alpm_list_t *targets) strcat(rpath + rlen, "/"); } + if (access(rel_path, F_OK) == -1) { + pm_printf(ALPM_LOG_ERROR, _("File does not exist: %s\n"), rel_path); + goto targcleanup; + } + for(i = packages; i && (!found || is_dir); i = alpm_list_next(i)) { if(alpm_filelist_contains(alpm_pkg_get_files(i->data), rel_path)) { print_query_fileowner(rpath, i->data); -- 2.27.0

2 3

[pacman-dev] [PATCH] pacman-key: change signing key to RSA4096
by Allan McRae 15 Jun '20

15 Jun '20

RSA2048 may have been fine when this was written many moons ago, but time this has a bump. Signed-off-by: Allan McRae <allan(a)archlinux.org> --- scripts/pacman-key.sh.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/pacman-key.sh.in b/scripts/pacman-key.sh.in index 3e952c1b..ccfd1b96 100644 --- a/scripts/pacman-key.sh.in +++ b/scripts/pacman-key.sh.in @@ -148,7 +148,7 @@ generate_master_key() { "${GPG_PACMAN[@]}" --gen-key --batch <<EOF %echo Generating pacman keyring master key... Key-Type: RSA -Key-Length: 2048 +Key-Length: 4096 Key-Usage: sign Name-Real: Pacman Keyring Master Key Name-Email: pacman@localhost -- 2.27.0

1 0

[pacman-dev] [PATCH] doc/pacman.8: fix typo
by Eli Schwartz 14 Jun '20

14 Jun '20

Fixes FS#67000 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- doc/pacman.8.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/pacman.8.asciidoc b/doc/pacman.8.asciidoc index fee5aa15..476c16f3 100644 --- a/doc/pacman.8.asciidoc +++ b/doc/pacman.8.asciidoc @@ -141,7 +141,7 @@ Options guest system. See '\--sysroot' instead. *-v, \--verbose*:: - Output paths such as as the Root, Conf File, DB Path, Cache Dirs, etc. + Output paths such as the Root, Conf File, DB Path, Cache Dirs, etc. *\--arch* <arch>:: Specify an alternate architecture. -- 2.27.0

1 0

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.2.1-95-g02ae97b0
by Allan McRae 11 Jun '20

11 Jun '20

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 02ae97b0da220d9079c6c2c1ac3e3ab0d12c1ac7 (commit) via 899d39b635d46f9e2daff1aada75ea07f08fef64 (commit) via bf458cced7c0845f7b6fabb887d3878ae4cd51b2 (commit) via 381e11375569fa7588b1297e0e744749bdafe8f5 (commit) via 817f9fb715b4e90d28bc141dfccfc530b9e951dc (commit) via 3bd88821bbfc8066a63ddfc9959e78984bc17750 (commit) via e348ba38814c9cc7c1c9892d0451096234dc39ab (commit) via 40bbaead44db8fcdef7087f4b05820bf90dee090 (commit) from 5f6ef895b1dac04c7fb1b63acab2d42c19f91922 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 02ae97b0da220d9079c6c2c1ac3e3ab0d12c1ac7 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon Jun 8 22:03:18 2020 -0400 makepkg/repo-add: do not accept public-only keys for signing If it's not listed by --list-secret-key we don't care if it has been imported into your keyring, it's unusable. And you might not have a private key at all in the no-keyid-specified case. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 899d39b635d46f9e2daff1aada75ea07f08fef64 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon Jun 8 21:59:18 2020 -0400 makepkg/repo-add: handle GPGKEY with spaces We pass this to gpg -u and this gpg option can accept a number of different formats, not just the historical hexadecimal fingerprint we assumed. We should not barf hard if a format is used which happens to contain spaces. This also fixes a validation bug. When we initially check if the desired key is available, we don't quote spaces, so gpg goes ahead and treats each space-separated string as a *different key* to search for, returning partial matches, and returning success if at least one key is found. But gpg --detach-sign -u will certainly not accept multiple keys! Fixes FS#66949 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit bf458cced7c0845f7b6fabb887d3878ae4cd51b2 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Tue Jun 2 17:50:24 2020 -0400 libmakepkg: fix regression in sending plain() output to stderr In commit 882e707e40bbade0111cf3bdedbdac4d4b70453b we changed message output to go to stdout by default, unless it was an error. The plain() function doesn't *look* like an error function, but in practice it was -- it's used to continue multiline messages, and all in-tree uses were for warning/error. This is a problem both because we're sending output to the wrong place, and because in some cases, we were performing error logging from a function which would otherwise return a value to be captured in a variable using command substution. Fix this and straighten out the API by providing two functions: one for continuing msg output, and one which wraps this by sending output to stderr, for continuing error output. Change all callers to use the second function. commit 381e11375569fa7588b1297e0e744749bdafe8f5 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Tue Jun 2 18:16:48 2020 -0400 makepkg: correctly handle missing download clients This was broken in commit 882e707e40bbade0111cf3bdedbdac4d4b70453b, which changed 'plain()' messages to go to stdout, which was then captured as the download client in question: cmdline=("Aborting..."). The result was a very confusing error message e.g. /usr/share/makepkg/source/file.sh: line 72: $'\E[1m': command not found or with makepkg --nocolor: /usr/share/makepkg/source/file.sh: line 72: Aborting...: command not found The problem here is that we checked to see if an asynchronous subshell, in our case <(...), failed, by checking if its captured stdout is non-empty. Which is terrible, and also a limitation of old bash. But bash 4.4 can use wait $! to retrieve the return value of an asynchronous subshell. Now we target that as our minimum, we can sanely handle errors in such functions. Losing error messages on stdout by capturing them in a variable instead of printing them, continues to be a problem, but this will be fixed systematically in a later commit. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 817f9fb715b4e90d28bc141dfccfc530b9e951dc Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon May 25 23:52:16 2020 -0400 makepkg: guard against undefined git pinned sources If something like source=(..."#commit=") is used, e.g. due to failed variable expansion, we try to check out an empty refspec as nothing at all, and end up just running "git checkout". This happens because we fail at variable expansion too -- so let's quote our variables properly and make sure git sees this as an empty refspec, so it can error out. Also make sure it is interpreted as a ref instead of a path. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit 3bd88821bbfc8066a63ddfc9959e78984bc17750 Author: Eli Schwartz <eschwartz(a)archlinux.org> Date: Mon May 18 15:18:12 2020 -0400 build: add libintl dependency to meson and the .pc file In order to use gettext on systems where it is not part of libc, the correct linker flags are needed in libalpm.pc (for static compilation). This has never been the case. The new meson build system currently only checks for ngettext in libc, but does not fall back to searching for the existence of -lintl; add it to the libalpm dependencies. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> Signed-off-by: Allan McRae <allan(a)archlinux.org> commit e348ba38814c9cc7c1c9892d0451096234dc39ab Author: Filipe Laíns <lains(a)archlinux.org> Date: Thu May 21 00:38:41 2020 +0100 ci: cache packages Results in ~40s saved in each job. Signed-off-by: Filipe Laíns <lains(a)archlinux.org> commit 40bbaead44db8fcdef7087f4b05820bf90dee090 Author: Allan McRae <allan(a)archlinux.org> Date: Tue Mar 17 14:03:58 2020 +1000 Revert "makepkg: drop duplicate reporting of missing dependencies" This removed all information on dependency failures if the --syncdeps flag was not used. A better approach is needed. This reverts commit 4246a4cc4f0f87642cbbb6b375524b2e4c713412. Signed-off-by: Allan McRae <allan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: .gitlab-ci.yml | 15 +++++++++++--- lib/libalpm/libalpm.pc.in | 2 +- meson.build | 8 ++++++-- scripts/libmakepkg/executable/vcs.sh.in | 2 +- .../libmakepkg/integrity/generate_signature.sh.in | 6 +++--- .../libmakepkg/integrity/verify_signature.sh.in | 2 +- scripts/libmakepkg/source/bzr.sh.in | 8 ++++---- scripts/libmakepkg/source/file.sh.in | 6 +++--- scripts/libmakepkg/source/git.sh.in | 16 +++++++-------- scripts/libmakepkg/source/hg.sh.in | 8 ++++---- scripts/libmakepkg/source/svn.sh.in | 4 ++-- scripts/libmakepkg/util/config.sh.in | 2 +- scripts/libmakepkg/util/message.sh.in | 8 ++++++++ scripts/libmakepkg/util/source.sh.in | 4 ++-- scripts/libmakepkg/util/util.sh.in | 2 +- scripts/makepkg.sh.in | 24 ++++++++++++++-------- scripts/repo-add.sh.in | 8 ++++---- 17 files changed, 76 insertions(+), 49 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [PATCH] srcinfo.sh: remove trailing newline
by Denton Liu 11 Jun '20

11 Jun '20

When the .SRCINFO file is generated via `makepkg --printsrcinfo`, each section is concluded with an empty line. This means that at the end of the file, an empty line remains. When running `git diff --check`, Git will complain about this as a whitespace error, saying "new blank line at EOF." Remove the empty line after sections. Replace the empty echo with a placeholder `true` call in case in the future, we do want to close the section with something. --- scripts/libmakepkg/srcinfo.sh.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/libmakepkg/srcinfo.sh.in b/scripts/libmakepkg/srcinfo.sh.in index 6e783279..e7b5c4be 100644 --- a/scripts/libmakepkg/srcinfo.sh.in +++ b/scripts/libmakepkg/srcinfo.sh.in @@ -31,7 +31,7 @@ srcinfo_open_section() { } srcinfo_close_section() { - echo + true # nothing to be done } srcinfo_write_attr() { -- 2.27.0.rc1.69.g3229fa6e15

4 8

[pacman-dev] [PATCH] makepkg: guard against undefined git pinned sources
by Eli Schwartz 11 Jun '20

11 Jun '20

If something like source=(..."#commit=") is used, e.g. due to failed variable expansion, we try to check out an empty refspec as nothing at all, and end up just running "git checkout". This happens because we fail at variable expansion too -- so let's quote our variables properly and make sure git sees this as an empty refspec, so it can error out. Also make sure it is interpreted as a ref instead of a path. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- This ensures that something like https://bugs.archlinux.org/task/66729 cannot happen again. scripts/libmakepkg/source/git.sh.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/libmakepkg/source/git.sh.in b/scripts/libmakepkg/source/git.sh.in index aee944f7..a29be3c5 100644 --- a/scripts/libmakepkg/source/git.sh.in +++ b/scripts/libmakepkg/source/git.sh.in @@ -125,7 +125,7 @@ extract_git() { fi if [[ $ref != "origin/HEAD" ]] || (( updating )) ; then - if ! git checkout --force --no-track -B makepkg $ref; then + if ! git checkout --force --no-track -B makepkg "$ref" --; then error "$(gettext "Failure while creating working copy of %s %s repo")" "${repo}" "git" plain "$(gettext "Aborting...")" exit 1 -- 2.26.2

2 2

[pacman-dev] [PATCH] makepkg/repo-add: handle GPGKEY with spaces
by Eli Schwartz 11 Jun '20

11 Jun '20

We pass this to gpg -u and this gpg option can accept a number of different formats, not just the historical hexadecimal fingerprint we assumed. We should not barf hard if a format is used which happens to contain spaces. This also fixes a validation bug. When we initially check if the desired key is available, we don't quote spaces, so gpg goes ahead and treats each space-separated string as a *different key* to search for, returning partial matches, and returning success if at least one key is found. But gpg --detach-sign -u will certainly not accept multiple keys! Fixes FS#66949 Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- scripts/libmakepkg/integrity/generate_signature.sh.in | 6 +++--- scripts/makepkg.sh.in | 2 +- scripts/repo-add.sh.in | 8 ++++---- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/scripts/libmakepkg/integrity/generate_signature.sh.in b/scripts/libmakepkg/integrity/generate_signature.sh.in index aec96c03..748087c2 100644 --- a/scripts/libmakepkg/integrity/generate_signature.sh.in +++ b/scripts/libmakepkg/integrity/generate_signature.sh.in @@ -29,12 +29,12 @@ create_signature() { local ret=0 local filename="$1" - local SIGNWITHKEY="" + local SIGNWITHKEY=() if [[ -n $GPGKEY ]]; then - SIGNWITHKEY="-u ${GPGKEY}" + SIGNWITHKEY=(-u "${GPGKEY}") fi - gpg --detach-sign --use-agent ${SIGNWITHKEY} --no-armor "$filename" &>/dev/null || ret=$? + gpg --detach-sign --use-agent "${SIGNWITHKEY[@]}" --no-armor "$filename" &>/dev/null || ret=$? if (( ! ret )); then diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 7261fb2c..84150cff 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1293,7 +1293,7 @@ fi # check if gpg signature is to be created and if signing key is valid if { [[ -z $SIGNPKG ]] && check_buildenv "sign" "y"; } || [[ $SIGNPKG == 'y' ]]; then SIGNPKG='y' - if ! gpg --list-key ${GPGKEY} &>/dev/null; then + if ! gpg --list-key "${GPGKEY}" &>/dev/null; then if [[ ! -z $GPGKEY ]]; then error "$(gettext "The key %s does not exist in your keyring.")" "${GPGKEY}" else diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index 545c2929..b1b1b2b7 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -137,7 +137,7 @@ check_gpg() { fi if (( ! VERIFY )); then - if ! gpg --list-key ${GPGKEY} &>/dev/null; then + if ! gpg --list-key "${GPGKEY}" &>/dev/null; then if [[ ! -z $GPGKEY ]]; then error "$(gettext "The key ${GPGKEY} does not exist in your keyring.")" elif (( ! KEY )); then @@ -155,11 +155,11 @@ create_signature() { local ret=0 msg "$(gettext "Signing database '%s'...")" "${dbfile##*/.tmp.}" - local SIGNWITHKEY="" + local SIGNWITHKEY=() if [[ -n $GPGKEY ]]; then - SIGNWITHKEY="-u ${GPGKEY}" + SIGNWITHKEY=(-u "${GPGKEY}") fi - gpg --detach-sign --use-agent --no-armor ${SIGNWITHKEY} "$dbfile" &>/dev/null || ret=$? + gpg --detach-sign --use-agent --no-armor "${SIGNWITHKEY[@]}" "$dbfile" &>/dev/null || ret=$? if (( ! ret )); then msg2 "$(gettext "Created signature file '%s'")" "${dbfile##*/.tmp.}.sig" -- 2.27.0

2 2

[pacman-dev] [PATCH] makepkg: correctly handle missing download clients
by Eli Schwartz 11 Jun '20

11 Jun '20

This was broken in commit 882e707e40bbade0111cf3bdedbdac4d4b70453b, which changed 'plain()' messages to go to stdout, which was then captured as the download client in question: cmdline=("Aborting..."). The result was a very confusing error message e.g. /usr/share/makepkg/source/file.sh: line 72: $'\E[1m': command not found or with makepkg --nocolor: /usr/share/makepkg/source/file.sh: line 72: Aborting...: command not found Solve this on two levels: - redirect this error() continuation to stderr so the user sees it. - catch erroring returns in get_downloadclient and propagate them bash 4.4 can use wait $! to retrieve the return value of an asynchronous subshell such as <(...), which means, now we target that as our minimum, we can sanely handle errors in such functions. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- Actually, maybe every use of plain() needs to do this. Or else make plain() do this by default. But it's technically used to "continue the previous message", and there's no guarantee it merits going to stderr, even though every time we use plain(), it does. What to do... scripts/libmakepkg/source/file.sh.in | 1 + scripts/libmakepkg/util/source.sh.in | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/libmakepkg/source/file.sh.in b/scripts/libmakepkg/source/file.sh.in index 819320c2..28f373fb 100644 --- a/scripts/libmakepkg/source/file.sh.in +++ b/scripts/libmakepkg/source/file.sh.in @@ -42,6 +42,7 @@ download_file() { # find the client we should use for this URL local -a cmdline IFS=' ' read -a cmdline < <(get_downloadclient "$proto") + wait $! || exit (( ${#cmdline[@]} )) || exit local filename=$(get_filename "$netfile") diff --git a/scripts/libmakepkg/util/source.sh.in b/scripts/libmakepkg/util/source.sh.in index e0490661..1bf5b814 100644 --- a/scripts/libmakepkg/util/source.sh.in +++ b/scripts/libmakepkg/util/source.sh.in @@ -165,7 +165,7 @@ get_downloadclient() { if [[ ! -x $program ]]; then local baseprog="${program##*/}" error "$(gettext "The download program %s is not installed.")" "$baseprog" - plain "$(gettext "Aborting...")" + plain "$(gettext "Aborting...")" >&2 exit 1 # $E_MISSING_PROGRAM fi -- 2.26.2

2 4

[pacman-dev] [PATCH] build: add libintl dependency to meson and the .pc file
by Eli Schwartz 11 Jun '20

11 Jun '20

In order to use gettext on systems where it is not part of libc, the correct linker flags are needed in libalpm.pc (for static compilation). This has never been the case. The new meson build system currently only checks for ngettext in libc, but does not fall back to searching for the existence of -lintl; add it to the libalpm dependencies. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- Fixes build on MSYS2, and possibly others. Part of https://github.com/msys2/MSYS2-packages/pull/1951 lib/libalpm/libalpm.pc.in | 2 +- meson.build | 8 ++++++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/lib/libalpm/libalpm.pc.in b/lib/libalpm/libalpm.pc.in index 126a6e6a..da7f96aa 100644 --- a/lib/libalpm/libalpm.pc.in +++ b/lib/libalpm/libalpm.pc.in @@ -10,4 +10,4 @@ Version: @LIB_VERSION@ Requires.private: libarchive @pc_crypto@ @pc_libcurl@ @pc_gpgme@ Cflags: -I${includedir} @LFS_CFLAGS@ Libs: -L${libdir} -lalpm -Libs.private: @LIBS@ @pc_gpgme_libs@ +Libs.private: @LIBS@ @pc_gpgme_libs@ @LIBINTL@ diff --git a/meson.build b/meson.build index 331f02b8..aa3bd4d9 100644 --- a/meson.build +++ b/meson.build @@ -77,9 +77,13 @@ conf.set_quoted('CACHEDIR', join_paths(LOCALSTATEDIR, 'cache/pacman/pkg/')) conf.set_quoted('HOOKDIR', join_paths(SYSCONFDIR, 'pacman.d/hooks/')) conf.set_quoted('ROOTDIR', ROOTDIR) +libintl = dependency('', required: false) if get_option('i18n') if not cc.has_function('ngettext') - error('ngettext not found but NLS support requested') + libintl = cc.find_library('intl', required : false, static: get_option('buildstatic')) + if not libintl.found() + error('ngettext not found but NLS support requested') + endif endif conf.set('ENABLE_NLS', 1) endif @@ -301,7 +305,7 @@ libcommon = static_library( include_directories : includes, install : false) -alpm_deps = [crypto_provider, libarchive, libcurl, gpgme] +alpm_deps = [crypto_provider, libarchive, libcurl, libintl, gpgme] libalpm_a = static_library( 'alpm_objlib', -- 2.26.2

2 1

[pacman-dev] [PATCH] ci: cache packages
by Filipe Laíns 11 Jun '20

11 Jun '20

Results in ~40s saved in each job. Signed-off-by: Filipe Laíns <lains(a)archlinux.org> --- .gitlab-ci.yml | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 4961352eb..46da925cf 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,12 +1,19 @@ variables: MAKEFLAGS: "-j10" VERBOSE: 1 + PACMAN_OPTS: --needed --noconfirm --cachedir .pkg-cache + +cache: + key: pkgs-v1 + paths: + # For some reason Gitlab CI only supports storing cache/artifacts in a path relative to the build directory + - .pkg-cache .arch-test: image: archlinux/base before_script: - > - pacman -Syu --needed --noconfirm + pacman -Syu $PACMAN_OPTS base-devel git gpgme libarchive curl python @@ -30,14 +37,14 @@ arch-debug: arch-docs: extends: .arch-test script: - - pacman -Syu --needed --noconfirm asciidoc + - pacman -Syu $PACMAN_OPTS asciidoc - meson -Ddoc=enabled build - ninja -C build arch-autoconf: extends: .arch-test script: - - pacman -Syu --needed --noconfirm autoconf-archive + - pacman -Syu $PACMAN_OPTS autoconf-archive - autoreconf -i - ./configure --enable-warningflags --disable-doc - make @@ -46,7 +53,7 @@ arch-autoconf: arch-distcheck: extends: .arch-test script: - - pacman -Syu --needed --noconfirm autoconf-archive asciidoc + - pacman -Syu $PACMAN_OPTS autoconf-archive asciidoc - autoreconf -i - ./configure --enable-warningflags - fakechroot make distcheck @@ -54,7 +61,7 @@ arch-distcheck: arch-clang: extends: .arch-test script: - - pacman -Syu --needed --noconfirm clang + - pacman -Syu $PACMAN_OPTS clang - CC=clang meson build - ninja -C build - fakechroot meson test -C build @@ -62,7 +69,7 @@ arch-clang: arch-valgrind: extends: .arch-test script: - - pacman -Syu --needed --noconfirm autoconf-archive valgrind + - pacman -Syu $PACMAN_OPTS autoconf-archive valgrind - autoreconf -i - ./configure --enable-warningflags --disable-doc - make @@ -98,6 +105,7 @@ arch-no-nls: debian: image: debian:bullseye + cache: {} before_script: - apt update - > @@ -113,6 +121,7 @@ debian: fedora: image: fedora + cache: {} before_script: - > dnf -y install -- 2.26.2

2 1

[pacman-dev] [PATCH] makepkg/repo-add: do not accept public-only keys for signing
by Eli Schwartz 09 Jun '20

09 Jun '20

If it's not listed by --list-secret-key we don't care if it has been imported into your keyring, it's unusable. And you might not have a private key at all in the no-keyid-specified case. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- Overlaps previous patch to fix GPGKEY handling. scripts/makepkg.sh.in | 2 +- scripts/repo-add.sh.in | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index cc8de5aa..9e75ef17 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1293,7 +1293,7 @@ fi # check if gpg signature is to be created and if signing key is valid if { [[ -z $SIGNPKG ]] && check_buildenv "sign" "y"; } || [[ $SIGNPKG == 'y' ]]; then SIGNPKG='y' - if ! gpg --list-key ${GPGKEY:+"$GPGKEY"} &>/dev/null; then + if ! gpg --list-secret-key ${GPGKEY:+"$GPGKEY"} &>/dev/null; then if [[ ! -z $GPGKEY ]]; then error "$(gettext "The key %s does not exist in your keyring.")" "${GPGKEY}" else diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index 272d8d22..160fd93a 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -137,7 +137,7 @@ check_gpg() { fi if (( ! VERIFY )); then - if ! gpg --list-key ${GPGKEY:+"$GPGKEY"} &>/dev/null; then + if ! gpg --list-secret-key ${GPGKEY:+"$GPGKEY"} &>/dev/null; then if [[ ! -z $GPGKEY ]]; then error "$(gettext "The key ${GPGKEY} does not exist in your keyring.")" elif (( ! KEY )); then -- 2.27.0

1 0

[pacman-dev] makepkg: package_...() can export variables to each other
by Erich Eckner 05 Jun '20

05 Jun '20

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I stumbled upon the fact, that variables set in one package_...() function of a split package are accessible by following package_...() functions. Is this by design or may I try to provide a patch to restore env after the call to package_...()? Note: packaging-relevant variables (e.g. depends_x86_64) are cleaned across function calls. Simple example: - ------8<------8<------8<------ arch=(any) pkgbase=test pkgname=(ta tb) pkgver=0 pkgrel=1 package_ta() { _test='hi, there!' } package_tb() { echo "$_test" } - ------>8------>8------>8------ regards, Erich -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAl7Z9qsACgkQCu7JB1Xa e1pc+xAAtKJyAv0JgX0skAmVmMEit+q456ZwKJAL0Kxe8059RNrwzwHvpcvso183 Pc3eqSbxMIRbTGrA2dyweWX42uO8DJuq39LTYNyHQR6KPY7rtaoAMEq/ypF6prc9 EKfD8wq+E5P3i3Ti4YyS8kKfEYfPLvA6O99r/U4GI8GZiFwNoq8LwvKvMnXYxqS2 6oEviKxYFw8wXlYjcCOOTb6WzgNZVyXbpos3B1kx0yuDRW6qhqicVa0levnWUp+M wh8Cx4LXIwCJKLawaA3VzmGXnsbHQq6UatMKeT4z+OcFFHq80q19KJlR8mfP41mc 1Wt+PYwQm+6zyekzcuzz/b6CX0RHK/hEHJJgXGu1awxpzADAWAt3l0HvjGJ+zKiK qhpnbWykGVAwBCE6EwrQsOFAfL9vqzwz9c1tIycKEmbIUs+aFPMbSQVOynnolXGj m1wpbs58+HOHVMcA/ozzkOT1Of+rlLZQHfc87QQuCaJp419sXEPTzFVdNg7d/ok2 m/Z3KmARycCeEu9TpNLYCcR1/OeayBlHnbjhGaESsgiqenqap27CKRiy2TextYt6 56kiGxIj3EbZz7tE3oJE0eQbAeefFctU42O1nbiPpmUZMHdobJqzNtBxB45pKmYm 3YRQqgd+WzMdY583ifqK7dlooUeLkc61eX+AppJrAZ2UNaHAoP4= =q2Yz -----END PGP SIGNATURE-----

3 2