Pacman-dev
Threads by month
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
June 2020
- 10 participants
- 36 discussions
When a .SRCINFO file is generated via `makepkg --printsrcinfo`, each
section is concluded with an empty line. This means that at the end of
the file, an empty line remains. This is considered a trailing
whitespace error. In fact, `git diff --check` will warn about this,
saying "new blank line at EOF."
Instead of closing each section off with an empty line, use the empty
line to separate sections, omitting the empty line at the end of the
file.
Signed-off-by: Denton Liu <liu.denton(a)gmail.com>
---
scripts/libmakepkg/srcinfo.sh.in | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/scripts/libmakepkg/srcinfo.sh.in b/scripts/libmakepkg/srcinfo.sh.in
index 6e783279..d1e39f7d 100644
--- a/scripts/libmakepkg/srcinfo.sh.in
+++ b/scripts/libmakepkg/srcinfo.sh.in
@@ -30,7 +30,7 @@ srcinfo_open_section() {
printf '%s = %s\n' "$1" "$2"
}
-srcinfo_close_section() {
+srcinfo_separate_section() {
echo
}
@@ -94,7 +94,6 @@ srcinfo_write_global() {
srcinfo_open_section 'pkgbase' "${pkgbase:-$pkgname}"
srcinfo_write_section_details ''
- srcinfo_close_section
}
srcinfo_write_package() {
@@ -104,7 +103,6 @@ srcinfo_write_package() {
srcinfo_open_section 'pkgname' "$1"
srcinfo_write_section_details "$1"
- srcinfo_close_section
}
write_srcinfo_header() {
@@ -118,6 +116,7 @@ write_srcinfo_content() {
srcinfo_write_global
for pkg in "${pkgname[@]}"; do
+ srcinfo_separate_section
srcinfo_write_package "$pkg"
done
}
--
2.27.0.326.gff61950410
2
1
25 Jun '20
Hi,
this is a bit of an OCD thing, but `makepkg --printsrcinfo` usually
generates a file with two newlines at the end.
This logic happens in pacman/scripts/libmakepkg/srcinfo.sh.in
(/usr/share/makepkg/srcinfo.sh)
The relevant functions:
srcinfo_close_section() {
echo
}
srcinfo_write_global() {
...
srcinfo_open_section 'pkgbase' "${pkgbase:-$pkgname}"
srcinfo_write_section_details ''
srcinfo_close_section
}
srcinfo_write_package() {
...
srcinfo_open_section 'pkgname' "$1"
srcinfo_write_section_details "$1"
srcinfo_close_section
}
As you can see, the functions process section details, and then print
out a newline no matter if details printed something or not.
The write_section_details eventually leads to:
srcinfo_write_attr() {
...
printf "\t$attrname = %s\n" "${attrvalues[@]}"
}
Which seems to always print out a newline, so I believe the
srcinfo_close_section function is not needed there and simply
removing it and the two lines that call it would solve this issue, as
per the patchfile in the attachment.
Thanks for taking a look at this,
Marti
4
4
Hello,
I have requested joining the Czech translation team more than a week ago and as far as I can see, there has been no activity since then. The organization name is "toofishes" but I cannot find a way to contact the person in charge via Transifex to ask what is the issue. What are my other options?
Lukas Kucharczyk
2
1
[pacman-dev] [PATCH] Move signature payload creation to download engine
by Anatol Pomozov 23 Jun '20
by Anatol Pomozov 23 Jun '20
23 Jun '20
Until now callee of ALPM download functionality has been in charge of
payload creation both for the main file (e.g. *.pkg) and for the accompanied
*.sig file. One advantage of such solution is that all payloads are
independent and can be fetched in parallel thus exploiting the maximum
level of download parallelism.
To build *.sig file url we've been using a simple string concatenation:
$requested_url + ".sig". Unfortunately there are cases when it does not
work. For example an archlinux.org "Download From Mirror" link looks like
this https://www.archlinux.org/packages/core/x86_64/bash/download/ and
it gets redirected to some mirror. But if we append ".sig" to the end of
the link url and try to download it then archlinux.org returns 404 error.
To overcome this issue we need to follow redirects for the main payload
first, find the final url and only then append '.sig' suffix.
This implies 2 things:
- the signature payload initialization need to be moved to dload.c
as it is the place where we have access to the resolved url
- *.sig is downloaded serially with the main payload and this reduces
level of parallelism
Move *.sig payload creation to dload.c. Once the main payload is fetched
successfully we check if the callee asked to download the accompanied
signature. If yes - create a new payload and add it to mcurl.
*.sig payload does not use server list of the main payload and thus does
not support mirror failover. *.sig file comes from the same server as
the main payload.
Refactor event loop in curl_multi_download_internal() a bit. Instead of
relying on curl_multi_check_finished_download() to return number of new
payloads we simply rerun the loop iteration one more time to check if
there are any active downloads left.
---
lib/libalpm/be_sync.c | 34 +++-------------
lib/libalpm/dload.c | 91 ++++++++++++++++++++++++++-----------------
lib/libalpm/dload.h | 4 +-
3 files changed, 65 insertions(+), 64 deletions(-)
diff --git a/lib/libalpm/be_sync.c b/lib/libalpm/be_sync.c
index 82018e15..41675d21 100644
--- a/lib/libalpm/be_sync.c
+++ b/lib/libalpm/be_sync.c
@@ -180,12 +180,10 @@ int SYMEXPORT alpm_db_update(alpm_handle_t *handle, alpm_list_t *dbs, int force)
dbforce = 1;
}
- CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
+ siglevel = alpm_db_get_siglevel(db);
- /* set hard upper limit of 128 MiB */
- payload->max_size = 128 * 1024 * 1024;
+ CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
payload->servers = db->servers;
-
/* print server + filename into a buffer */
len = strlen(db->treename) + strlen(dbext) + 1;
MALLOC(payload->filepath, len,
@@ -194,31 +192,11 @@ int SYMEXPORT alpm_db_update(alpm_handle_t *handle, alpm_list_t *dbs, int force)
payload->handle = handle;
payload->force = dbforce;
payload->unlink_on_fail = 1;
-
+ payload->download_signature = (siglevel & ALPM_SIG_DATABASE);
+ payload->signature_optional = (siglevel & ALPM_SIG_DATABASE_OPTIONAL);
+ /* set hard upper limit of 128 MiB */
+ payload->max_size = 128 * 1024 * 1024;
payloads = alpm_list_add(payloads, payload);
-
- siglevel = alpm_db_get_siglevel(db);
- if(siglevel & ALPM_SIG_DATABASE) {
- struct dload_payload *sig_payload;
- CALLOC(sig_payload, 1, sizeof(*sig_payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
- sig_payload->signature = 1;
-
- /* print filename into a buffer (leave space for separator and .sig) */
- len = strlen(db->treename) + strlen(dbext) + 5;
- MALLOC(sig_payload->filepath, len,
- FREE(sig_payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
- snprintf(sig_payload->filepath, len, "%s%s.sig", db->treename, dbext);
-
- sig_payload->handle = handle;
- sig_payload->force = dbforce;
- sig_payload->errors_ok = (siglevel & ALPM_SIG_DATABASE_OPTIONAL);
-
- /* set hard upper limit of 16 KiB */
- sig_payload->max_size = 16 * 1024;
- sig_payload->servers = db->servers;
-
- payloads = alpm_list_add(payloads, sig_payload);
- }
}
event.type = ALPM_EVENT_DB_RETRIEVE_START;
diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
index 4dbb011f..b68dcf6d 100644
--- a/lib/libalpm/dload.c
+++ b/lib/libalpm/dload.c
@@ -18,6 +18,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <stdbool.h>
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
@@ -49,6 +50,10 @@
#include "handle.h"
#ifdef HAVE_LIBCURL
+
+static int curl_multi_add_payload(alpm_handle_t *handle, CURLM *curlm,
+ struct dload_payload *payload, const char *localpath);
+
static const char *get_filename(const char *url)
{
char *filename = strrchr(url, '/');
@@ -476,6 +481,25 @@ static int curl_multi_check_finished_download(CURLM *curlm, CURLMsg *msg,
curl_easy_getinfo(curl, CURLINFO_CONDITION_UNMET, &timecond);
curl_easy_getinfo(curl, CURLINFO_EFFECTIVE_URL, &effective_url);
+ /* Let's check if client requested downloading accompanion *.sig file */
+ if(!payload->signature && payload->download_signature && curlerr == CURLE_OK && payload->respcode < 400) {
+ struct dload_payload *sig = NULL;
+
+ int len = strlen(effective_url) + 5;
+ CALLOC(sig, 1, sizeof(*sig), GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
+ MALLOC(sig->fileurl, len, FREE(sig); GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
+ snprintf(sig->fileurl, len, "%s.sig", effective_url);
+ sig->signature = 1;
+ sig->handle = handle;
+ sig->force = payload->force;
+ sig->unlink_on_fail = payload->unlink_on_fail;
+ sig->errors_ok = payload->signature_optional;
+ /* set hard upper limit of 16KiB */
+ sig->max_size = 16 * 1024;
+
+ curl_multi_add_payload(handle, curlm, sig, localpath);
+ }
+
/* time condition was met and we didn't download anything. we need to
* clean up the 0 byte .part file that's left behind. */
if(timecond == 1 && DOUBLE_EQ(bytes_dl, 0)) {
@@ -565,6 +589,13 @@ cleanup:
if(ret == -1 && payload->errors_ok) {
ret = -2;
}
+
+ if(payload->signature) {
+ /* free signature payload memory that was allocated earlier in dload.c */
+ _alpm_dload_payload_reset(payload);
+ FREE(payload);
+ }
+
return ret;
}
@@ -669,12 +700,12 @@ static int curl_multi_download_internal(alpm_handle_t *handle,
int still_running = 0;
int err = 0;
int parallel_downloads = handle->parallel_downloads;
-
CURLM *curlm = handle->curlm;
- CURLMsg *msg;
+ bool msg_done = false;
- while(still_running || payloads) {
- int msgs_left = -1;
+ while(still_running || payloads || msg_done) {
+ int msgs_left = 0;
+ CURLMcode mc;
for(; still_running < parallel_downloads && payloads; still_running++) {
struct dload_payload *payload = payloads->data;
@@ -687,15 +718,19 @@ static int curl_multi_download_internal(alpm_handle_t *handle,
payloads = payloads->next;
} else {
- // the payload failed to start, do not start any new downloads just wait until
- // active one complete.
+ /* The payload failed to start. Do not start any new downloads.
+ * Wait until all active downloads complete.
+ */
_alpm_log(handle, ALPM_LOG_ERROR, _("failed to setup a download payload for %s\n"), payload->remote_name);
payloads = NULL;
err = -1;
}
}
- CURLMcode mc = curl_multi_perform(curlm, &still_running);
+ mc = curl_multi_perform(curlm, &still_running);
+ if(mc == CURLM_OK) {
+ mc = curl_multi_wait(curlm, NULL, 0, 1000, NULL);
+ }
if(mc != CURLM_OK) {
_alpm_log(handle, ALPM_LOG_ERROR, _("curl returned error %d from transfer\n"), mc);
@@ -703,28 +738,28 @@ static int curl_multi_download_internal(alpm_handle_t *handle,
err = -1;
}
- while((msg = curl_multi_info_read(curlm, &msgs_left))) {
+ msg_done = false;
+ while(true) {
+ CURLMsg *msg = curl_multi_info_read(curlm, &msgs_left);
+ if(!msg) {
+ break;
+ }
if(msg->msg == CURLMSG_DONE) {
int ret = curl_multi_check_finished_download(curlm, msg, localpath);
if(ret == -1) {
- /* if current payload failed to download then stop adding new payloads but wait for the
- * current ones
- */
+ /* if current payload failed to download then stop adding new payloads */
payloads = NULL;
err = -1;
- } else if(ret == 2) {
- /* in case of a retry increase the counter of active requests
- * to avoid exiting the loop early
- */
- still_running++;
}
+ /* curl_multi_check_finished_download() might add more payloads e.g. in case of a retry
+ * from the next mirror. We need to execute curl_multi_perform() at least one more time
+ * to make sure new payload requests are processed.
+ */
+ msg_done = true;
} else {
_alpm_log(handle, ALPM_LOG_ERROR, _("curl transfer error: %d\n"), msg->msg);
}
}
- if(still_running) {
- curl_multi_wait(curlm, NULL, 0, 1000, NULL);
- }
}
_alpm_log(handle, ALPM_LOG_DEBUG, "curl_multi_download_internal return code is %d\n", err);
@@ -803,12 +838,10 @@ int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls,
{
const char *cachedir;
alpm_list_t *payloads = NULL;
- int download_sigs;
const alpm_list_t *i;
alpm_event_t event;
CHECK_HANDLE(handle, return -1);
- download_sigs = handle->siglevel & ALPM_SIG_PACKAGE;
/* find a valid cache dir to download to */
cachedir = _alpm_filecache_setup(handle);
@@ -830,21 +863,9 @@ int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls,
payload->allow_resume = 1;
payload->handle = handle;
payload->trust_remote_name = 1;
+ payload->download_signature = (handle->siglevel & ALPM_SIG_PACKAGE);
+ payload->signature_optional = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL);
payloads = alpm_list_add(payloads, payload);
-
- if(download_sigs) {
- int len = strlen(url) + 5;
- CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
- payload->signature = 1;
- MALLOC(payload->fileurl, len, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
- snprintf(payload->fileurl, len, "%s.sig", url);
- payload->handle = handle;
- payload->trust_remote_name = 1;
- payload->errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL);
- /* set hard upper limit of 16KiB */
- payload->max_size = 16 * 1024;
- payloads = alpm_list_add(payloads, payload);
- }
}
}
diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h
index d13bc1b5..facafca2 100644
--- a/lib/libalpm/dload.h
+++ b/lib/libalpm/dload.h
@@ -47,11 +47,13 @@ struct dload_payload {
int errors_ok;
int unlink_on_fail;
int trust_remote_name;
- int signature; /* specifies if the payload is a signature file */
+ int download_signature; /* specifies if an accompanion *.sig file need to be downloaded*/
+ int signature_optional; /* *.sig file is optional */
#ifdef HAVE_LIBCURL
CURL *curl;
char error_buffer[CURL_ERROR_SIZE];
FILE *localf; /* temp download file */
+ int signature; /* specifies if this payload is for a signature file */
#endif
};
--
2.26.2
2
3
23 Jun '20
All users of _alpm_download() have been refactored to the new API.
It is time to remove the old _alpm_download() functionality now.
This change also removes obsolete SIGPIPE signal handler functionality
(this is a leftover from libfetch days).
Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com>
---
lib/libalpm/dload.c | 323 +-------------------------------------------
lib/libalpm/dload.h | 4 -
2 files changed, 3 insertions(+), 324 deletions(-)
diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
index 43fe9847..4dbb011f 100644
--- a/lib/libalpm/dload.c
+++ b/lib/libalpm/dload.c
@@ -78,10 +78,6 @@ enum {
};
static int dload_interrupted;
-static void inthandler(int UNUSED signum)
-{
- dload_interrupted = ABORT_SIGINT;
-}
static int dload_progress_cb(void *file, curl_off_t dltotal, curl_off_t dlnow,
curl_off_t UNUSED ultotal, curl_off_t UNUSED ulnow)
@@ -236,8 +232,7 @@ static size_t dload_parseheader_cb(void *ptr, size_t size, size_t nmemb, void *u
return realsize;
}
-static void curl_set_handle_opts(struct dload_payload *payload,
- CURL *curl, char *error_buffer)
+static void curl_set_handle_opts(CURL *curl, struct dload_payload *payload)
{
alpm_handle_t *handle = payload->handle;
const char *useragent = getenv("HTTP_USER_AGENT");
@@ -247,7 +242,7 @@ static void curl_set_handle_opts(struct dload_payload *payload,
* to reset the handle's parameters for each time it's used. */
curl_easy_reset(curl);
curl_easy_setopt(curl, CURLOPT_URL, payload->fileurl);
- curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, error_buffer);
+ curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, payload->error_buffer);
curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 10L);
curl_easy_setopt(curl, CURLOPT_MAXREDIRS, 10L);
curl_easy_setopt(curl, CURLOPT_FILETIME, 1L);
@@ -301,24 +296,6 @@ static void curl_set_handle_opts(struct dload_payload *payload,
}
}
-static void mask_signal(int signum, void (*handler)(int),
- struct sigaction *origaction)
-{
- struct sigaction newaction;
-
- newaction.sa_handler = handler;
- sigemptyset(&newaction.sa_mask);
- newaction.sa_flags = 0;
-
- sigaction(signum, NULL, origaction);
- sigaction(signum, &newaction, NULL);
-}
-
-static void unmask_signal(int signum, struct sigaction *sa)
-{
- sigaction(signum, sa, NULL);
-}
-
static FILE *create_tempfile(struct dload_payload *payload, const char *localpath)
{
int fd;
@@ -353,259 +330,6 @@ static FILE *create_tempfile(struct dload_payload *payload, const char *localpat
/* RFC1123 states applications should support this length */
#define HOSTNAME_SIZE 256
-static int curl_download_internal(struct dload_payload *payload,
- const char *localpath, char **final_file, const char **final_url)
-{
- int ret = -1;
- FILE *localf = NULL;
- char *effective_url;
- char hostname[HOSTNAME_SIZE];
- char error_buffer[CURL_ERROR_SIZE] = {0};
- struct stat st;
- long timecond, remote_time = -1;
- double remote_size, bytes_dl;
- struct sigaction orig_sig_pipe, orig_sig_int;
- CURLcode curlerr;
- alpm_download_event_init_t init_cb_data = {0};
- alpm_download_event_completed_t completed_cb_data = {0};
- /* shortcut to our handle within the payload */
- alpm_handle_t *handle = payload->handle;
- CURL *curl = curl_easy_init();
- handle->pm_errno = ALPM_ERR_OK;
- payload->curl = curl;
-
- /* make sure these are NULL */
- FREE(payload->tempfile_name);
- FREE(payload->destfile_name);
- FREE(payload->content_disp_name);
-
- payload->tempfile_openmode = "wb";
- if(!payload->remote_name) {
- STRDUP(payload->remote_name, get_filename(payload->fileurl),
- RET_ERR(handle, ALPM_ERR_MEMORY, -1));
- }
- if(curl_gethost(payload->fileurl, hostname, sizeof(hostname)) != 0) {
- _alpm_log(handle, ALPM_LOG_ERROR, _("url '%s' is invalid\n"), payload->fileurl);
- RET_ERR(handle, ALPM_ERR_SERVER_BAD_URL, -1);
- }
-
- if(payload->remote_name && strlen(payload->remote_name) > 0 &&
- strcmp(payload->remote_name, ".sig") != 0) {
- payload->destfile_name = get_fullpath(localpath, payload->remote_name, "");
- payload->tempfile_name = get_fullpath(localpath, payload->remote_name, ".part");
- if(!payload->destfile_name || !payload->tempfile_name) {
- goto cleanup;
- }
- } else {
- /* URL doesn't contain a filename, so make a tempfile. We can't support
- * resuming this kind of download; partial transfers will be destroyed */
- payload->unlink_on_fail = 1;
-
- localf = create_tempfile(payload, localpath);
- if(localf == NULL) {
- goto cleanup;
- }
- }
-
- curl_set_handle_opts(payload, curl, error_buffer);
-
- if(payload->max_size == payload->initial_size && payload->max_size != 0) {
- /* .part file is complete */
- ret = 0;
- goto cleanup;
- }
-
- if(localf == NULL) {
- localf = fopen(payload->tempfile_name, payload->tempfile_openmode);
- if(localf == NULL) {
- _alpm_log(handle, ALPM_LOG_ERROR,
- _("could not open file %s: %s\n"),
- payload->tempfile_name, strerror(errno));
- GOTO_ERR(handle, ALPM_ERR_RETRIEVE, cleanup);
- }
- }
-
- _alpm_log(handle, ALPM_LOG_DEBUG,
- "opened tempfile for download: %s (%s)\n", payload->tempfile_name,
- payload->tempfile_openmode);
-
- curl_easy_setopt(curl, CURLOPT_WRITEDATA, localf);
-
- /* Ignore any SIGPIPE signals. With libcurl, these shouldn't be happening,
- * but better safe than sorry. Store the old signal handler first. */
- mask_signal(SIGPIPE, SIG_IGN, &orig_sig_pipe);
- dload_interrupted = 0;
- mask_signal(SIGINT, &inthandler, &orig_sig_int);
-
- handle->dlcb(payload->remote_name, ALPM_DOWNLOAD_INIT, &init_cb_data);
-
- /* perform transfer */
- curlerr = curl_easy_perform(curl);
- _alpm_log(handle, ALPM_LOG_DEBUG, "curl returned error %d from transfer\n",
- curlerr);
-
- /* disconnect relationships from the curl handle for things that might go out
- * of scope, but could still be touched on connection teardown. This really
- * only applies to FTP transfers. */
- curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 1L);
- curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, (char *)NULL);
-
- /* was it a success? */
- switch(curlerr) {
- case CURLE_OK:
- /* get http/ftp response code */
- _alpm_log(handle, ALPM_LOG_DEBUG, "response code: %ld\n", payload->respcode);
- if(payload->respcode >= 400) {
- payload->unlink_on_fail = 1;
- if(!payload->errors_ok) {
- handle->pm_errno = ALPM_ERR_RETRIEVE;
- /* non-translated message is same as libcurl */
- snprintf(error_buffer, sizeof(error_buffer),
- "The requested URL returned error: %ld", payload->respcode);
- _alpm_log(handle, ALPM_LOG_ERROR,
- _("failed retrieving file '%s' from %s : %s\n"),
- payload->remote_name, hostname, error_buffer);
- }
- goto cleanup;
- }
- break;
- case CURLE_ABORTED_BY_CALLBACK:
- /* handle the interrupt accordingly */
- if(dload_interrupted == ABORT_OVER_MAXFILESIZE) {
- curlerr = CURLE_FILESIZE_EXCEEDED;
- payload->unlink_on_fail = 1;
- handle->pm_errno = ALPM_ERR_LIBCURL;
- _alpm_log(handle, ALPM_LOG_ERROR,
- _("failed retrieving file '%s' from %s : expected download size exceeded\n"),
- payload->remote_name, hostname);
- }
- goto cleanup;
- case CURLE_COULDNT_RESOLVE_HOST:
- payload->unlink_on_fail = 1;
- handle->pm_errno = ALPM_ERR_SERVER_BAD_URL;
- _alpm_log(handle, ALPM_LOG_ERROR,
- _("failed retrieving file '%s' from %s : %s\n"),
- payload->remote_name, hostname, error_buffer);
- goto cleanup;
- default:
- /* delete zero length downloads */
- if(fstat(fileno(localf), &st) == 0 && st.st_size == 0) {
- payload->unlink_on_fail = 1;
- }
- if(!payload->errors_ok) {
- handle->pm_errno = ALPM_ERR_LIBCURL;
- _alpm_log(handle, ALPM_LOG_ERROR,
- _("failed retrieving file '%s' from %s : %s\n"),
- payload->remote_name, hostname, error_buffer);
- } else {
- _alpm_log(handle, ALPM_LOG_DEBUG,
- "failed retrieving file '%s' from %s : %s\n",
- payload->remote_name, hostname, error_buffer);
- }
- goto cleanup;
- }
-
- /* retrieve info about the state of the transfer */
- curl_easy_getinfo(curl, CURLINFO_FILETIME, &remote_time);
- curl_easy_getinfo(curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &remote_size);
- curl_easy_getinfo(curl, CURLINFO_SIZE_DOWNLOAD, &bytes_dl);
- curl_easy_getinfo(curl, CURLINFO_CONDITION_UNMET, &timecond);
- curl_easy_getinfo(curl, CURLINFO_EFFECTIVE_URL, &effective_url);
-
- if(final_url != NULL) {
- *final_url = effective_url;
- }
-
- /* time condition was met and we didn't download anything. we need to
- * clean up the 0 byte .part file that's left behind. */
- if(timecond == 1 && DOUBLE_EQ(bytes_dl, 0)) {
- _alpm_log(handle, ALPM_LOG_DEBUG, "file met time condition\n");
- ret = 1;
- unlink(payload->tempfile_name);
- goto cleanup;
- }
-
- /* remote_size isn't necessarily the full size of the file, just what the
- * server reported as remaining to download. compare it to what curl reported
- * as actually being transferred during curl_easy_perform() */
- if(!DOUBLE_EQ(remote_size, -1) && !DOUBLE_EQ(bytes_dl, -1) &&
- !DOUBLE_EQ(bytes_dl, remote_size)) {
- _alpm_log(handle, ALPM_LOG_ERROR, _("%s appears to be truncated: %jd/%jd bytes\n"),
- payload->remote_name, (intmax_t)bytes_dl, (intmax_t)remote_size);
- GOTO_ERR(handle, ALPM_ERR_RETRIEVE, cleanup);
- }
-
- if(payload->trust_remote_name) {
- if(payload->content_disp_name) {
- /* content-disposition header has a better name for our file */
- free(payload->destfile_name);
- payload->destfile_name = get_fullpath(localpath,
- get_filename(payload->content_disp_name), "");
- } else {
- const char *effective_filename = strrchr(effective_url, '/');
- if(effective_filename && strlen(effective_filename) > 2) {
- effective_filename++;
-
- /* if destfile was never set, we wrote to a tempfile. even if destfile is
- * set, we may have followed some redirects and the effective url may
- * have a better suggestion as to what to name our file. in either case,
- * refactor destfile to this newly derived name. */
- if(!payload->destfile_name || strcmp(effective_filename,
- strrchr(payload->destfile_name, '/') + 1) != 0) {
- free(payload->destfile_name);
- payload->destfile_name = get_fullpath(localpath, effective_filename, "");
- }
- }
- }
- }
-
- ret = 0;
-
-cleanup:
- if(localf != NULL) {
- fclose(localf);
- utimes_long(payload->tempfile_name, remote_time);
- }
-
- if(ret == 0) {
- const char *realname = payload->tempfile_name;
- if(payload->destfile_name) {
- realname = payload->destfile_name;
- if(rename(payload->tempfile_name, payload->destfile_name)) {
- _alpm_log(handle, ALPM_LOG_ERROR, _("could not rename %s to %s (%s)\n"),
- payload->tempfile_name, payload->destfile_name, strerror(errno));
- ret = -1;
- }
- }
- if(ret != -1 && final_file) {
- STRDUP(*final_file, strrchr(realname, '/') + 1,
- RET_ERR(handle, ALPM_ERR_MEMORY, -1));
- }
- }
-
- if((ret == -1 || dload_interrupted) && payload->unlink_on_fail &&
- payload->tempfile_name) {
- unlink(payload->tempfile_name);
- }
-
- curl_easy_cleanup(curl);
- payload->curl = NULL;
-
- /* restore the old signal handlers */
- unmask_signal(SIGINT, &orig_sig_int);
- unmask_signal(SIGPIPE, &orig_sig_pipe);
- /* if we were interrupted, trip the old handler */
- if(dload_interrupted == ABORT_SIGINT) {
- raise(SIGINT);
- }
-
- completed_cb_data.total = bytes_dl;
- completed_cb_data.result = ret;
- handle->dlcb(payload->remote_name, ALPM_DOWNLOAD_COMPLETED, &completed_cb_data);
-
- return ret;
-}
-
/* Return 0 if retry was successful, -1 otherwise */
static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_payload *payload)
{
@@ -901,7 +625,7 @@ static int curl_multi_add_payload(alpm_handle_t *handle, CURLM *curlm,
}
}
- curl_set_handle_opts(payload, curl, payload->error_buffer);
+ curl_set_handle_opts(curl, payload);
if(payload->max_size == payload->initial_size && payload->max_size != 0) {
/* .part file is complete */
@@ -1009,36 +733,6 @@ static int curl_multi_download_internal(alpm_handle_t *handle,
#endif
-/** Download a file given by a URL to a local directory.
- * Does not overwrite an existing file if the download fails.
- * @param payload the payload context
- * @param localpath the directory to save the file in
- * @param final_file the real name of the downloaded file (may be NULL)
- * @return 0 on success, -1 on error (pm_errno is set accordingly if errors_ok == 0)
- */
-int _alpm_download(struct dload_payload *payload, const char *localpath,
- char **final_file, const char **final_url)
-{
- alpm_handle_t *handle = payload->handle;
-
- if(handle->fetchcb == NULL) {
-#ifdef HAVE_LIBCURL
- return curl_download_internal(payload, localpath, final_file, final_url);
-#else
- /* work around unused warnings when building without libcurl */
- (void)final_file;
- (void)final_url;
- RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1);
-#endif
- } else {
- int ret = handle->fetchcb(payload->fileurl, localpath, payload->force);
- if(ret == -1 && !payload->errors_ok) {
- RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1);
- }
- return ret;
- }
-}
-
int _alpm_multi_download(alpm_handle_t *handle,
alpm_list_t *payloads /* struct dload_payload */,
const char *localpath)
@@ -1213,14 +907,3 @@ void _alpm_dload_payload_reset(struct dload_payload *payload)
FREE(payload->filepath);
*payload = (struct dload_payload){0};
}
-
-void _alpm_dload_payload_reset_for_retry(struct dload_payload *payload)
-{
- ASSERT(payload, return);
-
- FREE(payload->fileurl);
- FREE(payload->filepath);
- payload->initial_size += payload->prevprogress;
- payload->prevprogress = 0;
- payload->unlink_on_fail = 0;
-}
diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h
index f119fc2e..d13bc1b5 100644
--- a/lib/libalpm/dload.h
+++ b/lib/libalpm/dload.h
@@ -56,10 +56,6 @@ struct dload_payload {
};
void _alpm_dload_payload_reset(struct dload_payload *payload);
-void _alpm_dload_payload_reset_for_retry(struct dload_payload *payload);
-
-int _alpm_download(struct dload_payload *payload, const char *localpath,
- char **final_file, const char **final_url);
int _alpm_multi_download(alpm_handle_t *handle,
alpm_list_t *payloads /* struct dload_payload */,
--
2.26.2
2
4
[pacman-dev] [PATCH v2] Convert '-U pkg1 pkg2' codepath to parallel download
by Anatol Pomozov 23 Jun '20
by Anatol Pomozov 23 Jun '20
23 Jun '20
Installing remote packages using its URL is an interesting case for ALPM
API. Unlike package sync ('pacman -S pkg1 pkg2') '-U' does not deal with
server mirror list. Thus _alpm_multi_download() should be able to
handle file download for payloads that either have 'fileurl' field
or pair of fields ('servers' and 'filepath') set.
Signature for alpm_fetch_pkgurl() has changed and it accepts an
output list that is populated with filepaths to fetched packages.
Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com>
---
lib/libalpm/alpm.h | 15 ++--
lib/libalpm/dload.c | 205 +++++++++++++++++++++++++------------------
lib/libalpm/dload.h | 5 ++
src/pacman/upgrade.c | 102 ++++++++++-----------
4 files changed, 184 insertions(+), 143 deletions(-)
diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h
index 3ea66ccc..977c292c 100644
--- a/lib/libalpm/alpm.h
+++ b/lib/libalpm/alpm.h
@@ -755,12 +755,17 @@ typedef void (*alpm_cb_totaldl)(off_t total);
typedef int (*alpm_cb_fetch)(const char *url, const char *localpath,
int force);
-/** Fetch a remote pkg.
+/** Fetch a list of remote packages.
* @param handle the context handle
- * @param url URL of the package to download
- * @return the downloaded filepath on success, NULL on error
- */
-char *alpm_fetch_pkgurl(alpm_handle_t *handle, const char *url);
+ * @param urls list of package URLs to download
+ * @param fetched list of filepaths to the fetched packages, each item
+ * corresponds to one in `urls` list. This is an output parameter,
+ * the caller should provide a pointer to an empty list
+ * (*fetched === NULL) and the callee fills the list with data.
+ * @return 0 on success or -1 on failure
+ */
+int alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls,
+ alpm_list_t **fetched);
/** @addtogroup alpm_api_options Options
* Libalpm option getters and setters
diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
index 13aa4086..c1ec1e70 100644
--- a/lib/libalpm/dload.c
+++ b/lib/libalpm/dload.c
@@ -613,7 +613,9 @@ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_p
size_t len;
alpm_handle_t *handle = payload->handle;
- payload->servers = payload->servers->next;
+ if(payload->servers) {
+ payload->servers = payload->servers->next;
+ }
if(!payload->servers) {
_alpm_log(payload->handle, ALPM_LOG_DEBUG,
"%s: no more servers to retry\n", payload->remote_name);
@@ -622,7 +624,7 @@ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_p
server = payload->servers->data;
/* regenerate a new fileurl */
- free(payload->fileurl);
+ FREE(payload->fileurl);
len = strlen(server) + strlen(payload->filepath) + 2;
MALLOC(payload->fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1));
snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath);
@@ -849,20 +851,28 @@ static int curl_multi_add_payload(alpm_handle_t *handle, CURLM *curlm,
struct dload_payload *payload, const char *localpath)
{
size_t len;
- const char *server;
CURL *curl = NULL;
char hostname[HOSTNAME_SIZE];
int ret = -1;
- ASSERT(payload->servers, RET_ERR(handle, ALPM_ERR_SERVER_NONE, -1));
- server = payload->servers->data;
-
curl = curl_easy_init();
payload->curl = curl;
- len = strlen(server) + strlen(payload->filepath) + 2;
- MALLOC(payload->fileurl, len, GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
- snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath);
+ if(payload->fileurl) {
+ ASSERT(!payload->servers, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup));
+ ASSERT(!payload->filepath, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup));
+ } else {
+ const char *server;
+
+ ASSERT(payload->servers, GOTO_ERR(handle, ALPM_ERR_SERVER_NONE, cleanup));
+ ASSERT(payload->filepath, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup));
+
+ server = payload->servers->data;
+
+ len = strlen(server) + strlen(payload->filepath) + 2;
+ MALLOC(payload->fileurl, len, GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
+ snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath);
+ }
payload->tempfile_openmode = "wb";
if(!payload->remote_name) {
@@ -1046,22 +1056,29 @@ int _alpm_multi_download(alpm_handle_t *handle,
alpm_list_t *s;
int success = 0;
- for(s = payload->servers; s; s = s->next) {
- const char *server = s->data;
- char *fileurl;
- int ret;
-
- size_t len = strlen(server) + strlen(payload->filepath) + 2;
- MALLOC(fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1));
- snprintf(fileurl, len, "%s/%s", server, payload->filepath);
-
- ret = handle->fetchcb(fileurl, localpath, payload->force);
- free(fileurl);
-
- if (ret != -1) {
+ if(payload->fileurl) {
+ if (handle->fetchcb(payload->fileurl, localpath, payload->force) != -1) {
success = 1;
break;
}
+ } else {
+ for(s = payload->servers; s; s = s->next) {
+ const char *server = s->data;
+ char *fileurl;
+ int ret;
+
+ size_t len = strlen(server) + strlen(payload->filepath) + 2;
+ MALLOC(fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1));
+ snprintf(fileurl, len, "%s/%s", server, payload->filepath);
+
+ ret = handle->fetchcb(fileurl, localpath, payload->force);
+ free(fileurl);
+
+ if (ret != -1) {
+ success = 1;
+ break;
+ }
+ }
}
if(!success && !payload->errors_ok) {
RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1);
@@ -1087,82 +1104,102 @@ static char *filecache_find_url(alpm_handle_t *handle, const char *url)
return _alpm_filecache_find(handle, filebase);
}
-char SYMEXPORT *alpm_fetch_pkgurl(alpm_handle_t *handle, const char *url)
+int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls,
+ alpm_list_t **fetched)
{
- char *filepath;
- const char *cachedir, *final_pkg_url = NULL;
- char *final_file = NULL;
- struct dload_payload payload = {0};
- int ret = 0;
+ const char *cachedir;
+ alpm_list_t *payloads = NULL;
+ int download_sigs;
+ const alpm_list_t *i;
+ alpm_event_t event;
- CHECK_HANDLE(handle, return NULL);
- ASSERT(url, RET_ERR(handle, ALPM_ERR_WRONG_ARGS, NULL));
+ CHECK_HANDLE(handle, return -1);
+ ASSERT(*fetched == NULL, RET_ERR(handle, ALPM_ERR_WRONG_ARGS, -1));
+ download_sigs = handle->siglevel & ALPM_SIG_PACKAGE;
/* find a valid cache dir to download to */
cachedir = _alpm_filecache_setup(handle);
- /* attempt to find the file in our pkgcache */
- filepath = filecache_find_url(handle, url);
- if(filepath == NULL) {
- STRDUP(payload.fileurl, url, RET_ERR(handle, ALPM_ERR_MEMORY, NULL));
- payload.allow_resume = 1;
- payload.handle = handle;
- payload.trust_remote_name = 1;
-
- /* download the file */
- ret = _alpm_download(&payload, cachedir, &final_file, &final_pkg_url);
- _alpm_dload_payload_reset(&payload);
- if(ret == -1) {
- _alpm_log(handle, ALPM_LOG_WARNING, _("failed to download %s\n"), url);
- free(final_file);
- return NULL;
+ for(i = urls; i; i = i->next) {
+ char *url = i->data;
+
+ /* attempt to find the file in our pkgcache */
+ char *filepath = filecache_find_url(handle, url);
+ if(filepath) {
+ /* the file is locally cached so add it to the output right away */
+ alpm_list_append(fetched, filepath);
+ } else {
+ struct dload_payload *payload = NULL;
+
+ ASSERT(url, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, err));
+ CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
+ STRDUP(payload->fileurl, url, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
+ payload->allow_resume = 1;
+ payload->handle = handle;
+ payload->trust_remote_name = 1;
+ payloads = alpm_list_add(payloads, payload);
+
+ if(download_sigs) {
+ int len = strlen(url) + 5;
+ CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
+ payload->signature = 1;
+ MALLOC(payload->fileurl, len, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
+ snprintf(payload->fileurl, len, "%s.sig", url);
+ payload->handle = handle;
+ payload->trust_remote_name = 1;
+ payload->errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL);
+ /* set hard upper limit of 16KiB */
+ payload->max_size = 16 * 1024;
+ payloads = alpm_list_add(payloads, payload);
+ }
}
- _alpm_log(handle, ALPM_LOG_DEBUG, "successfully downloaded %s\n", url);
}
- /* attempt to download the signature */
- if(ret == 0 && final_pkg_url && (handle->siglevel & ALPM_SIG_PACKAGE)) {
- char *sig_filepath, *sig_final_file = NULL;
- size_t len;
-
- len = strlen(final_pkg_url) + 5;
- MALLOC(payload.fileurl, len, free(final_file); RET_ERR(handle, ALPM_ERR_MEMORY, NULL));
- snprintf(payload.fileurl, len, "%s.sig", final_pkg_url);
-
- sig_filepath = filecache_find_url(handle, payload.fileurl);
- if(sig_filepath == NULL) {
- payload.signature = 1;
- payload.handle = handle;
- payload.trust_remote_name = 1;
- payload.force = 1;
- payload.errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL);
-
- /* set hard upper limit of 16KiB */
- payload.max_size = 16 * 1024;
-
- ret = _alpm_download(&payload, cachedir, &sig_final_file, NULL);
- if(ret == -1 && !payload.errors_ok) {
- _alpm_log(handle, ALPM_LOG_WARNING,
- _("failed to download %s\n"), payload.fileurl);
- /* Warn now, but don't return NULL. We will fail later during package
- * load time. */
- } else if(ret == 0) {
- _alpm_log(handle, ALPM_LOG_DEBUG,
- "successfully downloaded %s\n", payload.fileurl);
+ if(payloads) {
+ event.type = ALPM_EVENT_PKG_RETRIEVE_START;
+ EVENT(handle, &event);
+ if(_alpm_multi_download(handle, payloads, cachedir) == -1) {
+ _alpm_log(handle, ALPM_LOG_WARNING, _("failed to retrieve some files\n"));
+ event.type = ALPM_EVENT_PKG_RETRIEVE_FAILED;
+ EVENT(handle, &event);
+
+ GOTO_ERR(handle, ALPM_ERR_RETRIEVE, err);
+ } else {
+ event.type = ALPM_EVENT_PKG_RETRIEVE_DONE;
+ EVENT(handle, &event);
+ }
+
+ for(i = payloads; i; i = i->next) {
+ struct dload_payload *payload = i->data;
+ const char *filename;
+ char *filepath;
+
+ if(payload->signature) {
+ continue;
+ }
+
+ filename = mbasename(payload->destfile_name);
+ filepath = _alpm_filecache_find(handle, filename);
+ if(filepath) {
+ alpm_list_append(fetched, filepath);
+ } else {
+ _alpm_log(handle, ALPM_LOG_WARNING, _("download completed successfully but no file in the cache\n"));
+ GOTO_ERR(handle, ALPM_ERR_RETRIEVE, err);
}
- FREE(sig_final_file);
}
- free(sig_filepath);
- _alpm_dload_payload_reset(&payload);
- }
- /* we should be able to find the file the second time around */
- if(filepath == NULL) {
- filepath = _alpm_filecache_find(handle, final_file);
+ alpm_list_free_inner(payloads, (alpm_list_fn_free)_alpm_dload_payload_reset);
+ FREELIST(payloads);
}
- free(final_file);
- return filepath;
+ return 0;
+
+err:
+ alpm_list_free_inner(payloads, (alpm_list_fn_free)_alpm_dload_payload_reset);
+ FREELIST(payloads);
+ FREELIST(*fetched);
+
+ return -1;
}
void _alpm_dload_payload_reset(struct dload_payload *payload)
diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h
index 1e4af755..b9ef9c99 100644
--- a/lib/libalpm/dload.h
+++ b/lib/libalpm/dload.h
@@ -30,6 +30,11 @@ struct dload_payload {
char *tempfile_name;
char *destfile_name;
char *content_disp_name;
+ /* client has to provide either
+ * 1) fileurl - full URL to the file
+ * 2) pair of (servers, filepath), in this case ALPM iterates over the
+ * server list and tries to download "$server/$filepath"
+ */
char *fileurl;
char *filepath; /* download URL path */
alpm_list_t *servers;
diff --git a/src/pacman/upgrade.c b/src/pacman/upgrade.c
index 5f984e44..840e3a31 100644
--- a/src/pacman/upgrade.c
+++ b/src/pacman/upgrade.c
@@ -30,6 +30,34 @@
#include "conf.h"
#include "util.h"
+/* add targets to the created transaction */
+static int load_packages(alpm_list_t *targets, int siglevel)
+{
+ alpm_list_t *i;
+ int retval = 0;
+
+ for(i = targets; i; i = alpm_list_next(i)) {
+ const char *targ = i->data;
+ alpm_pkg_t *pkg;
+
+ if(alpm_pkg_load(config->handle, targ, 1, siglevel, &pkg) != 0) {
+ pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
+ targ, alpm_strerror(alpm_errno(config->handle)));
+ retval = 1;
+ continue;
+ }
+ if(alpm_add_pkg(config->handle, pkg) == -1) {
+ pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
+ targ, alpm_strerror(alpm_errno(config->handle)));
+ alpm_pkg_free(pkg);
+ retval = 1;
+ continue;
+ }
+ config->explicit_adds = alpm_list_add(config->explicit_adds, pkg);
+ }
+ return retval;
+}
+
/**
* @brief Upgrade a specified list of packages.
*
@@ -39,43 +67,30 @@
*/
int pacman_upgrade(alpm_list_t *targets)
{
- int retval = 0, *file_is_remote;
+ int retval = 0;
+ alpm_list_t *remote_targets = NULL, *fetched_files = NULL;
+ alpm_list_t *local_targets = NULL;
alpm_list_t *i;
- unsigned int n, num_targets;
if(targets == NULL) {
pm_printf(ALPM_LOG_ERROR, _("no targets specified (use -h for help)\n"));
return 1;
}
- num_targets = alpm_list_count(targets);
-
- /* Check for URL targets and process them */
- file_is_remote = malloc(num_targets * sizeof(int));
- if(file_is_remote == NULL) {
- pm_printf(ALPM_LOG_ERROR, _("memory exhausted\n"));
- return 1;
- }
-
- for(i = targets, n = 0; i; i = alpm_list_next(i), n++) {
+ /* carve out remote targets and move it into a separate list */
+ for(i = targets; i; i = alpm_list_next(i)) {
if(strstr(i->data, "://")) {
- char *str = alpm_fetch_pkgurl(config->handle, i->data);
- if(str == NULL) {
- pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
- (char *)i->data, alpm_strerror(alpm_errno(config->handle)));
- retval = 1;
- } else {
- free(i->data);
- i->data = str;
- file_is_remote[n] = 1;
- }
+ remote_targets = alpm_list_add(remote_targets, i->data);
} else {
- file_is_remote[n] = 0;
+ local_targets = alpm_list_add(local_targets, i->data);
}
}
- if(retval) {
- goto fail_free;
+ if(remote_targets) {
+ retval = alpm_fetch_pkgurl(config->handle, remote_targets, &fetched_files);
+ if(retval) {
+ goto fail_free;
+ }
}
/* Step 1: create a new transaction */
@@ -85,39 +100,16 @@ int pacman_upgrade(alpm_list_t *targets)
}
printf(_("loading packages...\n"));
- /* add targets to the created transaction */
- for(i = targets, n = 0; i; i = alpm_list_next(i), n++) {
- const char *targ = i->data;
- alpm_pkg_t *pkg;
- int siglevel;
-
- if(file_is_remote[n]) {
- siglevel = alpm_option_get_remote_file_siglevel(config->handle);
- } else {
- siglevel = alpm_option_get_local_file_siglevel(config->handle);
- }
-
- if(alpm_pkg_load(config->handle, targ, 1, siglevel, &pkg) != 0) {
- pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
- targ, alpm_strerror(alpm_errno(config->handle)));
- retval = 1;
- continue;
- }
- if(alpm_add_pkg(config->handle, pkg) == -1) {
- pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
- targ, alpm_strerror(alpm_errno(config->handle)));
- alpm_pkg_free(pkg);
- retval = 1;
- continue;
- }
- config->explicit_adds = alpm_list_add(config->explicit_adds, pkg);
- }
+ retval |= load_packages(local_targets, alpm_option_get_local_file_siglevel(config->handle));
+ retval |= load_packages(fetched_files, alpm_option_get_remote_file_siglevel(config->handle));
if(retval) {
goto fail_release;
}
- free(file_is_remote);
+ alpm_list_free(remote_targets);
+ alpm_list_free(local_targets);
+ FREELIST(fetched_files);
/* now that targets are resolved, we can hand it all off to the sync code */
return sync_prepare_execute();
@@ -125,7 +117,9 @@ int pacman_upgrade(alpm_list_t *targets)
fail_release:
trans_release();
fail_free:
- free(file_is_remote);
+ alpm_list_free(remote_targets);
+ alpm_list_free(local_targets);
+ FREELIST(fetched_files);
return retval;
}
--
2.27.0
1
0
[pacman-dev] [PATCH] Convert '-U pkg1 pkg2' codepath to parallel download
by Anatol Pomozov 23 Jun '20
by Anatol Pomozov 23 Jun '20
23 Jun '20
Installing remote packages using its URL is an interesting case for ALPM
API. Unlike package sync ('pacman -S pkg1 pkg2') '-U' does not deal with
server mirror list. Thus _alpm_multi_download() should be able to
handle file download for payloads that either have 'fileurl' field
or pair of fields ('servers' and 'filepath') set.
Signature for alpm_fetch_pkgurl() has changed and it accepts an
output list that is populated with filepaths to fetched packages.
Signed-off-by: Anatol Pomozov <anatol.pomozov(a)gmail.com>
---
lib/libalpm/alpm.h | 11 ++-
lib/libalpm/dload.c | 204 +++++++++++++++++++++++++------------------
lib/libalpm/dload.h | 7 +-
src/pacman/upgrade.c | 102 ++++++++++------------
4 files changed, 181 insertions(+), 143 deletions(-)
diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h
index 3ea66ccc..c6a13273 100644
--- a/lib/libalpm/alpm.h
+++ b/lib/libalpm/alpm.h
@@ -755,12 +755,15 @@ typedef void (*alpm_cb_totaldl)(off_t total);
typedef int (*alpm_cb_fetch)(const char *url, const char *localpath,
int force);
-/** Fetch a remote pkg.
+/** Fetch a list of remote packages.
* @param handle the context handle
- * @param url URL of the package to download
- * @return the downloaded filepath on success, NULL on error
+ * @param urls list of package URLs to download
+ * @param fetched list of filepaths to the fetched packages, each item
+ * corresponds to one in `urls` list
+ * @return 0 on success or -1 on failure
*/
-char *alpm_fetch_pkgurl(alpm_handle_t *handle, const char *url);
+int alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls,
+ alpm_list_t **fetched);
/** @addtogroup alpm_api_options Options
* Libalpm option getters and setters
diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
index 13aa4086..43fe9847 100644
--- a/lib/libalpm/dload.c
+++ b/lib/libalpm/dload.c
@@ -613,7 +613,9 @@ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_p
size_t len;
alpm_handle_t *handle = payload->handle;
- payload->servers = payload->servers->next;
+ if(payload->servers) {
+ payload->servers = payload->servers->next;
+ }
if(!payload->servers) {
_alpm_log(payload->handle, ALPM_LOG_DEBUG,
"%s: no more servers to retry\n", payload->remote_name);
@@ -622,7 +624,7 @@ static int curl_multi_retry_next_server(CURLM *curlm, CURL *curl, struct dload_p
server = payload->servers->data;
/* regenerate a new fileurl */
- free(payload->fileurl);
+ FREE(payload->fileurl);
len = strlen(server) + strlen(payload->filepath) + 2;
MALLOC(payload->fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1));
snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath);
@@ -849,20 +851,28 @@ static int curl_multi_add_payload(alpm_handle_t *handle, CURLM *curlm,
struct dload_payload *payload, const char *localpath)
{
size_t len;
- const char *server;
CURL *curl = NULL;
char hostname[HOSTNAME_SIZE];
int ret = -1;
- ASSERT(payload->servers, RET_ERR(handle, ALPM_ERR_SERVER_NONE, -1));
- server = payload->servers->data;
-
curl = curl_easy_init();
payload->curl = curl;
- len = strlen(server) + strlen(payload->filepath) + 2;
- MALLOC(payload->fileurl, len, GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
- snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath);
+ if(payload->fileurl) {
+ ASSERT(!payload->servers, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup));
+ ASSERT(!payload->filepath, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup));
+ } else {
+ const char *server;
+
+ ASSERT(payload->servers, GOTO_ERR(handle, ALPM_ERR_SERVER_NONE, cleanup));
+ ASSERT(payload->filepath, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, cleanup));
+
+ server = payload->servers->data;
+
+ len = strlen(server) + strlen(payload->filepath) + 2;
+ MALLOC(payload->fileurl, len, GOTO_ERR(handle, ALPM_ERR_MEMORY, cleanup));
+ snprintf(payload->fileurl, len, "%s/%s", server, payload->filepath);
+ }
payload->tempfile_openmode = "wb";
if(!payload->remote_name) {
@@ -1046,22 +1056,29 @@ int _alpm_multi_download(alpm_handle_t *handle,
alpm_list_t *s;
int success = 0;
- for(s = payload->servers; s; s = s->next) {
- const char *server = s->data;
- char *fileurl;
- int ret;
-
- size_t len = strlen(server) + strlen(payload->filepath) + 2;
- MALLOC(fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1));
- snprintf(fileurl, len, "%s/%s", server, payload->filepath);
-
- ret = handle->fetchcb(fileurl, localpath, payload->force);
- free(fileurl);
-
- if (ret != -1) {
+ if(payload->fileurl) {
+ if (handle->fetchcb(payload->fileurl, localpath, payload->force) != -1) {
success = 1;
break;
}
+ } else {
+ for(s = payload->servers; s; s = s->next) {
+ const char *server = s->data;
+ char *fileurl;
+ int ret;
+
+ size_t len = strlen(server) + strlen(payload->filepath) + 2;
+ MALLOC(fileurl, len, RET_ERR(handle, ALPM_ERR_MEMORY, -1));
+ snprintf(fileurl, len, "%s/%s", server, payload->filepath);
+
+ ret = handle->fetchcb(fileurl, localpath, payload->force);
+ free(fileurl);
+
+ if (ret != -1) {
+ success = 1;
+ break;
+ }
+ }
}
if(!success && !payload->errors_ok) {
RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1);
@@ -1087,82 +1104,101 @@ static char *filecache_find_url(alpm_handle_t *handle, const char *url)
return _alpm_filecache_find(handle, filebase);
}
-char SYMEXPORT *alpm_fetch_pkgurl(alpm_handle_t *handle, const char *url)
+int SYMEXPORT alpm_fetch_pkgurl(alpm_handle_t *handle, const alpm_list_t *urls,
+ alpm_list_t **fetched)
{
- char *filepath;
- const char *cachedir, *final_pkg_url = NULL;
- char *final_file = NULL;
- struct dload_payload payload = {0};
- int ret = 0;
+ const char *cachedir;
+ alpm_list_t *payloads = NULL;
+ int download_sigs;
+ const alpm_list_t *i;
+ alpm_event_t event;
- CHECK_HANDLE(handle, return NULL);
- ASSERT(url, RET_ERR(handle, ALPM_ERR_WRONG_ARGS, NULL));
+ CHECK_HANDLE(handle, return -1);
+ download_sigs = handle->siglevel & ALPM_SIG_PACKAGE;
/* find a valid cache dir to download to */
cachedir = _alpm_filecache_setup(handle);
- /* attempt to find the file in our pkgcache */
- filepath = filecache_find_url(handle, url);
- if(filepath == NULL) {
- STRDUP(payload.fileurl, url, RET_ERR(handle, ALPM_ERR_MEMORY, NULL));
- payload.allow_resume = 1;
- payload.handle = handle;
- payload.trust_remote_name = 1;
-
- /* download the file */
- ret = _alpm_download(&payload, cachedir, &final_file, &final_pkg_url);
- _alpm_dload_payload_reset(&payload);
- if(ret == -1) {
- _alpm_log(handle, ALPM_LOG_WARNING, _("failed to download %s\n"), url);
- free(final_file);
- return NULL;
+ for(i = urls; i; i = i->next) {
+ char *url = i->data;
+
+ /* attempt to find the file in our pkgcache */
+ char *filepath = filecache_find_url(handle, url);
+ if(filepath) {
+ /* the file is locally cached so add it to the output right away */
+ alpm_list_append(fetched, filepath);
+ } else {
+ struct dload_payload *payload = NULL;
+
+ ASSERT(url, GOTO_ERR(handle, ALPM_ERR_WRONG_ARGS, err));
+ CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
+ STRDUP(payload->fileurl, url, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
+ payload->allow_resume = 1;
+ payload->handle = handle;
+ payload->trust_remote_name = 1;
+ payloads = alpm_list_add(payloads, payload);
+
+ if(download_sigs) {
+ int len = strlen(url) + 5;
+ CALLOC(payload, 1, sizeof(*payload), GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
+ payload->signature = 1;
+ MALLOC(payload->fileurl, len, FREE(payload); GOTO_ERR(handle, ALPM_ERR_MEMORY, err));
+ snprintf(payload->fileurl, len, "%s.sig", url);
+ payload->handle = handle;
+ payload->trust_remote_name = 1;
+ payload->errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL);
+ /* set hard upper limit of 16KiB */
+ payload->max_size = 16 * 1024;
+ payloads = alpm_list_add(payloads, payload);
+ }
}
- _alpm_log(handle, ALPM_LOG_DEBUG, "successfully downloaded %s\n", url);
}
- /* attempt to download the signature */
- if(ret == 0 && final_pkg_url && (handle->siglevel & ALPM_SIG_PACKAGE)) {
- char *sig_filepath, *sig_final_file = NULL;
- size_t len;
-
- len = strlen(final_pkg_url) + 5;
- MALLOC(payload.fileurl, len, free(final_file); RET_ERR(handle, ALPM_ERR_MEMORY, NULL));
- snprintf(payload.fileurl, len, "%s.sig", final_pkg_url);
-
- sig_filepath = filecache_find_url(handle, payload.fileurl);
- if(sig_filepath == NULL) {
- payload.signature = 1;
- payload.handle = handle;
- payload.trust_remote_name = 1;
- payload.force = 1;
- payload.errors_ok = (handle->siglevel & ALPM_SIG_PACKAGE_OPTIONAL);
-
- /* set hard upper limit of 16KiB */
- payload.max_size = 16 * 1024;
-
- ret = _alpm_download(&payload, cachedir, &sig_final_file, NULL);
- if(ret == -1 && !payload.errors_ok) {
- _alpm_log(handle, ALPM_LOG_WARNING,
- _("failed to download %s\n"), payload.fileurl);
- /* Warn now, but don't return NULL. We will fail later during package
- * load time. */
- } else if(ret == 0) {
- _alpm_log(handle, ALPM_LOG_DEBUG,
- "successfully downloaded %s\n", payload.fileurl);
+ if(payloads) {
+ event.type = ALPM_EVENT_PKG_RETRIEVE_START;
+ EVENT(handle, &event);
+ if(_alpm_multi_download(handle, payloads, cachedir) == -1) {
+ _alpm_log(handle, ALPM_LOG_WARNING, _("failed to retrieve some files\n"));
+ event.type = ALPM_EVENT_PKG_RETRIEVE_FAILED;
+ EVENT(handle, &event);
+
+ GOTO_ERR(handle, ALPM_ERR_RETRIEVE, err);
+ } else {
+ event.type = ALPM_EVENT_PKG_RETRIEVE_DONE;
+ EVENT(handle, &event);
+ }
+
+ for(i = payloads; i; i = i->next) {
+ struct dload_payload *payload = i->data;
+ const char *filename;
+ char *filepath;
+
+ if(payload->signature) {
+ continue;
+ }
+
+ filename = mbasename(payload->destfile_name);
+ filepath = _alpm_filecache_find(handle, filename);
+ if(filepath) {
+ alpm_list_append(fetched, filepath);
+ } else {
+ _alpm_log(handle, ALPM_LOG_WARNING, _("download completed successfully but no file in the cache\n"));
+ GOTO_ERR(handle, ALPM_ERR_RETRIEVE, err);
}
- FREE(sig_final_file);
}
- free(sig_filepath);
- _alpm_dload_payload_reset(&payload);
- }
- /* we should be able to find the file the second time around */
- if(filepath == NULL) {
- filepath = _alpm_filecache_find(handle, final_file);
+ alpm_list_free_inner(payloads, (alpm_list_fn_free)_alpm_dload_payload_reset);
+ FREELIST(payloads);
}
- free(final_file);
- return filepath;
+ return 0;
+
+err:
+ alpm_list_free_inner(payloads, (alpm_list_fn_free)_alpm_dload_payload_reset);
+ FREELIST(payloads);
+ FREELIST(*fetched);
+
+ return -1;
}
void _alpm_dload_payload_reset(struct dload_payload *payload)
diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h
index 1e4af755..f119fc2e 100644
--- a/lib/libalpm/dload.h
+++ b/lib/libalpm/dload.h
@@ -30,9 +30,14 @@ struct dload_payload {
char *tempfile_name;
char *destfile_name;
char *content_disp_name;
+ /* client has to provide either
+ * 1) fileurl - full URL to the file
+ * 2) pair of (servers, filepath), in this case ALPM iterates over the
+ * server list and tries to download "$server/$filepath"
+ */
char *fileurl;
- char *filepath; /* download URL path */
alpm_list_t *servers;
+ char *filepath; /* download URL path */
long respcode;
off_t initial_size;
off_t max_size;
diff --git a/src/pacman/upgrade.c b/src/pacman/upgrade.c
index 5f984e44..840e3a31 100644
--- a/src/pacman/upgrade.c
+++ b/src/pacman/upgrade.c
@@ -30,6 +30,34 @@
#include "conf.h"
#include "util.h"
+/* add targets to the created transaction */
+static int load_packages(alpm_list_t *targets, int siglevel)
+{
+ alpm_list_t *i;
+ int retval = 0;
+
+ for(i = targets; i; i = alpm_list_next(i)) {
+ const char *targ = i->data;
+ alpm_pkg_t *pkg;
+
+ if(alpm_pkg_load(config->handle, targ, 1, siglevel, &pkg) != 0) {
+ pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
+ targ, alpm_strerror(alpm_errno(config->handle)));
+ retval = 1;
+ continue;
+ }
+ if(alpm_add_pkg(config->handle, pkg) == -1) {
+ pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
+ targ, alpm_strerror(alpm_errno(config->handle)));
+ alpm_pkg_free(pkg);
+ retval = 1;
+ continue;
+ }
+ config->explicit_adds = alpm_list_add(config->explicit_adds, pkg);
+ }
+ return retval;
+}
+
/**
* @brief Upgrade a specified list of packages.
*
@@ -39,43 +67,30 @@
*/
int pacman_upgrade(alpm_list_t *targets)
{
- int retval = 0, *file_is_remote;
+ int retval = 0;
+ alpm_list_t *remote_targets = NULL, *fetched_files = NULL;
+ alpm_list_t *local_targets = NULL;
alpm_list_t *i;
- unsigned int n, num_targets;
if(targets == NULL) {
pm_printf(ALPM_LOG_ERROR, _("no targets specified (use -h for help)\n"));
return 1;
}
- num_targets = alpm_list_count(targets);
-
- /* Check for URL targets and process them */
- file_is_remote = malloc(num_targets * sizeof(int));
- if(file_is_remote == NULL) {
- pm_printf(ALPM_LOG_ERROR, _("memory exhausted\n"));
- return 1;
- }
-
- for(i = targets, n = 0; i; i = alpm_list_next(i), n++) {
+ /* carve out remote targets and move it into a separate list */
+ for(i = targets; i; i = alpm_list_next(i)) {
if(strstr(i->data, "://")) {
- char *str = alpm_fetch_pkgurl(config->handle, i->data);
- if(str == NULL) {
- pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
- (char *)i->data, alpm_strerror(alpm_errno(config->handle)));
- retval = 1;
- } else {
- free(i->data);
- i->data = str;
- file_is_remote[n] = 1;
- }
+ remote_targets = alpm_list_add(remote_targets, i->data);
} else {
- file_is_remote[n] = 0;
+ local_targets = alpm_list_add(local_targets, i->data);
}
}
- if(retval) {
- goto fail_free;
+ if(remote_targets) {
+ retval = alpm_fetch_pkgurl(config->handle, remote_targets, &fetched_files);
+ if(retval) {
+ goto fail_free;
+ }
}
/* Step 1: create a new transaction */
@@ -85,39 +100,16 @@ int pacman_upgrade(alpm_list_t *targets)
}
printf(_("loading packages...\n"));
- /* add targets to the created transaction */
- for(i = targets, n = 0; i; i = alpm_list_next(i), n++) {
- const char *targ = i->data;
- alpm_pkg_t *pkg;
- int siglevel;
-
- if(file_is_remote[n]) {
- siglevel = alpm_option_get_remote_file_siglevel(config->handle);
- } else {
- siglevel = alpm_option_get_local_file_siglevel(config->handle);
- }
-
- if(alpm_pkg_load(config->handle, targ, 1, siglevel, &pkg) != 0) {
- pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
- targ, alpm_strerror(alpm_errno(config->handle)));
- retval = 1;
- continue;
- }
- if(alpm_add_pkg(config->handle, pkg) == -1) {
- pm_printf(ALPM_LOG_ERROR, "'%s': %s\n",
- targ, alpm_strerror(alpm_errno(config->handle)));
- alpm_pkg_free(pkg);
- retval = 1;
- continue;
- }
- config->explicit_adds = alpm_list_add(config->explicit_adds, pkg);
- }
+ retval |= load_packages(local_targets, alpm_option_get_local_file_siglevel(config->handle));
+ retval |= load_packages(fetched_files, alpm_option_get_remote_file_siglevel(config->handle));
if(retval) {
goto fail_release;
}
- free(file_is_remote);
+ alpm_list_free(remote_targets);
+ alpm_list_free(local_targets);
+ FREELIST(fetched_files);
/* now that targets are resolved, we can hand it all off to the sync code */
return sync_prepare_execute();
@@ -125,7 +117,9 @@ int pacman_upgrade(alpm_list_t *targets)
fail_release:
trans_release();
fail_free:
- free(file_is_remote);
+ alpm_list_free(remote_targets);
+ alpm_list_free(local_targets);
+ FREELIST(fetched_files);
return retval;
}
--
2.26.2
2
2
[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.2.1-97-g59e751f7
by Allan McRae 19 Jun '20
by Allan McRae 19 Jun '20
19 Jun '20
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".
The branch, master has been updated
via 59e751f72d09390067045168ac45a09a89419389 (commit)
via 7ba8e5f376528fcef9f7613304d8f18b2f664ee8 (commit)
from 02ae97b0da220d9079c6c2c1ac3e3ab0d12c1ac7 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 59e751f72d09390067045168ac45a09a89419389
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Sun Jun 14 14:52:02 2020 -0400
doc/pacman.8: fix typo
Fixes FS#67000
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
commit 7ba8e5f376528fcef9f7613304d8f18b2f664ee8
Author: Allan McRae <allan(a)archlinux.org>
Date: Mon Jun 15 19:17:41 2020 +1000
pacman-key: change signing key to RSA4096
RSA2048 may have been fine when this was written many moons ago, but time
this has a bump.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
-----------------------------------------------------------------------
Summary of changes:
doc/pacman.8.asciidoc | 2 +-
scripts/pacman-key.sh.in | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
hooks/post-receive
--
The official pacman repository
1
0
[pacman-dev] [GIT] The official pacman repository branch, release/5.2.x, updated. v5.2.1-25-g50f5e484
by Allan McRae 18 Jun '20
by Allan McRae 18 Jun '20
18 Jun '20
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".
The branch, release/5.2.x has been updated
via 50f5e484f278d3550da71246bfb51d3d8053bae8 (commit)
from 1e9cd30e488cd133d24eac7ed9cac7806db2c406 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 50f5e484f278d3550da71246bfb51d3d8053bae8
Author: Allan McRae <allan(a)archlinux.org>
Date: Fri Jun 19 09:49:28 2020 +1000
Pull translation updates
Signed-off-by: Allan McRae <allan(a)archlinux.org>
-----------------------------------------------------------------------
Summary of changes:
lib/libalpm/po/ar.po | 28 ++---
lib/libalpm/po/ast.po | 34 +++---
lib/libalpm/po/bg.po | 28 ++---
lib/libalpm/po/br.po | 28 ++---
lib/libalpm/po/ca.po | 28 ++---
lib/libalpm/po/cs.po | 28 ++---
lib/libalpm/po/da.po | 28 ++---
lib/libalpm/po/de.po | 45 ++++----
lib/libalpm/po/el.po | 56 ++++-----
lib/libalpm/po/en_GB.po | 28 ++---
lib/libalpm/po/eo.po | 262 +++++++++++++++++++++---------------------
lib/libalpm/po/es.po | 28 ++---
lib/libalpm/po/es_419.po | 32 +++---
lib/libalpm/po/eu.po | 28 ++---
lib/libalpm/po/eu_ES.po | 28 ++---
lib/libalpm/po/fi.po | 28 ++---
lib/libalpm/po/fr.po | 28 ++---
lib/libalpm/po/gl.po | 28 ++---
lib/libalpm/po/hr.po | 28 ++---
lib/libalpm/po/hu.po | 28 ++---
lib/libalpm/po/id.po | 28 ++---
lib/libalpm/po/it.po | 34 +++---
lib/libalpm/po/ja.po | 28 ++---
lib/libalpm/po/kk.po | 28 ++---
lib/libalpm/po/ko.po | 30 ++---
lib/libalpm/po/libalpm.pot | 28 ++---
lib/libalpm/po/lt.po | 28 ++---
lib/libalpm/po/nb.po | 28 ++---
lib/libalpm/po/nl.po | 28 ++---
lib/libalpm/po/pl.po | 41 +++----
lib/libalpm/po/pt.po | 28 ++---
lib/libalpm/po/pt_BR.po | 28 ++---
lib/libalpm/po/ro.po | 32 +++---
lib/libalpm/po/ru.po | 28 ++---
lib/libalpm/po/sk.po | 32 +++---
lib/libalpm/po/sl.po | 30 ++---
lib/libalpm/po/sr.po | 28 ++---
lib/libalpm/po/sr(a)latin.po | 28 ++---
lib/libalpm/po/sv.po | 55 ++++-----
lib/libalpm/po/tr.po | 36 +++---
lib/libalpm/po/uk.po | 28 ++---
lib/libalpm/po/zh_CN.po | 47 ++++----
lib/libalpm/po/zh_TW.po | 28 ++---
scripts/po/bg.po | 176 ++++++++++++++--------------
scripts/po/br.po | 176 ++++++++++++++--------------
scripts/po/ca.po | 176 ++++++++++++++--------------
scripts/po/cs.po | 176 ++++++++++++++--------------
scripts/po/da.po | 176 ++++++++++++++--------------
scripts/po/de.po | 208 +++++++++++++++++----------------
scripts/po/el.po | 216 +++++++++++++++++-----------------
scripts/po/en_GB.po | 176 ++++++++++++++--------------
scripts/po/eo.po | 203 ++++++++++++++++----------------
scripts/po/es.po | 176 ++++++++++++++--------------
scripts/po/es_419.po | 180 ++++++++++++++---------------
scripts/po/eu.po | 176 ++++++++++++++--------------
scripts/po/eu_ES.po | 176 ++++++++++++++--------------
scripts/po/fi.po | 176 ++++++++++++++--------------
scripts/po/fr.po | 178 ++++++++++++++--------------
scripts/po/gl.po | 176 ++++++++++++++--------------
scripts/po/hu.po | 176 ++++++++++++++--------------
scripts/po/id.po | 176 ++++++++++++++--------------
scripts/po/it.po | 180 ++++++++++++++---------------
scripts/po/ja.po | 176 ++++++++++++++--------------
scripts/po/ko.po | 176 ++++++++++++++--------------
scripts/po/lt.po | 176 ++++++++++++++--------------
scripts/po/nb.po | 182 ++++++++++++++---------------
scripts/po/nl.po | 176 ++++++++++++++--------------
scripts/po/pacman-scripts.pot | 176 ++++++++++++++--------------
scripts/po/pl.po | 215 +++++++++++++++++-----------------
scripts/po/pt.po | 176 ++++++++++++++--------------
scripts/po/pt_BR.po | 176 ++++++++++++++--------------
scripts/po/ro.po | 176 ++++++++++++++--------------
scripts/po/ru.po | 176 ++++++++++++++--------------
scripts/po/sk.po | 180 ++++++++++++++---------------
scripts/po/sl.po | 203 ++++++++++++++++----------------
scripts/po/sr.po | 176 ++++++++++++++--------------
scripts/po/sr(a)latin.po | 176 ++++++++++++++--------------
scripts/po/sv.po | 229 ++++++++++++++++++------------------
scripts/po/tr.po | 186 +++++++++++++++---------------
scripts/po/uk.po | 176 ++++++++++++++--------------
scripts/po/zh_CN.po | 247 +++++++++++++++++++--------------------
scripts/po/zh_TW.po | 200 ++++++++++++++++----------------
src/pacman/po/ar.po | 8 +-
src/pacman/po/ast.po | 10 +-
src/pacman/po/bg.po | 8 +-
src/pacman/po/br.po | 8 +-
src/pacman/po/ca.po | 8 +-
src/pacman/po/cs.po | 8 +-
src/pacman/po/da.po | 8 +-
src/pacman/po/de.po | 27 ++---
src/pacman/po/el.po | 75 ++++++------
src/pacman/po/en_GB.po | 8 +-
src/pacman/po/eo.po | 42 +++----
src/pacman/po/es.po | 16 +--
src/pacman/po/es_419.po | 12 +-
src/pacman/po/eu.po | 8 +-
src/pacman/po/eu_ES.po | 8 +-
src/pacman/po/fi.po | 8 +-
src/pacman/po/fr.po | 21 ++--
src/pacman/po/gl.po | 8 +-
src/pacman/po/hr.po | 16 +--
src/pacman/po/hu.po | 8 +-
src/pacman/po/id.po | 8 +-
src/pacman/po/it.po | 14 +--
src/pacman/po/ja.po | 8 +-
src/pacman/po/ko.po | 16 +--
src/pacman/po/lt.po | 20 ++--
src/pacman/po/nb.po | 40 +++----
src/pacman/po/nl.po | 12 +-
src/pacman/po/pacman.pot | 8 +-
src/pacman/po/pl.po | 67 ++++++-----
src/pacman/po/pt.po | 8 +-
src/pacman/po/pt_BR.po | 8 +-
src/pacman/po/ro.po | 12 +-
src/pacman/po/ru.po | 8 +-
src/pacman/po/sk.po | 12 +-
src/pacman/po/sl.po | 10 +-
src/pacman/po/sr.po | 8 +-
src/pacman/po/sr(a)latin.po | 8 +-
src/pacman/po/sv.po | 47 ++++----
src/pacman/po/tr.po | 16 +--
src/pacman/po/uk.po | 14 +--
src/pacman/po/zh_CN.po | 68 +++++------
src/pacman/po/zh_TW.po | 8 +-
124 files changed, 4782 insertions(+), 4738 deletions(-)
hooks/post-receive
--
The official pacman repository
1
0
[pacman-dev] [GIT] The official pacman repository branch, release/5.2.x, updated. v5.2.1-24-g1e9cd30e
by Andrew Gregory 18 Jun '20
by Andrew Gregory 18 Jun '20
18 Jun '20
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".
The branch, release/5.2.x has been updated
via 1e9cd30e488cd133d24eac7ed9cac7806db2c406 (commit)
via 2a345604cd8bd6a8fe5041ea7ec046c8d20aa30b (commit)
via f4da297de252dcb00e6c83df00fea11c86606348 (commit)
via 22e6daa79455b1c59ab73af9438ca0c1d2a884b9 (commit)
via d69da08abef42a52d2fa81daf4c045c3305598cf (commit)
via 12503767c0be8f3eb28433fd58bcab0ec6f44a26 (commit)
via 03cfe9e21c6aaecaac21e932229a55f5f1041a3c (commit)
via b3be0ce99b29e5384969c7b3e5ff6f4b0f581c4b (commit)
via 68418c54427b2ecbd5f882ffad261e0ba6e67c42 (commit)
via bb50e8d73afda7f17777c464bd7bfa127a749b8d (commit)
via b9d397c7317b9b495da73051fc670b0e9f472a06 (commit)
via 39ce2b766303f046bb75ad900cc8a2695e947f83 (commit)
via 01d5a68c1ae07ad43b5255721d2429710794ea04 (commit)
via 08aa3d97ea3823f2919e7d66041438db9e7488a0 (commit)
via 0bf4779cda35cfb8dc049ec5f27017c65c8ce716 (commit)
via d61c398b2cb3ec41e62a6e258b5791d0ce9de720 (commit)
via 7faa79526874c09fc89fb769744d103bc4918e17 (commit)
via 0d0a4bd68099596ba17b8c314664484c3819fbf3 (commit)
via 76c50e34393d68a740f825a6af9df8b7ac677303 (commit)
via c2fa9f85cccc4b576816330762fb55167a1ba78c (commit)
via 6f1a9e6ea85a49b8aeb3b7bd2454215e9c34f565 (commit)
via 41c3b1d78c3807beb4246624b20a31e374fb5af1 (commit)
via d6dcc936457815cdda87a8507cf82169b437e4d2 (commit)
via 019f9386ef2c85b4c3c9f9293f462cfb5ddcaa32 (commit)
from f6564377a2b0a0dd6294fb366a3f91a31142e124 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 1e9cd30e488cd133d24eac7ed9cac7806db2c406
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Mon Jun 8 22:03:18 2020 -0400
makepkg/repo-add: do not accept public-only keys for signing
If it's not listed by --list-secret-key we don't care if it has been
imported into your keyring, it's unusable. And you might not have a
private key at all in the no-keyid-specified case.
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 02ae97b0da220d9079c6c2c1ac3e3ab0d12c1ac7)
commit 2a345604cd8bd6a8fe5041ea7ec046c8d20aa30b
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Mon Jun 8 21:59:18 2020 -0400
makepkg/repo-add: handle GPGKEY with spaces
We pass this to gpg -u and this gpg option can accept a number of
different formats, not just the historical hexadecimal fingerprint we
assumed. We should not barf hard if a format is used which happens to
contain spaces.
This also fixes a validation bug. When we initially check if the desired
key is available, we don't quote spaces, so gpg goes ahead and treats
each space-separated string as a *different key* to search for,
returning partial matches, and returning success if at least one key is
found. But gpg --detach-sign -u will certainly not accept multiple keys!
Fixes FS#66949
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 899d39b635d46f9e2daff1aada75ea07f08fef64)
commit f4da297de252dcb00e6c83df00fea11c86606348
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Tue Jun 2 17:50:24 2020 -0400
libmakepkg: fix regression in sending plain() output to stderr
In commit 882e707e40bbade0111cf3bdedbdac4d4b70453b we changed message
output to go to stdout by default, unless it was an error. The plain()
function doesn't *look* like an error function, but in practice it was
-- it's used to continue multiline messages, and all in-tree uses were
for warning/error.
This is a problem both because we're sending output to the wrong place,
and because in some cases, we were performing error logging from a
function which would otherwise return a value to be captured in a
variable using command substution.
Fix this and straighten out the API by providing two functions: one for
continuing msg output, and one which wraps this by sending output to
stderr, for continuing error output.
Change all callers to use the second function.
(cherry picked from commit bf458cced7c0845f7b6fabb887d3878ae4cd51b2)
commit 22e6daa79455b1c59ab73af9438ca0c1d2a884b9
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Tue Jun 2 18:16:48 2020 -0400
makepkg: correctly handle missing download clients
This was broken in commit 882e707e40bbade0111cf3bdedbdac4d4b70453b,
which changed 'plain()' messages to go to stdout, which was then
captured as the download client in question: cmdline=("Aborting...").
The result was a very confusing error message e.g.
/usr/share/makepkg/source/file.sh: line 72: $'\E[1m': command not found
or with makepkg --nocolor:
/usr/share/makepkg/source/file.sh: line 72: Aborting...: command not found
The problem here is that we checked to see if an asynchronous subshell,
in our case <(...), failed, by checking if its captured stdout is
non-empty. Which is terrible, and also a limitation of old bash. But
bash 4.4 can use wait $! to retrieve the return value of an asynchronous
subshell. Now we target that as our minimum, we can sanely handle errors
in such functions.
Losing error messages on stdout by capturing them in a variable instead
of printing them, continues to be a problem, but this will be fixed
systematically in a later commit.
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 381e11375569fa7588b1297e0e744749bdafe8f5)
commit d69da08abef42a52d2fa81daf4c045c3305598cf
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Mon May 25 23:52:16 2020 -0400
makepkg: guard against undefined git pinned sources
If something like source=(..."#commit=") is used, e.g. due to failed
variable expansion, we try to check out an empty refspec as nothing at
all, and end up just running "git checkout". This happens because we
fail at variable expansion too -- so let's quote our variables properly
and make sure git sees this as an empty refspec, so it can error out.
Also make sure it is interpreted as a ref instead of a path.
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 817f9fb715b4e90d28bc141dfccfc530b9e951dc)
commit 12503767c0be8f3eb28433fd58bcab0ec6f44a26
Author: Allan McRae <allan(a)archlinux.org>
Date: Wed May 20 14:17:11 2020 +1000
libalpm/signing.c: Fix calculation of packet size in parse_subpacket
Given RFC 4880 provides the code to do this calculation, I am not sure
how I managed to stuff that up! This bug was only exposed when a
signature made with "include-key-block" was added to the Arch repos,
which provided a subpacket with the required size to hit this issue.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 5f6ef895b1dac04c7fb1b63acab2d42c19f91922)
commit 03cfe9e21c6aaecaac21e932229a55f5f1041a3c
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Mon May 11 00:16:30 2020 -0400
libmakepkg/strip: don't re-add the same debug source multiple times
It's either a waste of work, or triggers edge cases in some packages
(like coreutils-8.31) where the source file is readonly and cp gets a
permission denied error trying to overwrite it with an identical copy of
itself.
Also while we are at it, make the variable names be something readable,
because I could barely tell what this was doing while editing it.
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 3674144a74cfe897ec3ff46c18681df293290caa)
commit b3be0ce99b29e5384969c7b3e5ff6f4b0f581c4b
Author: Levente Polyak <anthraxx(a)archlinux.org>
Date: Mon May 11 00:45:49 2020 +0200
makepkg: deterministic PKGINFO libprovides for multiple library versions
While iterating over the provides array, the find call for locating a
shared library may result in listing multiple entries which by itself
does not produce a stable deterministic order and may vary depending on
the underlying filesystem.
To provide a stable listing and a reproducible .PKGINFO file the result
of find is piped to sort with a static LC_ALL=C localisation.
Signed-off-by: Levente Polyak <anthraxx(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 8ce142a2552418f64a74e773f659d92b065d6209)
commit 68418c54427b2ecbd5f882ffad261e0ba6e67c42
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Sun May 10 00:32:39 2020 -0400
Log invalid conf settings as an error
This is not a warning, _parse_options() returns failure without even
parsing further lines and the attempted pacman/pacman-conf program
execution immediately aborts. Warnings are for when e.g. later on if we
don't recognize a setting at all, we skip over it and have enough
confidence in this to continue executing the program.
The current implementation results in pacman-conf aborting with:
warning: config file /etc/pacman.conf, line 60: invalid value for 'ParallelDownloads' : '2.5'
error parsing '/etc/pacman.conf'
or pacman -Syu aborting with the entirely more cryptic:
warning: config file /etc/pacman.conf, line 59: invalid value for 'ParallelDownloads' : '2.5'
and this isn't just a problem for the newly added ParallelDownloads
setting, either, you could get the same problem if you specified a
broken XferCommand, but that's harder as it's more accepting of input
and you probably don't hit this except with unbalanced quotes.
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 8e769ddb8a59a9fbacf4614283d2fb519f022386)
commit bb50e8d73afda7f17777c464bd7bfa127a749b8d
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Sun May 10 00:32:40 2020 -0400
pacman-conf: fix incomplete support for ILoveCandy
This was only partially implemented in the original implementation.
`pacman-conf | grep ILoveCandy` would tell you if it was set, but
querying directly by name would not.
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 7423b166047ff168ef337818958bf86ed1d42a0a)
commit b9d397c7317b9b495da73051fc670b0e9f472a06
Author: Allan McRae <allan(a)archlinux.org>
Date: Tue Apr 14 23:45:24 2020 +1000
pacman.8: Fix typo
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 3ea7fdd96c874e111588a6e64ee30b75b55db785)
commit 39ce2b766303f046bb75ad900cc8a2695e947f83
Author: Earnestly <zibeon(a)googlemail.com>
Date: Wed Apr 1 11:29:57 2020 -0400
Use noextract with pacman-conf NoExtract
Current code accidently uses noupgrade for the NoExtract directive.
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 1d39557aa070d2260cfda650e59c8d190397ba01)
commit 01d5a68c1ae07ad43b5255721d2429710794ea04
Author: Allan McRae <allan(a)archlinux.org>
Date: Mon Feb 10 10:46:03 2020 +1000
build-aux/update-copyright 2019 2020
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit e76ec94083235ddc5510ab57b7c2bc12a1d34e8a)
commit 08aa3d97ea3823f2919e7d66041438db9e7488a0
Author: morganamilo <morganamilo(a)archlinux.org>
Date: Thu Jan 23 06:38:35 2020 +0000
libalpm: fix alpm_option_set_assumeinstalled
It looks like this function has never actually worked. The current list
is never set to NULL after being freed. So the new deps were just
appended to the already freed list, leading to a segfault.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 0a25548cd0910f66dea2dfab21f75a6d15366d64)
commit 0bf4779cda35cfb8dc049ec5f27017c65c8ce716
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Thu Jan 16 12:27:34 2020 -0500
makepkg: make per-package files containing '$pkgname' consistently work
Extracting function variables containing arbitrarily scoped variables of
arbitrary nature is a disaster, but let's at least cover the common case
of using the actual '$pkgname' in an install/changelog file. It's the
odd case of actually being basically justified use of disambiguating
between the same variable used in multiple different split packages...
and also, --printsrcinfo already uses and overwrites the variable
'pkgname' in pkgbuild_extract_to_srcinfo, so this "works" in .SRCINFO
but doesn't work in .src.tar.gz
It doesn't work in lint_pkgbuild either, but in that case the problem is
being too permissive, not too restrictive -- we might end up checking
the same file twice, and printing that it is missing twice.
Fixes FS#64932
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit d626a17ef94d79975ff17d04250152ea46aaade3)
commit d61c398b2cb3ec41e62a6e258b5791d0ce9de720
Author: Allan McRae <allan(a)archlinux.org>
Date: Thu Jan 23 10:47:02 2020 +1000
Fix "pacman -U <url>" operations
Commit e6a6d307 detected complete part files by comparing a payload's
max_size to initial_size. However, these values are also equal when we
use pacman -U on a URL as max_size is set to 0 in that case. Add a further
condition to avoid that.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit e54617c7d554e0c14c039432b5f7bef66e43769c)
commit 7faa79526874c09fc89fb769744d103bc4918e17
Author: Allan McRae <allan(a)archlinux.org>
Date: Sun Jan 19 09:30:54 2020 +1000
Increase maximum database size
We previously has the maximum database size as 25MB. This was set in the days
before repos had as many packages as they do now, and before we started
distributing files databases. Increase this limit to 128MB.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 2856a7dea3c0d4584e126b5ca5957e13e23f83d1)
commit 0d0a4bd68099596ba17b8c314664484c3819fbf3
Author: Allan McRae <allan(a)archlinux.org>
Date: Thu Jan 16 21:20:19 2020 +1000
Remove unneeded ltmain patch
Many moons ago, libtool was bad - I mean worse than today! It gobbled all
--as-needed and we ended up with an overlinked libalpm. This was annoying,
particularly when dealing with soname bumps in libraries pacman/libalpm had
no business linking to. Luckily we had a fix, stolen from GNOME I believe.
And with that fix, we lived in harmony with libtool for many years. Until one
day, unbeknownst to us, libtool was "fixed". We kept applying our patch,
because it still applied, but it did worse than nothing. It gobbled up our
other LDFLAGS, and our libalpm started missing out on RELRO and BIND_NOW.
This made the Arch Security Team unhappy. We will make them happy again by
stopping the patch.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 2e32d51a262fef398a94a17c0a0d4ff3233990b5)
[Andrew: restore autogen.sh]
commit 76c50e34393d68a740f825a6af9df8b7ac677303
Author: Allan McRae <allan(a)archlinux.org>
Date: Fri Nov 15 23:29:20 2019 +1000
Handle .part files that are the size of the correct package
In rare cases, likely due to a well timed Ctrl+C, but possibly due to a
broken mirror, a ".part" file may have size at least that of the correct
package size.
When encountering this issue, currently pacman fails in different ways
depending on where the package falls in the list to download. If last,
"wrong or NULL argument passed" error is reported, or a "invalid or
corrupt package" issue if not.
Capture these .part files, and remove the extension. This lets pacman
either use the package if valid, or offer to remove it if it fails checksum
or signature verification.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit e6a6d3079315ce722fec39604fddbab1c7ac79b3)
commit c2fa9f85cccc4b576816330762fb55167a1ba78c
Author: Allan McRae <allan(a)archlinux.org>
Date: Tue Nov 12 07:29:52 2019 +1000
libalpm/sync.c: Do not download missing keys multiple times
We now store key structs of our missing key info, so can not search the list
for string matches. This caused missing keys to be downloaded once for every
package they signed.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 540b19164b1ab3a4950b4a828fb90d047f4a591d)
commit 6f1a9e6ea85a49b8aeb3b7bd2454215e9c34f565
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Mon Nov 25 19:07:56 2019 -0500
makepkg: fix regression that broke extraction of file:// sources
In commit 9c817b654996249b8022e189ee7e2692f4668431 we made these sources
extendable, and heuristically determined the correct extraction
functions to use. But our fallback for protos that didn't have an exact
extract_* function didn't take into account that 'extract_file' matches
an actual proto... so we passed the netfile in while the function
expected a file.
Solution: the function should expect a netfile too, thereby allowing us
to delay an attempted resolution of netfile -> file, to the one case
where it is actually used. This makes us slightly more efficient in the
non-file case, makes our functions a bit more consistent, and makes
file:// extraction work again.
Fixes FS#64648
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit 349c22d043290ccd0cce9f30981f5415e295442a)
commit 41c3b1d78c3807beb4246624b20a31e374fb5af1
Author: Daniel T. Borelli <danieltborelli(a)gmail.com>
Date: Sat Nov 16 15:14:30 2019 +1000
Dereference double pointer before assigning NULL
Daniel T. Borelli <danieltborelli(a)gmail.com>
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit eaa2770c80f8f3b168b99021e4258d696523d38a)
commit d6dcc936457815cdda87a8507cf82169b437e4d2
Author: Eli Schwartz <eschwartz(a)archlinux.org>
Date: Thu Jan 9 20:59:35 2020 -0500
makepkg: fix one more file-seccomp issue
When file is called via fakeroot, it doesn't matter whether you use -z
or not, it is still incompatible with seccomp. Fix by configuring it
with FILECMD when used in the fakeroot 'tidy' run.
Fixes FS#65100
Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org>
commit 019f9386ef2c85b4c3c9f9293f462cfb5ddcaa32
Author: Allan McRae <allan(a)archlinux.org>
Date: Tue Nov 12 16:14:57 2019 +1000
pactest: set package tar format to GNU_FORMAT
python-3.8 changed the default tar format to PAX_FORMAT. This caused
issues in our testsuite with package extraction of files with UTF-8
characters as we run the tests under the C locale.
sycn600.py:
error: error while reading package /tmp/pactest-xuhri4xa/var/cache/pacman/pkg/unicodechars-2.0-1.pkg.tar.gz: Pathname can't be converted from UTF-8 to current locale.
Set format back to GNU_FORMAT.
Signed-off-by: Allan McRae <allan(a)archlinux.org>
(cherry picked from commit b9faf652735c603d1bdf849a570185eb721f11c1)
-----------------------------------------------------------------------
Summary of changes:
autogen.sh | 1 -
build-aux/ltmain-asneeded.patch | 33 ---------------------
doc/index.asciidoc | 2 +-
doc/pacman.8.asciidoc | 2 +-
lib/libalpm/add.c | 2 +-
lib/libalpm/add.h | 2 +-
lib/libalpm/alpm.c | 2 +-
lib/libalpm/alpm.h | 2 +-
lib/libalpm/alpm_list.c | 2 +-
lib/libalpm/alpm_list.h | 2 +-
lib/libalpm/backup.c | 2 +-
lib/libalpm/backup.h | 2 +-
lib/libalpm/be_local.c | 2 +-
lib/libalpm/be_package.c | 2 +-
lib/libalpm/be_sync.c | 6 ++--
lib/libalpm/conflict.c | 2 +-
lib/libalpm/conflict.h | 2 +-
lib/libalpm/db.c | 2 +-
lib/libalpm/db.h | 2 +-
lib/libalpm/deps.c | 2 +-
lib/libalpm/deps.h | 2 +-
lib/libalpm/diskspace.c | 2 +-
lib/libalpm/diskspace.h | 2 +-
lib/libalpm/dload.c | 8 ++++-
lib/libalpm/dload.h | 2 +-
lib/libalpm/error.c | 2 +-
lib/libalpm/filelist.c | 2 +-
lib/libalpm/filelist.h | 2 +-
lib/libalpm/graph.c | 2 +-
lib/libalpm/graph.h | 2 +-
lib/libalpm/group.c | 2 +-
lib/libalpm/group.h | 2 +-
lib/libalpm/handle.c | 3 +-
lib/libalpm/handle.h | 2 +-
lib/libalpm/hook.c | 2 +-
lib/libalpm/hook.h | 2 +-
lib/libalpm/libarchive-compat.h | 2 +-
lib/libalpm/log.c | 2 +-
lib/libalpm/log.h | 2 +-
lib/libalpm/package.c | 2 +-
lib/libalpm/package.h | 2 +-
lib/libalpm/pkghash.c | 2 +-
lib/libalpm/pkghash.h | 2 +-
lib/libalpm/remove.c | 2 +-
lib/libalpm/remove.h | 2 +-
lib/libalpm/signing.c | 8 ++---
lib/libalpm/signing.h | 2 +-
lib/libalpm/sync.c | 28 ++++++++++++++----
lib/libalpm/sync.h | 2 +-
lib/libalpm/trans.c | 2 +-
lib/libalpm/trans.h | 2 +-
lib/libalpm/util.c | 2 +-
lib/libalpm/util.h | 2 +-
lib/libalpm/version.c | 2 +-
scripts/libmakepkg/buildenv.sh.in | 2 +-
scripts/libmakepkg/buildenv/buildflags.sh.in | 2 +-
scripts/libmakepkg/buildenv/compiler.sh.in | 2 +-
scripts/libmakepkg/buildenv/debugflags.sh.in | 2 +-
scripts/libmakepkg/buildenv/makeflags.sh.in | 2 +-
scripts/libmakepkg/executable.sh.in | 2 +-
scripts/libmakepkg/executable/ccache.sh.in | 2 +-
scripts/libmakepkg/executable/checksum.sh.in | 2 +-
scripts/libmakepkg/executable/distcc.sh.in | 2 +-
scripts/libmakepkg/executable/fakeroot.sh.in | 2 +-
scripts/libmakepkg/executable/gpg.sh.in | 2 +-
scripts/libmakepkg/executable/gzip.sh.in | 2 +-
scripts/libmakepkg/executable/pacman.sh.in | 2 +-
scripts/libmakepkg/executable/strip.sh.in | 2 +-
scripts/libmakepkg/executable/sudo.sh.in | 2 +-
scripts/libmakepkg/executable/vcs.sh.in | 4 +--
scripts/libmakepkg/integrity.sh.in | 2 +-
.../libmakepkg/integrity/generate_checksum.sh.in | 2 +-
.../libmakepkg/integrity/generate_signature.sh.in | 8 ++---
scripts/libmakepkg/integrity/verify_checksum.sh.in | 2 +-
.../libmakepkg/integrity/verify_signature.sh.in | 4 +--
scripts/libmakepkg/lint_config.sh.in | 2 +-
scripts/libmakepkg/lint_config/ext.sh.in | 2 +-
scripts/libmakepkg/lint_config/paths.sh.in | 2 +-
.../libmakepkg/lint_config/source_date_epoch.sh.in | 2 +-
scripts/libmakepkg/lint_config/variable.sh.in | 2 +-
scripts/libmakepkg/lint_package.sh.in | 2 +-
.../libmakepkg/lint_package/build_references.sh.in | 2 +-
scripts/libmakepkg/lint_package/dotfiles.sh.in | 2 +-
scripts/libmakepkg/lint_package/file_names.sh.in | 2 +-
.../libmakepkg/lint_package/missing_backup.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/arch.sh.in | 2 +-
.../libmakepkg/lint_pkgbuild/arch_specific.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/backup.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/changelog.sh.in | 12 +++++---
.../libmakepkg/lint_pkgbuild/checkdepends.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/conflicts.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/depends.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/epoch.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/fullpkgver.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/install.sh.in | 12 +++++---
scripts/libmakepkg/lint_pkgbuild/makedepends.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/optdepends.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/options.sh.in | 2 +-
.../lint_pkgbuild/package_function.sh.in | 2 +-
.../lint_pkgbuild/package_function_variable.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/pkgbase.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/pkglist.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/pkgname.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/pkgrel.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/pkgver.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/provides.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/source.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/util.sh.in | 2 +-
scripts/libmakepkg/lint_pkgbuild/variable.sh.in | 2 +-
scripts/libmakepkg/source.sh.in | 5 ++--
scripts/libmakepkg/source/bzr.sh.in | 10 +++----
scripts/libmakepkg/source/file.sh.in | 11 +++----
scripts/libmakepkg/source/git.sh.in | 18 ++++++------
scripts/libmakepkg/source/hg.sh.in | 10 +++----
scripts/libmakepkg/source/local.sh.in | 2 +-
scripts/libmakepkg/source/svn.sh.in | 6 ++--
scripts/libmakepkg/srcinfo.sh.in | 2 +-
scripts/libmakepkg/tidy.sh.in | 2 +-
scripts/libmakepkg/tidy/docs.sh.in | 2 +-
scripts/libmakepkg/tidy/emptydirs.sh.in | 2 +-
scripts/libmakepkg/tidy/libtool.sh.in | 2 +-
scripts/libmakepkg/tidy/purge.sh.in | 2 +-
scripts/libmakepkg/tidy/staticlibs.sh.in | 2 +-
scripts/libmakepkg/tidy/strip.sh.in | 15 ++++++----
scripts/libmakepkg/tidy/zipman.sh.in | 2 +-
scripts/libmakepkg/util.sh.in | 2 +-
scripts/libmakepkg/util/compress.sh.in | 2 +-
scripts/libmakepkg/util/config.sh.in | 4 +--
scripts/libmakepkg/util/dirsize.sh.in | 2 +-
scripts/libmakepkg/util/error.sh.in | 2 +-
scripts/libmakepkg/util/message.sh.in | 10 ++++++-
scripts/libmakepkg/util/option.sh.in | 2 +-
scripts/libmakepkg/util/parseopts.sh.in | 2 +-
scripts/libmakepkg/util/pkgbuild.sh.in | 2 +-
scripts/libmakepkg/util/schema.sh.in | 2 +-
scripts/libmakepkg/util/source.sh.in | 6 ++--
scripts/libmakepkg/util/util.sh.in | 4 +--
scripts/makepkg-template.pl.in | 4 +--
scripts/makepkg.sh.in | 34 ++++++++++++----------
scripts/pacman-db-upgrade.sh.in | 4 +--
scripts/pacman-key.sh.in | 4 +--
scripts/repo-add.sh.in | 12 ++++----
scripts/wrapper.sh.in | 2 +-
src/common/ini.c | 2 +-
src/common/ini.h | 2 +-
src/common/util-common.c | 2 +-
src/common/util-common.h | 2 +-
src/pacman/callback.c | 2 +-
src/pacman/callback.h | 2 +-
src/pacman/check.c | 2 +-
src/pacman/check.h | 2 +-
src/pacman/conf.c | 4 +--
src/pacman/conf.h | 2 +-
src/pacman/database.c | 2 +-
src/pacman/deptest.c | 2 +-
src/pacman/files.c | 2 +-
src/pacman/package.c | 2 +-
src/pacman/package.h | 2 +-
src/pacman/pacman-conf.c | 6 ++--
src/pacman/pacman.c | 4 +--
src/pacman/pacman.h | 2 +-
src/pacman/query.c | 2 +-
src/pacman/remove.c | 2 +-
src/pacman/sighandler.c | 2 +-
src/pacman/sighandler.h | 2 +-
src/pacman/sync.c | 2 +-
src/pacman/upgrade.c | 2 +-
src/pacman/util.c | 2 +-
src/pacman/util.h | 2 +-
src/util/testpkg.c | 2 +-
src/util/vercmp.c | 2 +-
test/pacman/pactest.py | 2 +-
test/pacman/pmdb.py | 2 +-
test/pacman/pmenv.py | 2 +-
test/pacman/pmfile.py | 2 +-
test/pacman/pmpkg.py | 4 +--
test/pacman/pmrule.py | 2 +-
test/pacman/pmtest.py | 2 +-
test/pacman/tap.py | 2 +-
test/pacman/util.py | 2 +-
test/util/vercmptest.sh | 2 +-
182 files changed, 309 insertions(+), 293 deletions(-)
delete mode 100644 build-aux/ltmain-asneeded.patch
hooks/post-receive
--
The official pacman repository
1
0