November 2022 - Pacman-dev - lists.archlinux.org

Working on SandboxUser patch
by Allan McRae 12 Feb '23

12 Feb '23

Just to avoid any potential overlap - I am working on the SandboxUser patch that Remi submitted a long while ago... https://lists.archlinux.org/archives/list/pacman-dev@lists.archlinux.org/th… The patch is far too big for me to confidently review in one go, so I am splitting it into smaller sections as I go. Progress can be seen here: https://gitlab.archlinux.org/pacman/pacman/-/commits/allan/privsep Allan

7 21

[PATCH] repo-add: do not recreate the database if nothing was changed
by Ivan Shapovalov 08 Feb '23

08 Feb '23

Signed-off-by: Ivan Shapovalov <intelfx(a)intelfx.name> --- When `repo-add` is used to add a package that already exists in the database, it will recompress the database unconditionally, even if `--new` is passed. Avoid doing that. scripts/repo-add.sh.in | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index 221abdc0..78195de1 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -44,6 +44,8 @@ USE_COLOR='y' PREVENT_DOWNGRADE=0 INCLUDE_SIGS=0 +ENTRIES_CHANGED=0 + # Import libmakepkg source "$LIBRARY"/util/compress.sh source "$LIBRARY"/util/message.sh @@ -343,6 +345,8 @@ db_write_entry() { rm -f ${oldfile} ${oldfile}.sig fi + (( ++ENTRIES_CHANGED )) + return 0 } # end db_write_entry @@ -365,6 +369,9 @@ db_remove_entry() { pkgentry=$(find_pkgentry "$pkgname") done + if [[ $notfound == 0 ]]; then + (( ++ENTRIES_CHANGED )) + fi return $notfound } # end db_remove_entry @@ -684,13 +691,13 @@ for arg in "${args[@]:1}"; do done # if the whole operation was a success, re-zip and rotate databases -if (( !fail )); then +if (( !fail )) && (( ENTRIES_CHANGED > 0 )); then msg "$(gettext "Creating updated database file '%s'")" "$REPO_DB_FILE" create_db rotate_db else msg "$(gettext "No packages modified, nothing to do.")" - exit 1 + exit $(( fail != 0 )) fi exit 0 -- 2.38.0

3 6

RFC: Simple package diff
by Allan McRae 13 Dec '22

13 Dec '22

The idea of package deltas just won't go away... However, binary diffs really are not ideal with pacman verifying the compressed package - that means we need to reconstruct the package on the users system to verify. Also our old approach using xdelta3 somewhat died when moving packages away from gz (or xz?) compression. Other binary diff approaches really suffered the same issue. In general, I find the approach of reconstructing the full package to be suboptimal. I also don't particulaly want to verify uncompressed packages. I wondered if this was a case of perfect being the enemy of good, so I have investigated a different, very lazy approach. Instead of taking a binary diff, we could just provide the files that have changed between package versions. This is super easy to do as we have checksums for all files in the mtree file. We could then extract this "diff" package directly, and use the mtree file to adjust timestamps/permissions/etc(?) on kept files, and it would be just like the full package had been installed. I ran some numbers to see if this was worth while. The results for the last bunch of updates for bash, coreutils, qt5-base and systemd are given here: https://wiki.archlinux.org/title/User:Allan/Pkgdiff On major version updates, this is approach is a waste of time. But for minor updates bash download would average 25% of the size, coreutils about 36% (though was ~1% for simple rebuilds!), qt5-base about 40% and systemd 60%. Not shown but worth noting note that when Arch changes gcc/binutils versions or updates CFLAGS etc, this can stop any binary diff being as useful. If we implemented using these diffs but only allowed it for updates from the previous package version (i.e. no diffs to package (current - 2) or earlier, or diff chaining), then this would be rather simple to implement (at least from the pacman side...). Any comments before I invest time implementing? Allan

3 4

[PATCH] Add support for a Proxy directive
by Brian Geffon 21 Nov '22

21 Nov '22

Currently the only way to use a proxy is via XferCommand; however, when doing so there is no support for parallel downloads. This change introduces a new directive Proxy which supports whatever proxy protocols are supported by libcurl. Signed-off-by: Brian Geffon <briang(a)apache.org> --- doc/pacman.conf.5.asciidoc | 7 +++++++ etc/pacman.conf.in | 5 +++++ lib/libalpm/alpm.h | 24 ++++++++++++++++++++++++ lib/libalpm/dload.c | 3 +++ lib/libalpm/handle.c | 22 ++++++++++++++++++++++ lib/libalpm/handle.h | 1 + scripts/completion/zsh_completion.in | 1 + src/pacman/conf.c | 10 ++++++++++ src/pacman/conf.h | 2 ++ src/pacman/pacman-conf.c | 4 +++- 10 files changed, 78 insertions(+), 1 deletion(-) diff --git a/doc/pacman.conf.5.asciidoc b/doc/pacman.conf.5.asciidoc index 41f3ea03..657264e1 100644 --- a/doc/pacman.conf.5.asciidoc +++ b/doc/pacman.conf.5.asciidoc @@ -132,6 +132,13 @@ Options HTTP/FTP support, or need the more advanced proxy support that comes with utilities like wget. +*Proxy =* https://host:port:: + If set, use this endpoint as the proxy for downloads. The currently + supported set of protocols are http, https, socks4, socks4a, socks5, + and socks5h. If no port is specified it defaults to 1080. This option + only applies when using the built in downloader. When using XferCommand + any proxy settings must be passed as command line arguments to the command. + *NoUpgrade =* file ...:: All files listed with a `NoUpgrade` directive will never be touched during a package install/upgrade, and the new files will be installed with a diff --git a/etc/pacman.conf.in b/etc/pacman.conf.in index 1799efc7..6b2ef8b9 100644 --- a/etc/pacman.conf.in +++ b/etc/pacman.conf.in @@ -21,6 +21,11 @@ HoldPkg = pacman glibc #CleanMethod = KeepInstalled Architecture = auto +# The Proxy directive only applies when not using an XferCommand. When using +# an XferCommand any proxy strings must be passed as arguments to the command. +#Proxy = socks5h://127.0.0.1:1080 +#Proxy = https://127.0.0.1:8080 + # Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup #IgnorePkg = #IgnoreGroup = diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index 07e16b9f..37dd414f 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -2166,6 +2166,30 @@ int alpm_option_set_dbext(alpm_handle_t *handle, const char *dbext); /* End of dbext accessors */ /** @} */ +/** @name Accessors for proxy server configuration + * + * This controls the proxy server that will be used for downloads. + * This may be necessary for computers that only have internet access + * via HTTP or Socks proxies. + * + * @{ + */ + +/** Gets the configured proxy. + * @param handle the context handle + * @return the configured proxy + */ +const char *alpm_option_get_proxy(alpm_handle_t *handle); + +/** Sets the proxy server. + * @param handle the context handle + * @param proxy the full proxy string to use. + * @return 0 on success, -1 on error (pm_errno is set accordingly) + */ +int alpm_option_set_proxy(alpm_handle_t *handle, const char *proxy); + +/* End of proxy accessors */ +/** @} */ /** @name Accessors for the signature levels * @{ diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 4fa17b35..171a802c 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -340,6 +340,9 @@ static void curl_set_handle_opts(CURL *curl, struct dload_payload *payload) curl_easy_setopt(curl, CURLOPT_TCP_KEEPINTVL, 60L); curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY); curl_easy_setopt(curl, CURLOPT_PRIVATE, (void *)payload); + if (handle->proxy) { + curl_easy_setopt(curl, CURLOPT_PROXY, handle->proxy); + } _alpm_log(handle, ALPM_LOG_DEBUG, "%s: url is %s\n", payload->remote_name, payload->fileurl); diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c index d1eafeda..e72627ea 100644 --- a/lib/libalpm/handle.c +++ b/lib/libalpm/handle.c @@ -73,6 +73,7 @@ void _alpm_handle_free(alpm_handle_t *handle) FREE(handle->root); FREE(handle->dbpath); FREE(handle->dbext); + FREE(handle->proxy); FREELIST(handle->cachedirs); FREELIST(handle->hookdirs); FREE(handle->logfile); @@ -330,6 +331,12 @@ const char SYMEXPORT *alpm_option_get_dbext(alpm_handle_t *handle) return handle->dbext; } +const char SYMEXPORT *alpm_option_get_proxy(alpm_handle_t *handle) +{ + CHECK_HANDLE(handle, return NULL); + return handle->proxy; +} + int SYMEXPORT alpm_option_get_parallel_downloads(alpm_handle_t *handle) { CHECK_HANDLE(handle, return -1); @@ -822,6 +829,21 @@ int SYMEXPORT alpm_option_set_dbext(alpm_handle_t *handle, const char *dbext) return 0; } +int SYMEXPORT alpm_option_set_proxy(alpm_handle_t *handle, const char *proxy) +{ + CHECK_HANDLE(handle, return -1); + ASSERT(proxy, RET_ERR(handle, ALPM_ERR_WRONG_ARGS, -1)); + + if(handle->proxy) { + FREE(handle->proxy); + } + + STRDUP(handle->proxy, proxy, RET_ERR(handle, ALPM_ERR_MEMORY, -1)); + + _alpm_log(handle, ALPM_LOG_DEBUG, "option 'proxy' = %s\n", handle->proxy); + return 0; +} + int SYMEXPORT alpm_option_set_default_siglevel(alpm_handle_t *handle, int level) { diff --git a/lib/libalpm/handle.h b/lib/libalpm/handle.h index 3a464689..32063ef9 100644 --- a/lib/libalpm/handle.h +++ b/lib/libalpm/handle.h @@ -107,6 +107,7 @@ struct _alpm_handle_t { int usesyslog; /* Use syslog instead of logfile? */ /* TODO move to frontend */ int checkspace; /* Check disk space before installing */ char *dbext; /* Sync DB extension */ + char *proxy; /* Proxy server to use */ int siglevel; /* Default signature verification level */ int localfilesiglevel; /* Signature verification level for local file upgrade operations */ diff --git a/scripts/completion/zsh_completion.in b/scripts/completion/zsh_completion.in index f65edeb2..69b19a18 100644 --- a/scripts/completion/zsh_completion.in +++ b/scripts/completion/zsh_completion.in @@ -516,6 +516,7 @@ _pacman_conf_general_directives=( 'DisableDownloadTimeout' 'NoProgressBar' 'ParallelDownloads' + 'Proxy' 'CleanMethod' 'SigLevel' 'LocalFileSigLevel' diff --git a/src/pacman/conf.c b/src/pacman/conf.c index f9edf75b..37691519 100644 --- a/src/pacman/conf.c +++ b/src/pacman/conf.c @@ -154,6 +154,7 @@ int config_free(config_t *oldconfig) free(oldconfig->rootdir); free(oldconfig->dbpath); free(oldconfig->logfile); + free(oldconfig->proxy); free(oldconfig->gpgdir); FREELIST(oldconfig->hookdirs); FREELIST(oldconfig->cachedirs); @@ -668,6 +669,11 @@ static int _parse_options(const char *key, char *value, config->logfile = strdup(value); pm_printf(ALPM_LOG_DEBUG, "config: logfile: %s\n", value); } + } else if(strcmp(key, "Proxy") == 0) { + if(!config->proxy) { + config->proxy = strdup(value); + pm_printf(ALPM_LOG_DEBUG, "config: proxy: %s\n", value); + } } else if(strcmp(key, "XferCommand") == 0) { char **c; if((config->xfercommand_argv = wordsplit(value)) == NULL) { @@ -901,6 +907,10 @@ static int setup_libalpm(void) pm_printf(ALPM_LOG_WARNING, _("no '%s' configured\n"), "XferCommand"); } + if (config->proxy) { + alpm_option_set_proxy(handle, config->proxy); + } + alpm_option_set_architectures(handle, config->architectures); alpm_option_set_checkspace(handle, config->checkspace); alpm_option_set_usesyslog(handle, config->usesyslog); diff --git a/src/pacman/conf.h b/src/pacman/conf.h index f7916ca9..c441cd80 100644 --- a/src/pacman/conf.h +++ b/src/pacman/conf.h @@ -115,6 +115,8 @@ typedef struct __config_t { unsigned short verbosepkglists; /* number of parallel download streams */ unsigned int parallel_downloads; + /* the proxy server to use */ + char *proxy; /* select -Sc behavior */ unsigned short cleanmethod; alpm_list_t *holdpkg; diff --git a/src/pacman/pacman-conf.c b/src/pacman/pacman-conf.c index a9d1f52b..baf3cebd 100644 --- a/src/pacman/pacman-conf.c +++ b/src/pacman/pacman-conf.c @@ -269,6 +269,7 @@ static void dump_config(void) show_bool("ILoveCandy", config->chomp); show_bool("NoProgressBar", config->noprogressbar); + show_str("Proxy", config->proxy); show_int("ParallelDownloads", config->parallel_downloads); show_cleanmethod("CleanMethod", config->cleanmethod); @@ -384,7 +385,8 @@ static int list_directives(void) } else if(strcasecmp(i->data, "ParallelDownloads") == 0) { show_int("ParallelDownloads", config->parallel_downloads); - + } else if(strcasecmp(i->data, "Proxy") == 0) { + show_str("Proxy", config->proxy); } else if(strcasecmp(i->data, "CleanMethod") == 0) { show_cleanmethod("CleanMethod", config->cleanmethod); -- 2.38.0

2 4

[PATCH] Add --noconfirm-unsafe for advanced script usage
by Laszlo Toth 09 Nov '22

09 Nov '22

This is intended for e.g., regular, automated cache clearing or updating/creating Docker images. There is --noconfirm, but that selects the default answer for yes/no questions, which may be no when we want yes. Then 'yes | pacman -Scc' could be used, but that does not work reliably w/ multiselection. So, add a new option which always answers yes for yes/no, and the default for multiselections. Now --confirm disables both --noconfirm and --noconfirm-unsafe. noask is not updated to use this mode as well... it's not safe. Signed-off-by: Laszlo Toth <laszlth(a)gmail.com> --- doc/pacman.8.asciidoc | 11 ++++++++--- src/pacman/conf.h | 4 +++- src/pacman/pacman.c | 8 ++++++++ src/pacman/util.c | 18 ++++++++++++++---- 4 files changed, 33 insertions(+), 8 deletions(-) diff --git a/doc/pacman.8.asciidoc b/doc/pacman.8.asciidoc index 630ff0d9..ec400175 100644 --- a/doc/pacman.8.asciidoc +++ b/doc/pacman.8.asciidoc @@ -185,11 +185,16 @@ Options the installation root setting. *\--noconfirm*:: - Bypass any and all ``Are you sure?'' messages. It's not a good idea to do - this unless you want to run pacman from a script. + Bypass any and all ``Are you sure?'' messages via using the default answer. + It's not a good idea to do this unless you want to run pacman from a script. + +*\--noconfirm-unsafe*:: + Bypass any and all ``Are you sure?'' messages via answering yes or + selecting the default option. + It's NOT a good idea to do this unless you want to run pacman from a script. *\--confirm*:: - Cancels the effects of a previous '\--noconfirm'. + Cancels the effects of a previous '\--noconfirm' or '\--noconfirm-unsafe'. *\--disable-download-timeout*:: Disable defaults for low speed limit and timeout on downloads. Use this diff --git a/src/pacman/conf.h b/src/pacman/conf.h index f7916ca9..fbed3055 100644 --- a/src/pacman/conf.h +++ b/src/pacman/conf.h @@ -57,6 +57,7 @@ typedef struct __config_t { unsigned short usesyslog; unsigned short color; unsigned short disable_dl_timeout; + unsigned short noconfirmunsafe; char *print_format; /* unfortunately, we have to keep track of paths both here and in the library * because they can come from both the command line or config file, and we @@ -210,7 +211,8 @@ enum { OP_DOWNLOADONLY, OP_REFRESH, OP_ASSUMEINSTALLED, - OP_DISABLEDLTIMEOUT + OP_DISABLEDLTIMEOUT, + OP_NOCONFIRMUNSAFE }; /* clean method */ diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c index e5c6e420..bb04fdc8 100644 --- a/src/pacman/pacman.c +++ b/src/pacman/pacman.c @@ -226,6 +226,8 @@ static void usage(int op, const char * const myname) addlist(_(" --confirm always ask for confirmation\n")); addlist(_(" --disable-download-timeout\n" " use relaxed timeouts for download\n")); + addlist(_(" --noconfirm-unsafe\n" + " answer yes or default for every confirmation\n")); } list = alpm_list_msort(list, alpm_list_count(list), options_cmp); for(i = list; i; i = alpm_list_next(i)) { @@ -442,8 +444,13 @@ static int parsearg_global(int opt) case OP_NOCONFIRM: config->noconfirm = 1; break; + case OP_NOCONFIRMUNSAFE: + config->noconfirm = 0; + config->noconfirmunsafe = 1; + break; case OP_CONFIRM: config->noconfirm = 0; + config->noconfirmunsafe = 0; break; case OP_DBPATH: case 'b': @@ -948,6 +955,7 @@ static int parseargs(int argc, char *argv[]) {"dbonly", no_argument, 0, OP_DBONLY}, {"color", required_argument, 0, OP_COLOR}, {"disable-download-timeout", no_argument, 0, OP_DISABLEDLTIMEOUT}, + {"noconfirm-unsafe", no_argument, 0, OP_NOCONFIRMUNSAFE}, {0, 0, 0, 0} }; diff --git a/src/pacman/util.c b/src/pacman/util.c index 5f5c7c54..532e058d 100644 --- a/src/pacman/util.c +++ b/src/pacman/util.c @@ -1527,7 +1527,7 @@ int multiselect_question(char *array, int count) FILE *stream; size_t response_len = 64; - if(config->noconfirm) { + if(config->noconfirm || config->noconfirmunsafe) { stream = stdout; } else { /* Use stderr so questions are always displayed when redirecting output */ @@ -1550,7 +1550,7 @@ int multiselect_question(char *array, int count) fprintf(stream, ": "); fflush(stream); - if(config->noconfirm) { + if(config->noconfirm || config->noconfirmunsafe) { fprintf(stream, "\n"); break; } @@ -1604,7 +1604,7 @@ int select_question(int count) FILE *stream; int preset = 1; - if(config->noconfirm) { + if(config->noconfirm || config->noconfirmunsafe) { stream = stdout; } else { /* Use stderr so questions are always displayed when redirecting output */ @@ -1617,6 +1617,11 @@ int select_question(int count) fprintf(stream, ": "); fflush(stream); + if(config->noconfirmunsafe) { + fprintf(stream, "y\n"); + break; + } + if(config->noconfirm) { fprintf(stream, "\n"); break; @@ -1682,7 +1687,7 @@ static int question(short preset, const char *format, va_list args) FILE *stream; int fd_in = fileno(stdin); - if(config->noconfirm) { + if(config->noconfirm || config->noconfirmunsafe) { stream = stdout; } else { /* Use stderr so questions are always displayed when redirecting output */ @@ -1705,6 +1710,11 @@ static int question(short preset, const char *format, va_list args) fputs(config->colstr.nocolor, stream); fflush(stream); + if(config->noconfirmunsafe) { + fprintf(stream, "y\n"); + return preset; + } + if(config->noconfirm) { fprintf(stream, "\n"); return preset; -- 2.38.1

3 2

[Git][pacman/pacman][master] 3 commits: Remove obsolete Doxyfile declarations
by Allan McRae (＠allan) 08 Nov '22

08 Nov '22

Allan McRae pushed to branch master at Pacman / Pacman Commits: 83e5326a by Allan McRae at 2022-11-08T11:49:31+10:00 Remove obsolete Doxyfile declarations Signed-off-by: Allan McRae <allan(a)archlinux.org> - - - - - 05e52ef7 by Allan McRae at 2022-11-08T14:19:35+10:00 Document alpm_pkg_xdata_t type and accessor Also more definitions to more relevant areas in alpm.h. Signed-off-by: Allan McRae <allan(a)archlinux.org> - - - - - 23aa8281 by Allan McRae at 2022-11-08T14:20:36+10:00 Fix typo in comment Signed-off-by: Allan McRae <allan(a)archlinux.org> - - - - - 2 changed files: - doc/Doxyfile.in - lib/libalpm/alpm.h View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/compare/b2c9543ea94bbfd00480ab… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/compare/b2c9543ea94bbfd00480ab… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[Git][pacman/pacman][master] Use modern openssl interfaces for calculating MD5 and SHA256 checksum
by Allan McRae (＠allan) 07 Nov '22

07 Nov '22

Allan McRae pushed to branch master at Pacman / Pacman Commits: b2c9543e by Allan McRae at 2022-11-07T16:58:29+10:00 Use modern openssl interfaces for calculating MD5 and SHA256 checksum The currently used openssl interfaces for calculating checksums have been deprecated in openssl-3.0. Move to the modern interfaces to avoid build warnings. Signed-off-by: Allan McRae <allan(a)archlinux.org> - - - - - 1 changed file: - lib/libalpm/util.c View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/b2c9543ea94bbfd00480ab6… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/b2c9543ea94bbfd00480ab6… You're receiving this email because of your account on gitlab.archlinux.org.

1 0

[Git][pacman/pacman][master] executable/meson.build: include debugedit.sh.in
by Allan McRae (＠allan) 07 Nov '22

07 Nov '22

Allan McRae pushed to branch master at Pacman / Pacman Commits: 3ed08f97 by Morten Linderud at 2022-11-07T01:02:12+01:00 executable/meson.build: include debugedit.sh.in Signed-off-by: Morten Linderud <morten(a)linderud.pw> - - - - - 1 changed file: - scripts/libmakepkg/executable/meson.build View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/3ed08f97ecd5941c3f72e3e… -- View it on GitLab: https://gitlab.archlinux.org/pacman/pacman/-/commit/3ed08f97ecd5941c3f72e3e… You're receiving this email because of your account on gitlab.archlinux.org.

1 0