This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 22a58f5420438f35effb991696d37529d7a31969 (commit) via 557845bc971ff272c53da773baea277a2d2d47b8 (commit) via 0346e0eef224ab8ba22b659026ffdf2bfe95f3ae (commit) via b96e0df4dceaa2677baa1a3563211950708d3e63 (commit) via c78eb48d915dc22146073162dda08ddf73c4a508 (commit) via 64c4669f579dc5ad8d05329abffbd752ad0ed8f2 (commit) via 6a331af27fe6dc7842725d067fd2fb4a1c60c139 (commit) via 1d42a8f954eed3205e84cfb946b67ba889b04135 (commit) via fa68c33fa821cd7ddc5ae32baf62af2a238e44dc (commit) via dc98d0ea09f3632cd28a12099f3f09d466dcad1d (commit) via a8a1a1bb3ec98a8471cb5cd13d096f39a267f789 (commit) via fe8e13341bdeae4a59c0270a632c29e71ae9deda (commit) via cffda331adca0aedd7c1fc17d739c27fc8041a20 (commit) from 411b12d09ddb47722acf6d354c17ee9ee5228964 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 22a58f5420438f35effb991696d37529d7a31969 Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Sat Apr 18 19:15:44 2020 -0700 Swap alpm_db_update() implementation to multiplexed version Now when all callers of the old alpm_db_update() function are gone we can remove this implementation. And then rename alpm_dbs_update() function to alpm_db_update(). Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit 557845bc971ff272c53da773baea277a2d2d47b8 Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Sat Apr 18 19:05:13 2020 -0700 Convert downloading databases to the new multiplexed API Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit 0346e0eef224ab8ba22b659026ffdf2bfe95f3ae Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Sat Apr 18 19:00:49 2020 -0700 Convert download packages logic to multiplexed API Create a list of dload_payloads and pass it to the new _alpm_multi_* interface. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit b96e0df4dceaa2677baa1a3563211950708d3e63 Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Fri Apr 17 10:36:38 2020 -0700 Implement multibar UI Multiplexed download requires ability to draw UI for multiple active progress bars. To implement it we use ANSI codes to move cursor up/down and then redraw the required progress bar. `pacman_multibar_ui.active_downloads` field represents the list of active downloads that correspond to progress bars. `struct pacman_progress_bar` is a data structure for a progress bar. In some cases (e.g. database downloads) we want to keep progress bars in order. In some other cases (package downloads) we want to move completed items to the top of the screen. Function `multibar_move_completed_up` allows to configure such behavior. Per discussion in the maillist we do not want to show download progress for signature files. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit c78eb48d915dc22146073162dda08ddf73c4a508 Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Tue May 5 19:50:51 2020 -0700 Extend download callback interface with start/complete events With the previous download interface the callback uses the first progress event as 'download has started' signal. Unfortunately it does not work with up-to-date files that never receive 'download progress' events. Up-to-date database messages are currently handled in sync_syncdbs() after the sequential download is completed and a result from ALPM is received. But this is not going to work with multiplexed download interface that returns the result only after all files are completed. Another problem with 'first progress event is the beginning of the download' is that such events time are unpredictable. Thus the UI progress bar order might differ from what has been passed by client to alpm_dbs_update() function. We actually want to keep the dbs progress bars in a strict order. To help to solve the given problems extend the download callback to allow 2 more events - download started and completed. 'Download started' events appear in the same order as in the list given by a client. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit 64c4669f579dc5ad8d05329abffbd752ad0ed8f2 Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Tue May 5 18:28:49 2020 -0700 Introduce event types for start/end database list download Multiplexed database/files downloads will use multiple progress bars. The UI logic is quite complicated and printing error messages while handling multiple progress bars is going to be challenging. Instead we are going to save all ALPM error messages to a list and flush it at the end of the download process. Use on_progress variable that blocks error messages printing. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit 6a331af27fe6dc7842725d067fd2fb4a1c60c139 Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Mon Apr 13 21:48:55 2020 -0700 Implement multiplexed download using mCURL curl_multi_download_internal() is the main loop that creates up to 'ParallelDownloads' easy curl handles, adds them to mcurl and then performs curl execution. This is when the paralled downloads happens. Once any of the downloads complete the function checks its result. In case if the download fails it initiates retry with the next server from payload->servers list. At the download completion all the payload resources are cleaned up. curl_multi_check_finished_download() is essentially refactored version of curl_download_internal() adopted for multi_curl. Once mcurl porting is complete curl_download_internal() will be removed. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit 1d42a8f954eed3205e84cfb946b67ba889b04135 Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Mon Apr 13 21:40:54 2020 -0700 Implement _alpm_multi_download It is an equivalent of _alpm_download but accepts a list of payloads. curl_multi_download_internal() is a stub at this moment and will be implemented in the later commits of this patch series. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit fa68c33fa821cd7ddc5ae32baf62af2a238e44dc Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Mon Mar 9 15:36:48 2020 -0700 Inline dload_payload->curlerr field into a local variable dload_payload->curlerr is a field that is used inside curl_download_internal() function only. It can be converted to a local variable. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit dc98d0ea09f3632cd28a12099f3f09d466dcad1d Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Mon Mar 9 15:23:12 2020 -0700 Add multi_curl handle to ALPM global context To be able to run multiple download in parallel efficiently we need to use curl_multi interface [1]. It introduces a set of APIs over new type of handler 'CURLM'. Create CURLM object at the application start and set it to global ALPM context. The 'single-download' CURL handle moves to payload struct. A new CURL handle is created for each payload with intention to be processed by CURLM. Note that curl_download_internal() is not ported to CURLM interface due to the fact that the function will go away soon. [1] https://curl.haxx.se/libcurl/c/libcurl-multi.html Signed-off-by: Allan McRae <allan@archlinux.org> commit a8a1a1bb3ec98a8471cb5cd13d096f39a267f789 Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Thu Mar 26 13:19:59 2020 -0700 Introduce alpm_dbs_update() function for parallel db updates This is an equivalent of alpm_db_update but for multiplexed (parallel) download. The difference is that this function accepts list of databases to update. And then ALPM internals download it in parallel if possible. Add a stub for _alpm_multi_download the function that will do parallel payloads downloads in the future. Introduce dload_payload->filepath field that contains url path to the file we download. It is like fileurl field but does not contain protocol/server part. The rationale for having this field is that with the curl multidownload the server retry logic is going to move to a curl callback. And the callback needs to be able to reconstruct the 'next' fileurl. One will be able to do it by getting the next server url from 'servers' list and then concat with filepath. Once the 'parallel download' refactoring is over 'fileurl' field will go away. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit fe8e13341bdeae4a59c0270a632c29e71ae9deda Author: Anatol Pomozov <anatol.pomozov@gmail.com> Date: Sun Mar 8 13:33:32 2020 -0700 Add config option to specify amount of parallel download streams It includes pacman.conf new 'ParallelDownloads' option that specifies how many concurrent downloads cURL starts in parallel. Add alpm_option_set_parallel_downloads() ALPM function that allows to set this config option programmatically. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org> commit cffda331adca0aedd7c1fc17d739c27fc8041a20 Author: Eli Schwartz <eschwartz@archlinux.org> Date: Tue May 5 22:09:42 2020 -0400 doc: remove vim modelines from BUILDINFO(5) We (thought we) removed all modelines from the project in commit 860e4c4943ad062bd0eff99f28e7d64804b3c08e, but apparently this one sneaked in by virtue of this manpage being added to the project after the "remove all the modelines" patch was submitted, but before it was applied. I must have failed to update the patch to remove it from this file also. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org> ----------------------------------------------------------------------- Summary of changes: README | 14 ++ doc/BUILDINFO.5.asciidoc | 3 - doc/pacman.conf.5.asciidoc | 5 + etc/pacman.conf.in | 1 + lib/libalpm/alpm.c | 12 +- lib/libalpm/alpm.h | 106 +++++---- lib/libalpm/be_sync.c | 195 ++++++++--------- lib/libalpm/dload.c | 521 +++++++++++++++++++++++++++++++++++++++++---- lib/libalpm/dload.h | 11 +- lib/libalpm/handle.c | 20 +- lib/libalpm/handle.h | 3 +- lib/libalpm/sync.c | 75 +++---- src/pacman/callback.c | 408 +++++++++++++++++++++++------------ src/pacman/callback.h | 7 +- src/pacman/conf.c | 58 ++++- src/pacman/conf.h | 2 + src/pacman/sync.c | 2 + src/pacman/util.c | 45 ++-- src/pacman/util.h | 4 + 19 files changed, 1086 insertions(+), 406 deletions(-) hooks/post-receive -- The official pacman repository