[pacman-dev] [PATCH 1/1] RFC: handle soft failure in downloads
From: Christian Hesse <mail@eworm.de> A server can indicate a soft failure by returning 412 (Precondition Failed). Only a warning (instead of error) message is logged and download is retried with next server, without counting server error. This http code can be used by servers that are not expected to be complete, for example when serving a local cache. Signed-off-by: Christian Hesse <mail@eworm.de> --- lib/libalpm/dload.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index f6a4f012..7116cb03 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -515,13 +515,23 @@ static int curl_check_finished_download(CURLM *curlm, CURLMsg *msg, if(!payload->errors_ok) { handle->pm_errno = ALPM_ERR_RETRIEVE; - /* non-translated message is same as libcurl */ - snprintf(payload->error_buffer, sizeof(payload->error_buffer), + if(payload->respcode == 412) { + /* A server can indicate a soft failure by returning 412 + (Precondition Failed). Only a debug message is logged and + download is retried with next server, without counting + server error. */ + _alpm_log(handle, ALPM_LOG_WARNING, + _("file '%s' is currently not available from %s\n"), + payload->remote_name, hostname); + } else { + /* non-translated message is same as libcurl */ + snprintf(payload->error_buffer, sizeof(payload->error_buffer), "The requested URL returned error: %ld", payload->respcode); - _alpm_log(handle, ALPM_LOG_ERROR, + _alpm_log(handle, ALPM_LOG_ERROR, _("failed retrieving file '%s' from %s : %s\n"), payload->remote_name, hostname, payload->error_buffer); - server_soft_error(handle, payload->fileurl); + server_soft_error(handle, payload->fileurl); + } } if(curl_retry_next_server(curlm, curl, payload) == 0) { (*active_downloads_num)++;
Christian Hesse <list@eworm.de> on Thu, 2021/05/20 23:05:
From: Christian Hesse <mail@eworm.de>
A server can indicate a soft failure by returning 412 (Precondition Failed). Only a warning (instead of error) message is logged and download is retried with next server, without counting server error.
This http code can be used by servers that are not expected to be complete, for example when serving a local cache.
I am open to changes, only important is that server error count does not increase. So how about... * http code: We need something >= 400 for the code to behave correctly. Nothing fits perfectly... Do you think anything else matches better? Or should we use a proprietary code 9xx? * log message: Is wording and severity ok? * Where to document the behavior? -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
On 21-05-2021 08:52, Christian Hesse wrote:
Christian Hesse <list@eworm.de> on Thu, 2021/05/20 23:05:
From: Christian Hesse <mail@eworm.de>
A server can indicate a soft failure by returning 412 (Precondition Failed). Only a warning (instead of error) message is logged and download is retried with next server, without counting server error.
This http code can be used by servers that are not expected to be complete, for example when serving a local cache. I am open to changes, only important is that server error count does not increase. So how about...
* http code: We need something >= 400 for the code to behave correctly. Nothing fits perfectly... Do you think anything else matches better? Or should we use a proprietary code 9xx? * log message: Is wording and severity ok? * Where to document the behavior?
I would say that 412 is not the right status code to use for this, since it already has a very specific different purpose. Another thing to consider is that normal server software won't know to generate a different error code, so it will just return a 404 anyway. If you need specific support from the server software, you are probably better off with a custom header: X-Pacman-Expected-Failure or something along those lines. Then you don't have to violate the HTTP standard to get the desired behavior. Although from a user perspective it would probably be simpler to put this information in the pacman.conf than to configure their HTTP server. Kind regards, Maarten
Maarten de Vries <maarten@de-vri.es> on Fri, 2021/05/21 09:11:
On 21-05-2021 08:52, Christian Hesse wrote:
Christian Hesse <list@eworm.de> on Thu, 2021/05/20 23:05:
From: Christian Hesse <mail@eworm.de>
A server can indicate a soft failure by returning 412 (Precondition Failed). Only a warning (instead of error) message is logged and download is retried with next server, without counting server error.
This http code can be used by servers that are not expected to be complete, for example when serving a local cache. I am open to changes, only important is that server error count does not increase. So how about...
* http code: We need something >= 400 for the code to behave correctly. Nothing fits perfectly... Do you think anything else matches better? Or should we use a proprietary code 9xx? * log message: Is wording and severity ok? * Where to document the behavior?
I would say that 412 is not the right status code to use for this, since it already has a very specific different purpose.
Another thing to consider is that normal server software won't know to generate a different error code, so it will just return a 404 anyway. If you need specific support from the server software, you are probably better off with a custom header: X-Pacman-Expected-Failure or something along those lines. Then you don't have to violate the HTTP standard to get the desired behavior.
Great idea... I will have a look at this.
Although from a user perspective it would probably be simpler to put this information in the pacman.conf than to configure their HTTP server.
What do you think this should look like? To give some background: There's pacredir [0][1] which tries to re-distribute database and package files for local network. Depending on whether or not the file is available it returns a temporary redirect (307) with location or not found (currently 404). I want to update that to send another http code or - even better as you suggested - add a http header to indicate the expected failure. [0] https://archlinux.org/packages/community/x86_64/pacredir/ [1] https://github.com/eworm-de/pacredir -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
From: Christian Hesse <mail@eworm.de> By setting an extra HTTP header 'X-Pacman-Expected-Failure' the server can indicate that the failure is expected. The next server is tried without error message and without increasing the server's error count. This can be used by servers that are not expected to be complete, for example when serving a local cache. Signed-off-by: Christian Hesse <mail@eworm.de> --- lib/libalpm/dload.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index f6a4f012..e6e0a6f5 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -277,6 +277,7 @@ static size_t dload_parseheader_cb(void *ptr, size_t size, size_t nmemb, void *u const char *fptr, *endptr = NULL; const char * const cd_header = "Content-Disposition:"; const char * const fn_key = "filename="; + const char * const xpef_header = "X-Pacman-Expected-Failure:"; struct dload_payload *payload = (struct dload_payload *)user; long respcode; @@ -300,6 +301,13 @@ static size_t dload_parseheader_cb(void *ptr, size_t size, size_t nmemb, void *u } } + /* By setting an extra HTTP header 'X-Pacman-Expected-Failure' the server can + indicate that the failure is expected. The next server is tried without + error message and without increasing the server's error count. */ + if(_alpm_raw_ncmp(xpef_header, ptr, strlen(xpef_header)) == 0) { + payload->errors_ok = 1; + } + curl_easy_getinfo(payload->curl, CURLINFO_RESPONSE_CODE, &respcode); if(payload->respcode != respcode) { payload->respcode = respcode;
Christian Hesse <list@eworm.de> on Fri, 2021/05/21 10:06:
By setting an extra HTTP header 'X-Pacman-Expected-Failure' the server can indicate that the failure is expected. The next server is tried without error message and without increasing the server's error count.
This can be used by servers that are not expected to be complete, for example when serving a local cache.
Any comment on this? Where to document the behavior? BTW, this is the matching branch for pacredir: https://github.com/eworm-de/pacredir/commits/404-header -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
On 05/24/21 at 08:31pm, Christian Hesse wrote:
Christian Hesse <list@eworm.de> on Fri, 2021/05/21 10:06:
By setting an extra HTTP header 'X-Pacman-Expected-Failure' the server can indicate that the failure is expected. The next server is tried without error message and without increasing the server's error count.
This can be used by servers that are not expected to be complete, for example when serving a local cache.
Any comment on this? Where to document the behavior?
BTW, this is the matching branch for pacredir: https://github.com/eworm-de/pacredir/commits/404-header
I'm not a fan of having this be server-side. I'd rather see https://bugs.archlinux.org/task/23407 implemented.
Andrew Gregory <andrew.gregory.8@gmail.com> on Mon, 2021/05/24 11:50:
On 05/24/21 at 08:31pm, Christian Hesse wrote:
Christian Hesse <list@eworm.de> on Fri, 2021/05/21 10:06:
By setting an extra HTTP header 'X-Pacman-Expected-Failure' the server can indicate that the failure is expected. The next server is tried without error message and without increasing the server's error count.
This can be used by servers that are not expected to be complete, for example when serving a local cache.
Any comment on this? Where to document the behavior?
BTW, this is the matching branch for pacredir: https://github.com/eworm-de/pacredir/commits/404-header
I'm not a fan of having this be server-side.
Would be a perfect fit for me and my use case. :-p For those interested... I created simple flow charts to describe what happens when pacman sends its requests to pacredir. https://git.eworm.de/cgit/pacredir/about/FLOW.md In the seconds chart pacredir returns the http code 404 to pacman just before the mirror is contacted. That is where the extra header should be included.
I'd rather see https://bugs.archlinux.org/task/23407 implemented.
Could work as well. But note that pacredir does handle database files as well. So limiting 'CacheServer' to package files would restrict pacredir's functionality. -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
On 26/5/21 6:22 pm, Christian Hesse wrote:
Andrew Gregory <andrew.gregory.8@gmail.com> on Mon, 2021/05/24 11:50:
On 05/24/21 at 08:31pm, Christian Hesse wrote:
Christian Hesse <list@eworm.de> on Fri, 2021/05/21 10:06:
By setting an extra HTTP header 'X-Pacman-Expected-Failure' the server can indicate that the failure is expected. The next server is tried without error message and without increasing the server's error count.
This can be used by servers that are not expected to be complete, for example when serving a local cache.
Any comment on this? Where to document the behavior?
BTW, this is the matching branch for pacredir: https://github.com/eworm-de/pacredir/commits/404-header de/pacredir/commits/404-header
I'm not a fan of having this be server-side.
Would be a perfect fit for me and my use case. :-p
For those interested... I created simple flow charts to describe what happens when pacman sends its requests to pacredir. https://git.eworm.de/cgit/pacredir/about/FLOW.md
Thanks - that helps a bit. I'm still confused about how pacredir updates databases from a mirror. I can see how it gets a database update from another host. Can you clarify? Tbanks, Allan
Allan McRae <allan@archlinux.org> on Fri, 2021/05/28 12:39:
On 26/5/21 6:22 pm, Christian Hesse wrote:
Andrew Gregory <andrew.gregory.8@gmail.com> on Mon, 2021/05/24 11:50:
On 05/24/21 at 08:31pm, Christian Hesse wrote:
Christian Hesse <list@eworm.de> on Fri, 2021/05/21 10:06:
By setting an extra HTTP header 'X-Pacman-Expected-Failure' the server can indicate that the failure is expected. The next server is tried without error message and without increasing the server's error count.
This can be used by servers that are not expected to be complete, for example when serving a local cache.
Any comment on this? Where to document the behavior?
BTW, this is the matching branch for pacredir: https://github.com/eworm-de/pacredir/commits/404-header de/pacredir/commits/404-header
I'm not a fan of having this be server-side.
Would be a perfect fit for me and my use case. :-p
For those interested... I created simple flow charts to describe what happens when pacman sends its requests to pacredir. https://git.eworm.de/cgit/pacredir/about/FLOW.md
Thanks - that helps a bit.
I'm still confused about how pacredir updates databases from a mirror. I can see how it gets a database update from another host. Can you clarify?
Ok, let's see a more detailed example: * pacman sends a request to pacredir, the header contains a timestamp from synced database file in /var/lib/pacman/sync/: If-Modified-Since: Fri, 28 May 2021 04:38:25 GMT * pacredir sends HEAD requests to pacserve on hosts in local network * pacserve answers with 200, but the header contains a timestamp: Last-Modified: Fri, 28 May 2021 04:38:25 GMT * pacredir sends a 307 with redirect to the host with most recent db file - or 404 if all requested db files are older or of same age as local file -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
On 28/5/21 7:07 pm, Christian Hesse wrote:
Allan McRae <allan@archlinux.org> on Fri, 2021/05/28 12:39:
On 26/5/21 6:22 pm, Christian Hesse wrote:
Andrew Gregory <andrew.gregory.8@gmail.com> on Mon, 2021/05/24 11:50:
On 05/24/21 at 08:31pm, Christian Hesse wrote:
Christian Hesse <list@eworm.de> on Fri, 2021/05/21 10:06:
By setting an extra HTTP header 'X-Pacman-Expected-Failure' the server can indicate that the failure is expected. The next server is tried without error message and without increasing the server's error count.
This can be used by servers that are not expected to be complete, for example when serving a local cache.
Any comment on this? Where to document the behavior?
BTW, this is the matching branch for pacredir: https://github.com/eworm-de/pacredir/commits/404-header de/pacredir/commits/404-header
I'm not a fan of having this be server-side.
Would be a perfect fit for me and my use case. :-p
For those interested... I created simple flow charts to describe what happens when pacman sends its requests to pacredir. https://git.eworm.de/cgit/pacredir/about/FLOW.md
Thanks - that helps a bit.
I'm still confused about how pacredir updates databases from a mirror. I can see how it gets a database update from another host. Can you clarify?
Ok, let's see a more detailed example:
* pacman sends a request to pacredir, the header contains a timestamp from synced database file in /var/lib/pacman/sync/: If-Modified-Since: Fri, 28 May 2021 04:38:25 GMT * pacredir sends HEAD requests to pacserve on hosts in local network * pacserve answers with 200, but the header contains a timestamp: Last-Modified: Fri, 28 May 2021 04:38:25 GMT * pacredir sends a 307 with redirect to the host with most recent db file - or 404 if all requested db files are older or of same age as local file
OK... so pacredir never checks for the latest database availability on a mirror if there is a newer database on one of the hosts. Does this mean the "new" database on the hosts network could be long out of date, but as long as it is newer than the local machine being updated, that is what will be used? The reason I am trying to figure this out is the approach in the bug report only provides cache servers for packages, and not databases. I'm trying to understand why you would cache databases. Allan
Allan McRae <allan@archlinux.org> on Fri, 2021/05/28 20:43:
Ok, let's see a more detailed example:
* pacman sends a request to pacredir, the header contains a timestamp from synced database file in /var/lib/pacman/sync/: If-Modified-Since: Fri, 28 May 2021 04:38:25 GMT * pacredir sends HEAD requests to pacserve on hosts in local network * pacserve answers with 200, but the header contains a timestamp: Last-Modified: Fri, 28 May 2021 04:38:25 GMT * pacredir sends a 307 with redirect to the host with most recent db file - or 404 if all requested db files are older or of same age as local file
OK... so pacredir never checks for the latest database availability on a mirror if there is a newer database on one of the hosts.
Please note that pacredir does not known about mirrors at all! It tries to redirect in local network - and returns 404 if it can not. It's pacman that skips to the mirrors. That's why pacman should not consider a 404 a permanent error: The next request can result in a successful redirect.
Does this mean the "new" database on the hosts network could be long out of date, but as long as it is newer than the local machine being updated, that is what will be used?
Well, out-of-date is a term that does barely match here... pacman does known about the date of its current database files only. So yes, more recent database files are used as long as they are newer than the local ones - even if out-of-date compared with a mirror. That's why the pacredir documentation tells you to run `pacman -Sy` twice to be sure: First run fetches the newest database from local network, second run (where pacredir returns 404) fetches from mirror if a newer version is available.
The reason I am trying to figure this out is the approach in the bug report only provides cache servers for packages, and not databases. I'm trying to understand why you would cache databases.
Because database file are available on every host anyway. :) Usually it does not add any harm to transfer a database file in local network, even if you have to retransfer an newer one from mirror later. Chance are you do not have to. -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On Fri, 28 May 2021, Christian Hesse wrote:
Well, out-of-date is a term that does barely match here... pacman does known about the date of its current database files only. So yes, more recent database files are used as long as they are newer than the local ones - even if out-of-date compared with a mirror.
That's why the pacredir documentation tells you to run `pacman -Sy` twice to be sure: First run fetches the newest database from local network, second run (where pacredir returns 404) fetches from mirror if a newer version is available.
Just some side-idea: why not consider all databases which are older than x minutes (where x is configurable) "out of date"? This would rely on having systemtimes in sync, but otherwise would get rid of the cumbersome double-`pacman -Syu`. regards, Erich P.S.: I'm running something similar as you - but by far not as elaborate: two (for redundancy) nginx servers which cache packages and download databases directly. -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAmCw4XYACgkQCu7JB1Xa e1oxlA/9Gw07gaaSs14YXaScYg6bxUZtPLxU308uXzBXnPGWxbPaHdyMTzoKbUcK WK/khLbrU8akJtzcvhS6JUwgWGLhgZ9rKign2oqsHd/Vehzq0AMs4t6E0uBo+t4q IXsNfWE/R7X0KIBfrvSvZBrgZRbAqZ3C2Om0YbtxJIyQHZYzb+zxMix9GH0TvJdU hvZU/nSedSl3tL/5TloFTU7giMmUr4endXwCQkZemGvBtav45CwZNLlTy121WcRI F5qQfr8nAIj46zn4xH3xBVZa5aPBDg4S5Xp0Ey0rPlBWEDqx5rDTdo268hIMX7Fy Wq5ki7+Uaf+AVz7LD95dIExfgwajqUPpO6dctb91IPAAuxTM5afLcff84U3KtprC ugHqArk76m65HKW/ETOQxLGw+2/aexXXGFNoueh3IEXMShyX7reNNlCAA2rTRkuP 6RHXOS/BODNaJCuHzo3g2V3xcTwDztmkGJK6oVEyOZJczeBd3CwW82d0AqOY5+KF kUNvNGHa//QyQsjCRTFcp2Az5trPRmnYI+/uYyGbpll19nW2jnXZuaQbNMwg+GG9 GtA9qBdCuubMdkcW+zIIYv9AHbyiYjxO7b4eVsLSJAgluPaEzc1GVP+lTqmQP/v7 HpMTGB/53rxDHIhnPQJSZMd68RG8eSrsppDHoh+M9dJ2mdberZs= =7LPG -----END PGP SIGNATURE-----
Erich Eckner <arch@eckner.net> on Fri, 2021/05/28 14:26:
Well, out-of-date is a term that does barely match here... pacman does known about the date of its current database files only. So yes, more recent database files are used as long as they are newer than the local ones - even if out-of-date compared with a mirror.
That's why the pacredir documentation tells you to run `pacman -Sy` twice to be sure: First run fetches the newest database from local network, second run (where pacredir returns 404) fetches from mirror if a newer version is available.
Just some side-idea: why not consider all databases which are older than x minutes (where x is configurable) "out of date"? This would rely on having systemtimes in sync, but otherwise would get rid of the cumbersome double-`pacman -Syu`.
Great idea... About to add that. :)
P.S.: I'm running something similar as you - but by far not as elaborate: two (for redundancy) nginx servers which cache packages and download databases directly.
Try pacredir for an out-of-the-box solution. ;) BTW, if my solution with server side headers is accepted... In nginx configuration your location block could look something like this: location / { root /var/cache/pacman/pkg/ add_header X-Pacman-Expected-Failure true; } -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
Hi On Fri, May 28, 2021, 05:26 Erich Eckner <arch@eckner.net> wrote:
P.S.: I'm running something similar as you - but by far not as elaborate: two (for redundancy) nginx servers which cache packages and download databases directly.
FYI there are plenty of package caching solutions exist. See https://wiki.archlinux.org/title/Pacman/Tips_and_tricks#Network_shared_pacma...
On 28/5/21 10:18 pm, Christian Hesse wrote:
Allan McRae <allan@archlinux.org> on Fri, 2021/05/28 20:43:
Ok, let's see a more detailed example:
* pacman sends a request to pacredir, the header contains a timestamp from synced database file in /var/lib/pacman/sync/: If-Modified-Since: Fri, 28 May 2021 04:38:25 GMT * pacredir sends HEAD requests to pacserve on hosts in local network * pacserve answers with 200, but the header contains a timestamp: Last-Modified: Fri, 28 May 2021 04:38:25 GMT * pacredir sends a 307 with redirect to the host with most recent db file - or 404 if all requested db files are older or of same age as local file
OK... so pacredir never checks for the latest database availability on a mirror if there is a newer database on one of the hosts.
Please note that pacredir does not known about mirrors at all! It tries to redirect in local network - and returns 404 if it can not. It's pacman that skips to the mirrors.
I was using "mirror" to distinguish external servers from hosts on the local network.
That's why pacman should not consider a 404 a permanent error: The next request can result in a successful redirect.
Does this mean the "new" database on the hosts network could be long out of date, but as long as it is newer than the local machine being updated, that is what will be used?
Well, out-of-date is a term that does barely match here... pacman does known about the date of its current database files only. So yes, more recent database files are used as long as they are newer than the local ones - even if out-of-date compared with a mirror.
That's why the pacredir documentation tells you to run `pacman -Sy` twice to be sure: First run fetches the newest database from local network, second run (where pacredir returns 404) fetches from mirror if a newer version is available.
This does very little to convince me that the CacheServer proposal should be used for databases. A
Allan McRae <allan@archlinux.org> on Fri, 2021/05/28 22:52:
Does this mean the "new" database on the hosts network could be long out of date, but as long as it is newer than the local machine being updated, that is what will be used?
Well, out-of-date is a term that does barely match here... pacman does known about the date of its current database files only. So yes, more recent database files are used as long as they are newer than the local ones - even if out-of-date compared with a mirror.
That's why the pacredir documentation tells you to run `pacman -Sy` twice to be sure: First run fetches the newest database from local network, second run (where pacredir returns 404) fetches from mirror if a newer version is available.
This does very little to convince me that the CacheServer proposal should be used for databases.
So let's implement CacheServer for package files only as described in FS#23407. Apply my patch anyway to make pacman happy with pacredir. :) -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
On 05/28/21 at 05:55pm, Christian Hesse wrote:
Allan McRae <allan@archlinux.org> on Fri, 2021/05/28 22:52:
Does this mean the "new" database on the hosts network could be long out of date, but as long as it is newer than the local machine being updated, that is what will be used?
Well, out-of-date is a term that does barely match here... pacman does known about the date of its current database files only. So yes, more recent database files are used as long as they are newer than the local ones - even if out-of-date compared with a mirror.
That's why the pacredir documentation tells you to run `pacman -Sy` twice to be sure: First run fetches the newest database from local network, second run (where pacredir returns 404) fetches from mirror if a newer version is available.
This does very little to convince me that the CacheServer proposal should be used for databases.
So let's implement CacheServer for package files only as described in FS#23407.
Apply my patch anyway to make pacman happy with pacredir. :)
-1. This sounds like exactly the kind of esoteric setup that we keep the external downloader around for.
Christian Hesse <list@eworm.de> on Fri, 2021/05/28 17:55:
Allan McRae <allan@archlinux.org> on Fri, 2021/05/28 22:52:
Does this mean the "new" database on the hosts network could be long out of date, but as long as it is newer than the local machine being updated, that is what will be used?
Well, out-of-date is a term that does barely match here... pacman does known about the date of its current database files only. So yes, more recent database files are used as long as they are newer than the local ones - even if out-of-date compared with a mirror.
That's why the pacredir documentation tells you to run `pacman -Sy` twice to be sure: First run fetches the newest database from local network, second run (where pacredir returns 404) fetches from mirror if a newer version is available.
This does very little to convince me that the CacheServer proposal should be used for databases.
So let's implement CacheServer for package files only as described in FS#23407.
Apply my patch anyway to make pacman happy with pacredir. :)
Now that pacman 6.0.0 is in [core] we should have an idea of a solution at least. Given that a number of caching solutions exist [0] having the header solution around may be not the worst idea. So does this have a chance to be committed? Still the CacheServer directive for package files would not work with database files. Thus it would not be a real solution for use with pacredir. So would you accept a patch to disable the server error limit from configuration and command line switch? That would kind of resolve the issue but result in error messages still being thrown for me, however... Had hoped to get rid of that. [0] https://wiki.archlinux.org/title/Pacman/Tips_and_tricks#Network_shared_pacma... -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}
On 2/6/21 12:28 am, Christian Hesse wrote:
Christian Hesse <list@eworm.de> on Fri, 2021/05/28 17:55:
Allan McRae <allan@archlinux.org> on Fri, 2021/05/28 22:52:
Does this mean the "new" database on the hosts network could be long out of date, but as long as it is newer than the local machine being updated, that is what will be used?
Well, out-of-date is a term that does barely match here... pacman does known about the date of its current database files only. So yes, more recent database files are used as long as they are newer than the local ones - even if out-of-date compared with a mirror.
That's why the pacredir documentation tells you to run `pacman -Sy` twice to be sure: First run fetches the newest database from local network, second run (where pacredir returns 404) fetches from mirror if a newer version is available.
This does very little to convince me that the CacheServer proposal should be used for databases.
So let's implement CacheServer for package files only as described in FS#23407.
Apply my patch anyway to make pacman happy with pacredir. :)
Now that pacman 6.0.0 is in [core] we should have an idea of a solution at least. Given that a number of caching solutions exist [0] having the header solution around may be not the worst idea. So does this have a chance to be committed?
Andrew already gave it a -1. That means there is zero chance of it being committed.
Still the CacheServer directive for package files would not work with database files. Thus it would not be a real solution for use with pacredir. So would you accept a patch to disable the server error limit from configuration and command line switch? That would kind of resolve the issue but result in error messages still being thrown for me, however... Had hoped to get rid of that.
Going through other caching solutions, CacheServer ignoring missing packages would solve most issues. So I'll implement that when I have a spare 30 minutes. Once that is done pacredir can probably do "pacman -Sy; pacman -Sy; pacman -Su" as a workaround until we handle dbs too. Allan
participants (6)
-
Allan McRae
-
Anatol Pomozov
-
Andrew Gregory
-
Christian Hesse
-
Erich Eckner
-
Maarten de Vries