On 26/9/22 21:54, Christian Hesse wrote:

Allan McRae on Mon, 2022/09/26 20:44:

...

On 19/9/22 22:52, Christian Hesse wrote:

...

From: Christian Hesse

By setting the HTTP header 'Cache-Control: no-cache' when returning with the status code 404 (not found) the server can indicate that this is a soft failure. No error message is shown, and server's error count is not increased.

This can be used by servers that are not expected to be complete, for example when serving a local cache [0]. In nginx this can be achived by adding a single directive in location block:

add_header Cache-Control "no-cache";

I am not accepting this patch. 1) "no-cache" to indicate this a cache of packages...

You missed the point. This does not tell not to cache packages. It does tell not to cache an error. ;)

Fun fact: Searching for the keywords yields a lot of matches for some well known cloud providers and their proxy software. Looks like the same is used there not to cache errors.

This is the sort of information that would make considering this as an option more likely. Can you expand on this with some examples? Allan

On 09/26/22 at 04:24pm, Christian Hesse wrote:

Allan McRae on Mon, 2022/09/26 22:35:

...

On 26/9/22 21:54, Christian Hesse wrote:

...

Allan McRae on Mon, 2022/09/26 20:44:

...

On 19/9/22 22:52, Christian Hesse wrote:

...

From: Christian Hesse

By setting the HTTP header 'Cache-Control: no-cache' when returning with the status code 404 (not found) the server can indicate that this is a soft failure. No error message is shown, and server's error count is not increased.

This can be used by servers that are not expected to be complete, for example when serving a local cache [0]. In nginx this can be achived by adding a single directive in location block:

add_header Cache-Control "no-cache";

I am not accepting this patch. 1) "no-cache" to indicate this a cache of packages...

You missed the point. This does not tell not to cache packages. It does tell not to cache an error. ;)

Fun fact: Searching for the keywords yields a lot of matches for some well known cloud providers and their proxy software. Looks like the same is used there not to cache errors.

This is the sort of information that would make considering this as an option more likely. Can you expand on this with some examples?

Oh, did not expect that to decide the thing.

Here is a selection of links on the topic...

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/HTTPStatu... https://cloud.google.com/cdn/docs/using-negative-caching https://community.cloudflare.com/t/how-to-avoid-caching-404/196262 https://community.cloudflare.com/t/cache-everything-and-dont-cache-404-at-al...

For Cloudflare I found the forum discussions only, not sure if the relevant parts have been removed from documentation... There must have been more. :)

Default caching on 404 happens for something between several seconds and minutes by default. This can be changed to different time (CacheControl: age=...) or disabled (CacheControl: no-cache).

None of these links looks to me like what you want pacman to do with the no-cache header. The caching is still all per-object. Getting a 404 with no-cache set means to retry the request for that same object, not to make inferences about whether requests for other objects will 404. pacman isn't caching anything in the sense that Cache-Control was meant to affect, so I don't see how the Cache-Control header has any relevance here. apg

On 10/25/22 at 04:55pm, Christian Hesse wrote:

Andrew Gregory on Mon, 2022/10/10 19:55:

...

None of these links looks to me like what you want pacman to do with the no-cache header. The caching is still all per-object. Getting a 404 with no-cache set means to retry the request for that same object, not to make inferences about whether requests for other objects will 404.

pacman isn't caching anything in the sense that Cache-Control was meant to affect, so I don't see how the Cache-Control header has any relevance here.

I have to agree with what you say.

But I can argue the other way round: A 404 response is for a specific object, not the hole server. The server error limit excludes servers for some missed objects, and does not try others - though it did not check the server for that same object. Is that any better?

I can understand to skip a server for hard errors. For example I do not want to run into the same timeout again and again, or resolve a non-existent host again and again.

For soft errors (where the server exists and answers) this should be different. Well, perhaps the current behavior is even fine for default behavior and the majority of people, but there should by a way to opt out.

In an older thread someone denied to add an option to disable the server error limit. How about a configuration options that disables the server error limit for soft errors only (and keep the behavior for hard errors as is)? Something like `NoServerSoftErrorLimit` (and possibly `--noserversofterrorlimit`) that makes cache servers work?

Soft errors are not resource hungry in any way. This es even more true with parallel downloads where no time is wasted for the request when a download is still running.

A 404 may not necessarily mean that the server is missing other files as well, but it does indicate that the server is not fully synced, which is generally the expectation. There are countless varieties of unusual setups and it has been our policy of late to not support the many options required by them in the internal downloader. Users with specialized needs can use the external downloader, which lacks any special error handling, or a proxy server that conforms to pacman's expectations. As I understand pacredir, users should still be able to use it fine with the external downloader right now, or pacredir could be adjusted to redirect to an appropriate full mirror instead of returning 404. The only real advantage I see to changing pacman is to allow more intelligent behavior for different types of download (e.g. you should probably not pull the databases from the cache). The simpler ignore-404s or cache-headers options do not provide that ability. apg

kpcyrd on Mon, 2022/09/26 17:47:

On 9/19/22 14:52, Christian Hesse wrote:

...

In nginx this can be achived by adding a single directive in location block:

add_header Cache-Control "no-cache";

If it's added directly to the location block this would disable caching for all responses.

Yes, true... Though pacman ignores this. :) And I found another issue: By default nginx does not add headers when return code is 404, but it can be forced with extra parameter "always" [0]. So we should show something like this in the commit message for a complete server block: server { server name _default; listen 8080; root /var/cache/pacman/pkg; error_page 404 = @no-cache; location @no-cache { add_header CacheControl "no-cache" always; } } Allan, if you decide to accept the patch I would like to re-send with an updated commit message. [0] https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}