On Mon, Jan 26, 2015 at 08:29:51AM -0500, Konstantin Ryabitsev wrote:

On 25/01/15 11:53 PM, Dave Reisner wrote:

...

Would it be possible to turn off chunked transfer so that nginx serves a Content-Length header? This is highly preferrable -- the overhead in calculating the response size is that of a simple stat syscall. In addition, knowing the response body size up front potentially allows downloaders to match the remote file size against local metadata, as a method of detecting corrupted or tampered-with files.

Thanks for the suggestion -- I turned it off. It doesn't make sense to have it on a static-only site.

Great, thanks!

...

Also, I offhandedly highlight that your cache varies on querystring. Do you really need to do this for static content? This actually works against you in a the case of a DoS attack -- a malicious user could potentially evict a large amount of the cache by flooding it with variations on a single large blob. If mirrors.kernel.org shares a cache with other sites, it might be a Bad Thing™. Actually, if the Varnish instance used for mirrors.kernel.org is shared with other subdomains, you might consider disabling it entirely for files below mirrors.kernel.org. Relying on the kernel's page cache alone seems like a better strategy.

Using varnish is a temporary but, unfortunately, necessary measure as we work with upstream to fix FS corruption problems we're seeing with dm-cache, libvirt and xfs.

https://plus.google.com/+KonstantinRyabitsev/posts/6YRFhcKKipP

Varnish+ssd is helping us last things out until the FS corruption is fixed.

Understood. Thanks again for the quick response! Cheers, dR