[aur-general] New AUR Metadata Archives

[Kevin Morris]

As per the previous archive spec in the project (PHP), we've
intended to persist the `Content-Type: text/plain` +
`Content-Encoding: gzip` headers with our new archives, as should
be supported with all archives found in the AUR.

This wasn't being handled correctly by aur.al's nginx frontend
and a patch has been merged in which resolves this issue you
were seeing (application/gzip encoded with gzip).

That being said, we do want to handle mimes better for these
archives, especially since we've got these new .json.gz archives;
supplying both encoded gzip transports as well as raw application/gzip
transports.

It won't happen immediately, but here's an issue I've just put
up in regards to this:
https://gitlab.archlinux.org/archlinux/aurweb/-/issues/175

Thanks for the heads up; didn't realize this was mismatched on live.

Regards,
Kevin

On Fri, Nov 12, 2021 at 12:16:21AM +0300, Dmitry Marakasov wrote:
> * Kevin Morris via aur-general (aur-general at lists.archlinux.org) wrote:
> 
> > The files are downloadable in gzip format; when you receive
> > the file, you'll need to uncompress it.
> 
> There might in fact be an issue here, as I've ran into it. The file
> might be unexpectedly returned uncompressed by some clients, for
> instance python requests:
> 
> % python3 -c 'import requests; print(requests.get("https://aur.archlinux.org/packages-meta-ext-v1.json.gz").content[:50])'
> b'[\n{"ID":208446,"Name":"bubblemon","PackageBaseID":'
> 
> If I'm not mistaken it's caused by excessive `content-encoding: gzip`
> header:
> 
> % curl -I https://aur.archlinux.org/packages-meta-ext-v1.json.gz
> HTTP/2 200
> server: nginx
> date: Thu, 11 Nov 2021 21:05:03 GMT
> content-type: application/gzip
> content-length: 7410251
> last-modified: Thu, 11 Nov 2021 21:05:03 GMT
> etag: "618d857f-71124b"
> expires: Thu, 11 Nov 2021 21:10:03 GMT
> cache-control: max-age=300
> content-encoding: gzip
> accept-ranges: bytes
> 
> which basically says that the transferred gzip file is additionally
> encoded with gzip (e.g. doubly compressed, which is probably not
> the case), so client uncompresses it upon retrieval, in fact
> uncompressing the original gzip and returning plain json.
> 
> -- 
> Dmitry Marakasov   .   55B5 0596 FF1E 8D84 5F56  9510 D35A 80DD F9D2 F77D
> amdmi3 at amdmi3.ru  ..:     https://github.com/AMDmi3  https://amdmi3.ru
> 

-- 
Kevin Morris
Software Developer

Identities:
 - kevr @ Libera
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.archlinux.org/pipermail/aur-general/attachments/20211111/5a57e3fe/attachment.sig>