[aur-general] Problem downloading a source package with new curl version

Mon Sep 20 20:16:11 UTC 2021

On 2021-09-20 21:50, Jan Kohnert via aur-general wrote:
> Hi,
>
> one of my aur packages (eccodes) has a problem getting the source package
> using the new version of curl (7.79.0).
>
> The output of the old curl version looks like:
> -----------------------------------------------------------------------------
> jankoh at kohni-mobil ~/projects/eccodes $ curl -v 
> https://confluence.ecmwf.int/
> download/attachments/45757960/eccodes-2.23.0-Source.tar.gz?api=v2
> * Trying 136.156.180.232:443...
> * Connected to confluence.ecmwf.int (136.156.180.232) port 443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/certs/ca-certificates.crt
> * CApath: none
> * TLSv1.3 (OUT), TLS handshake, Client hello (1):
> * TLSv1.3 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
> * ALPN, server did not agree to a protocol
> * Server certificate:
> * subject: jurisdictionC=GB; businessCategory=Non-Commercial Entity;
> serialNumber=ISBN: 9780101820028; C=GB; ST=West Berkshire; L=Reading;
> O=European Cen
> tre for Medium-Range Weather Forecasts; CN=confluence.ecmwf.int
> * start date: Mar 29 16:16:45 2021 GMT
> * expire date: Mar 29 16:26:00 2022 GMT
> * subjectAltName: host "confluence.ecmwf.int" matched cert's
> "confluence.ecmwf.int"
> * issuer: C=NL; O=QuoVadis Trustlink B.V.; CN=QuoVadis Europe EV SSL CA G1
> * SSL certificate verify ok.
>> GET 
>> /download/attachments/45757960/eccodes-2.23.0-Source.tar.gz?api=v2 HTTP/
> 1.1
>> Host: confluence.ecmwf.int
>> User-Agent: curl/7.78.0
>> Accept: */*
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 200 200
> < Date: Sun, 19 Sep 2021 19:48:11 GMT
> < Server: Apache
> < Cache-Control: no-cache, must-revalidate
> < Expires: Thu, 01 Jan 1970 00:00:00 GMT
> < X-Confluence-Request-Time: 1632080871142
> < Set-Cookie: JSESSIONID=659BDBFF1F03F2DC7848A7BEECF3CCC3; Path=/; 
> HttpOnly
> < Last-Modified: Mon, 19 Jan 1970 20:46:07 GMT
> < Accept-Ranges: bytes
> < X-Content-Type-Options: nosniff
> < Content-Disposition: inline; filename="eccodes-2.23.0-Source.tar.gz"
> < Content-Type: application/x-gzip;charset=UTF-8
> < Content-Length: 12037258
> < Strict-Transport-Security: max-age=15552000
> <
> Warning: Binary output can mess up your terminal. Use "--output -" to tell
> Warning: curl to output it to your terminal anyway, or consider "--output
> Warning: <FILE>" to save to a file.
> * Failure writing output to destination
> * Closing connection 0
> * TLSv1.2 (OUT), TLS alert, close notify (256):
> jankoh at kohni-mobil ~/projects/eccodes $
> -----------------------------------------------------------------------------
>
> The new version outputs:
> -----------------------------------------------------------------------------
> jankoh at kohni-mobil ~/projects/eccodes $ curl -v -i https://
> confluence.ecmwf.int/download/attachments/45757960/eccodes-2.23.0-
> Source.tar.gz?api=v2
> * Trying 136.156.180.232:443...
> * Connected to confluence.ecmwf.int (136.156.180.232) port 443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * successfully set certificate verify locations:
> * CAfile: /etc/ssl/certs/ca-certificates.crt
> * CApath: none
> * TLSv1.3 (OUT), TLS handshake, Client hello (1):
> * TLSv1.3 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
> * ALPN, server did not agree to a protocol
> * Server certificate:
> * subject: jurisdictionC=GB; businessCategory=Non-Commercial Entity;
> serialNumber=ISBN: 9780101820028; C=GB; ST=West Berkshire; L=Reading;
> O=European Centre for Medium-Range Weather Forecasts; 
> CN=confluence.ecmwf.int
> * start date: Mar 29 16:16:45 2021 GMT
> * expire date: Mar 29 16:26:00 2022 GMT
> * subjectAltName: host "confluence.ecmwf.int" matched cert's
> "confluence.ecmwf.int"
> * issuer: C=NL; O=QuoVadis Trustlink B.V.; CN=QuoVadis Europe EV SSL CA G1
> * SSL certificate verify ok.
>> GET 
>> /download/attachments/45757960/eccodes-2.23.0-Source.tar.gz?api=v2 HTTP/
> 1.1
>> Host: confluence.ecmwf.int
>> User-Agent: curl/7.79.0
>> Accept: */*
>>
> * Unsupported response code in HTTP response
> * Closing connection 0
> * TLSv1.2 (OUT), TLS alert, close notify (256):
> curl: (1) Unsupported response code in HTTP response
> jankoh at kohni-mobil ~/projects/eccodes $
> -----------------------------------------------------------------------------
>
> I see the response code is 200 200, which has an additional errournous 
> 200.
> Since the upgrade my automated daily test builds fail, so I'd like to 
> know if
> I should file a bug report either upstream or the arch bugtracker.
>
> I can download the file using chrome or wget, so it might be a 
> workaroud if I
> could tell makepkg to use wget instead of curl.
>
> However, executing
> -----------------------------------------------------------------------------
> sudo sed -i -r "s/(http[s]?::)\/usr\/bin\/curl\ -qgb\ \"\"\ -fLC\ -\ 
> --retry\
> 3\ --retry-delay\ 3 -o/\1\/usr\/sbin\/wget\ --no-cookies\ 
> --retry-connrefused\
> -t\ 3\ --waitretry\ 3 -O/" /etc/makepkg.conf
> -----------------------------------------------------------------------------
> inside the build container will not help users that upgrade or install the
> package. I could post the workaround to the packages page, but this 
> will also
> affect other packages to be build, and wget needs to be installed 
> before that
> workaround will actually work.
>
> Any tips on how to go on?
>
> TIA
>

This is likely due to a recent change in curl, being stricter about 
response codes.

 > HTTP 1.1: disallow >3-digit response codes
 > The HTTP protocol is defined to only allow three-digit numbers and 
now curl enforces that check stricter. This was in part made to align 
behavior when curl is built to use hyper.

 > ref 
https://daniel.haxx.se/blog/2021/09/15/curl-7-79-0-secure-local-cookies/

So the arch bugtracker is probably the appropriate one.

- v