[aur-general] Search-URL in PKGBUILD
Hi, I'm using the default DLAGENTS settings in makepkg.conf [1], curl is from the core repo and up to date [2]. Search-URLs never do work on my machine, see https://aur.archlinux.org/packages/perl-cpanel-json-xs/#comment-749091 What is the recommended solution to solve this issue? Regards, Ralf [1] $ grep -ehttp: /etc/makepkg.conf{,.pacnew} /etc/makepkg.conf: 'http::/usr/bin/curl -gqb "" -fLC - --retry 3 --retry-delay 3 -o %o %u' /etc/makepkg.conf.pacnew: 'http::/usr/bin/curl -gqb "" -fLC - --retry 3 --retry-delay 3 -o %o %u' [2] $ pacman -Q curl curl 7.70.0-1
On 6/3/20 8:06 AM, Ralf Mardorf wrote:
Hi,
I'm using the default DLAGENTS settings in makepkg.conf [1], curl is from the core repo and up to date [2].
Search-URLs never do work on my machine, see
https://aur.archlinux.org/packages/perl-cpanel-json-xs/#comment-749091
What is the recommended solution to solve this issue? What is a "search-URL"?
-- Eli Schwartz Bug Wrangler and Trusted User
On Wed, 3 Jun 2020 08:11:32 -0400, Eli Schwartz via aur-general wrote:
On 6/3/20 8:06 AM, Ralf Mardorf wrote:
https://aur.archlinux.org/packages/perl-cpanel-json-xs/#comment-749091
What is a "search-URL"?
I don't know the correct term. source=('http://search.cpan.org/CPAN/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-4.19.tar.g...') ^^^^^^ ^^^^^^ It seemingly should redirect, but I get a HTTP 403 forbidden. I always need to edit this and other packages with such URLs. source=("https://cpan.metacpan.org/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-${pkgver}.tar.gz")
On 6/3/20 3:26 PM, Ralf Mardorf wrote:
On Wed, 3 Jun 2020 08:11:32 -0400, Eli Schwartz via aur-general wrote:
On 6/3/20 8:06 AM, Ralf Mardorf wrote:
https://aur.archlinux.org/packages/perl-cpanel-json-xs/#comment-749091
What is a "search-URL"?
I don't know the correct term.
source=('http://search.cpan.org/CPAN/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-4.19.tar.g...') ^^^^^^ ^^^^^^
It seemingly should redirect, but I get a HTTP 403 forbidden. I always need to edit this and other packages with such URLs.
source=("https://cpan.metacpan.org/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-${pkgver}.tar.gz")
Are you always trying from the same computer and internet connection? Could it be you are getting some incorrect DNS responses or maybe your ISP doesn't like you connecting to a certain website in a certain way? Indeed, it is recommended to always use HTTPS in source=() array when possible, however, in your case I think it's something else, given the package builds fine from my side. For your reference, here's the DNS records I see: $ nslookup search.cpan.org Server: ::1 Address: ::1#53 Non-authoritative answer: search.cpan.org canonical name = dualstack.osff.map.fastly.net. Name: dualstack.osff.map.fastly.net Address: 151.101.2.217 Name: dualstack.osff.map.fastly.net Address: 151.101.66.217 Name: dualstack.osff.map.fastly.net Address: 151.101.130.217 Name: dualstack.osff.map.fastly.net Address: 151.101.194.217 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:200::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:400::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:600::729 $ nslookup cpan.metacpan.org Server: ::1 Address: ::1#53 Non-authoritative answer: cpan.metacpan.org canonical name = dualstack.osff.map.fastly.net. Name: dualstack.osff.map.fastly.net Address: 151.101.38.217 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:600::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:400::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:200::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42::729 If you see the same or similar try a different computer or try traceroute and see what happens. I doubt it's cURL's config, given you said Chrome and Firefox fail to access it as well. Also, if this persists, open a forum post and/or try to politely convince the maintainer to update to HTTPS citing the PKGBUILD guidelines. -- Regards, Konstantin
On Wed, 2020-06-03 at 15:35 +0300, Konstantin Gizdov wrote:
For your reference, here's the DNS records I see: $ nslookup search.cpan.org Server: ::1 Address: ::1#53
Non-authoritative answer: search.cpan.org canonical name = dualstack.osff.map.fastly.net. Name: dualstack.osff.map.fastly.net Address: 151.101.2.217 Name: dualstack.osff.map.fastly.net Address: 151.101.66.217 Name: dualstack.osff.map.fastly.net Address: 151.101.130.217 Name: dualstack.osff.map.fastly.net Address: 151.101.194.217 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:200::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:400::729 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:600::729
$ nslookup search.cpan.org Server: 192.168.1.1 Address: 192.168.1.1#53 Non-authoritative answer: search.cpan.org canonical name = dualstack.osff.map.fastly.net. Name: dualstack.osff.map.fastly.net Address: 151.101.14.217 Name: dualstack.osff.map.fastly.net Address: 2a04:4e42:3::729 On my machine resolve.conf isn't a link. It's a file, so the Internet is accessible when running one or another or the Arch install in a systemd-nspawn container, even without the boot option. Maybe it's related to this, let alone that... # resolvectl status Failed to get global data: Unit dbus-org.freedesktop.resolve1.service not found. # systemctl status systemd-resolved ● systemd-resolved.service - Network Name Resolution Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: man:systemd-resolved.service(8) https://www.freedesktop.org/wiki/Software/systemd/resolved https://www.freedesktop.org/wiki/Software/systemd/writing-network-configurat... https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients # systemctl start systemd-resolved # systemctl status systemd-resolved ● systemd-resolved.service - Network Name Resolution Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; disabled; vendor preset: enabled) Active: active (running) since Wed 2020-06-03 15:19:54 CEST; 5s ago Docs: man:systemd-resolved.service(8) https://www.freedesktop.org/wiki/Software/systemd/resolved https://www.freedesktop.org/wiki/Software/systemd/writing-network-configurat... https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients Main PID: 428640 (systemd-resolve) Status: "Processing requests..." Tasks: 1 (limit: 9399) Memory: 8.3M CGroup: /system.slice/systemd-resolved.service └─428640 /usr/lib/systemd/systemd-resolved Jun 03 15:19:53 archlinux systemd[1]: Starting Network Name Resolution... Jun 03 15:19:54 archlinux systemd-resolved[428640]: Positive Trust Anchors: Jun 03 15:19:54 archlinux systemd-resolved[428640]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d Jun 03 15:19:54 archlinux systemd-resolved[428640]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arp> Jun 03 15:19:54 archlinux systemd-resolved[428640]: Using system hostname 'archlinux'. Jun 03 15:19:54 archlinux systemd[1]: Started Network Name Resolution. # resolvectl status Global LLMNR setting: yes MulticastDNS setting: yes DNSOverTLS setting: no DNSSEC setting: allow-downgrade DNSSEC supported: yes Current DNS Server: 192.168.1.1 DNS Servers: 192.168.1.1 Fallback DNS Servers: 1.1.1.1 9.9.9.10 8.8.8.8 2606:4700:4700::1111 2620:fe::10 2001:4860:4860::8888 DNS Domain: localdomain DNSSEC NTA: 10.in-addr.arpa 16.172.in-addr.arpa 168.192.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa corp d.f.ip6.arpa home internal intranet lan local private test Link 2 (enp3s0) Current Scopes: LLMNR/IPv4 LLMNR/IPv6 DefaultRoute setting: no LLMNR setting: yes MulticastDNS setting: no DNSOverTLS setting: no DNSSEC setting: allow-downgrade DNSSEC supported: yes lines 3-51/51 (END) It doesn't solve the issue, but in the meantime the maintainer fixed the PKGBUILD. There still might be few PKGBUILDs using such a "search-"URL.
Is it a common issue for many or somehow cause by your network environment? I just tested the URL and was able to download the file without any problem. If it's just you, personally I'd suggest investigating your network or just writing a cURL wrapper to set an http_proxy for such URLs; alternatively you can have a look at projects like customizepkg [1] to automate the process of updating PKGBUILDs. If you believe your network is not at fault, I'd suggest you reach out to the Perl people to check e.g. if your IP is blocked by mistake. On 3/6/20 10:26 pm, Ralf Mardorf wrote:
On Wed, 3 Jun 2020 08:11:32 -0400, Eli Schwartz via aur-general wrote:
On 6/3/20 8:06 AM, Ralf Mardorf wrote:
https://aur.archlinux.org/packages/perl-cpanel-json-xs/#comment-749091
What is a "search-URL"?
I don't know the correct term.
source=('http://search.cpan.org/CPAN/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-4.19.tar.g...') ^^^^^^ ^^^^^^
It seemingly should redirect, but I get a HTTP 403 forbidden. I always need to edit this and other packages with such URLs.
source=("https://cpan.metacpan.org/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-${pkgver}.tar.gz")
-- Best regards, Frederick Zhang Email: frederick888@tsundere.moe PGP: 8BFB EA5B 4C44 BFAC C8EC 5F93 1F92 8BE6 0D8B C11D
Forgot to put down the URL of customizepkg [1]. Sorry. [1] https://github.com/ava1ar/customizepkg On 3/6/20 10:40 pm, Frederick Zhang wrote:
Is it a common issue for many or somehow cause by your network environment? I just tested the URL and was able to download the file without any problem.
If it's just you, personally I'd suggest investigating your network or just writing a cURL wrapper to set an http_proxy for such URLs; alternatively you can have a look at projects like customizepkg [1] to automate the process of updating PKGBUILDs.
If you believe your network is not at fault, I'd suggest you reach out to the Perl people to check e.g. if your IP is blocked by mistake.
On 3/6/20 10:26 pm, Ralf Mardorf wrote:
On Wed, 3 Jun 2020 08:11:32 -0400, Eli Schwartz via aur-general wrote:
On 6/3/20 8:06 AM, Ralf Mardorf wrote:
https://aur.archlinux.org/packages/perl-cpanel-json-xs/#comment-749091
What is a "search-URL"?
I don't know the correct term.
source=('http://search.cpan.org/CPAN/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-4.19.tar.g...') ^^^^^^ ^^^^^^
It seemingly should redirect, but I get a HTTP 403 forbidden. I always need to edit this and other packages with such URLs.
source=("https://cpan.metacpan.org/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-${pkgver}.tar.gz")
-- Best regards, Frederick Zhang Email: frederick888@tsundere.moe PGP: 8BFB EA5B 4C44 BFAC C8EC 5F93 1F92 8BE6 0D8B C11D
On 6/3/20 8:26 AM, Ralf Mardorf wrote:
On Wed, 3 Jun 2020 08:11:32 -0400, Eli Schwartz via aur-general wrote:
On 6/3/20 8:06 AM, Ralf Mardorf wrote:
https://aur.archlinux.org/packages/perl-cpanel-json-xs/#comment-749091
What is a "search-URL"?
I don't know the correct term.
source=('http://search.cpan.org/CPAN/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-4.19.tar.g...') ^^^^^^ ^^^^^^
So a "search-URL" is a perl concept. That's something I did not know, because I'm not a perl person. To translate this into concepts that don't depend on an understanding of perl... This package downloads source code from the search.cpan.org website.
It seemingly should redirect, but I get a HTTP 403 forbidden. I always need to edit this and other packages with such URLs.
source=("https://cpan.metacpan.org/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-${pkgver}.tar.gz")
... and apparently at least you are getting 403 forbidden errors. Which is interesting, because at least I'm getting 301 "moved permanently" redirects, to the cpan.metacpan.org website. My conclusion is: a) speak to the operators of this website and find out why you're being "forbidden" b) a PKGBUILD should use the second url, because "moved permanently" means people should not be using the old, legacy site; also, it saves a tiny bit of bandwidth and performs fewer network connections to use the authoritative site directly instead of first querying another site and being told "no really, you should be looking there instead" As for the package, the AUR maintainer updated it to use the cpan.metacpan.org website, and threw in an https for free. (Note that both search.cpan.org and cpan.metacpan.org support both http:// and https:// and, for me, are accessible over either one.) -- Eli Schwartz Bug Wrangler and Trusted User
participants (4)
-
Eli Schwartz
-
Frederick Zhang
-
Konstantin Gizdov
-
Ralf Mardorf