[aur-general] AUR package metadata dump
bts at square-r00t.net
Thu Nov 15 19:50:48 UTC 2018
On 11/15/18 14:26, Eli Schwartz via aur-general wrote:
> On 11/15/18 1:25 PM, Dmitry Marakasov wrote:
>> While fetching data from API, Repology does a 1 second pause between
>> requests to not create excess load on the server, but there are still
>> frequent 429 errors. I've tried 2 second delays, but the 429s are still
>> there, and fetch time increases dramatically as we have to do more than
>> 500 requests. Probably API is loaded by other clients as well.
> Our rate limit is 4000 per 24 hours. One-second pauses aren't taken into
> account, and our initial motivation to add rate limiting was to ban
> users who were using 5-second delays...
don't forget about the URI max length, too. staggering into requests of
100 pkgs would work fine, but worth noting the max length is 4443 bytes
GPG info: https://square-r00t.net/gpg-info
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 899 bytes
Desc: OpenPGP digital signature
More information about the aur-general