On 11/15/18 14:26, Eli Schwartz via aur-general wrote:
On 11/15/18 1:25 PM, Dmitry Marakasov wrote:
Hi!
(SNIP)
While fetching data from API, Repology does a 1 second pause between requests to not create excess load on the server, but there are still frequent 429 errors. I've tried 2 second delays, but the 429s are still there, and fetch time increases dramatically as we have to do more than 500 requests. Probably API is loaded by other clients as well.
Our rate limit is 4000 per 24 hours. One-second pauses aren't taken into account, and our initial motivation to add rate limiting was to ban users who were using 5-second delays...
(SNIP) don't forget about the URI max length, too. staggering into requests of 100 pkgs would work fine, but worth noting the max length is 4443 bytes https://wiki.archlinux.org/index.php/Aurweb_RPC_interface#Limitations -- brent saner https://square-r00t.net/ GPG info: https://square-r00t.net/gpg-info