[arch-devops] fail2ban deployed for the wiki

Jelle van der Waa jelle at vdwaa.nl
Sun Sep 1 18:35:34 UTC 2019

Hi All,

Today and earlier we suffered a huge influx of bots from china crawling
the wiki. We have taken a few measures to reduce the impact:

- fastcgi_cache for /load.php which caches the load.php page for 10
  minutes which contains assets such as css/js. This should offload
  php-fpm which was overloaded.
- Enabled a plugin to disable viewing wiki page revisions for anonymous
  users this reduces some heavy requests and hopefully the impact of

The last step was enabling fail2ban for HTTP/HTTPS requests, only for
the wiki now. It blocks every ip doing more then 300 requests in 30
minutes. This might be a bit too aggressive but for now it dropped our
load from ~ 20 -> ~ 2/3 and blocks 85 ips. This can be tweaked later,
maybe it should be 400/500?

To view the blocked ips execute:

fail2ban-client status wiki-nginx-dos

To unban a valid IP:

fail2ban-client unban $ip

fail2ban does use a lot of CPU which we should look into tuning, but
maybe it will get better over time when the log files are smaller due to
less bots coming through :-)

The fail2ban role is in ansible, but not suited yet to be re-used on
other hosts.



More information about the arch-devops mailing list