Hey! Looking at logs from my mirror (https://arch.jensgutermuth.de/) reveals at least Google and AHrefs are crawling my mirror, which is obviously a waste of resources for both sides. I'm thinking about blocking them (and all other crawlers) using a robots.txt file like so (nginx config snippet): location= /robots.txt { return 200 "User-agent: *\nDisallow: /\n"; allow all; access_log off; } Doing it this way prevents robots.txt from showing up in directory listings and circumvents all issues with the sync script. I know modifying mirror contents is a very touchy subject and rightfully so. I therefore wanted to ask if there is some kind of policy and if there is, if this would be allowed or a possible exception. Best regards Jens Gutermuth