在 GCP 上的 bitnami moodle 的 apache 访问日志中，有多个条目（每秒添加一次！！）如下所示。这导致非常高的出口流量。我很感激任何解决此问题的指示。
52.70.240.171 - - [17/Mar/2024:17:34:25 +0530] "GET /lib/requirejs.php/1684130153/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.imageUrl(%22t/expanded%22)+%22/%22/%22 HTTP/1.1" 200 315815
52.70.240.171 - - [17/Mar/2024:17:34:26 +0530] "GET /lib/requirejs.php/1684130153/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.imageUrl(%22t/expanded%22)+%22/%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.imageUrl(%22t/expanded%22)+%22/%22/ HTTP/1.1" 200 315815
23.22.35.162 - - [17/Mar/2024:17:34:26 +0530] "GET /lib/requirejs.php/1684130153/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+h.link+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+b.params.cookiePolicyUrl+%22/%22%22+h.link+%22/%22%22+b.imageUrl(%22t/collapsed%22)+%22/%22/%22 HTTP/1.1" 200 315815
我不确定是什么原因造成的。有没有办法找出这个问题的根源？提前感谢您的帮助。
答案1
IP 的反向查找显示主机名，例如：
dig -x 52.70.240.171-->52-70-240-171.crawl.amazonbot.amazon.
亚马逊爬行机器人行为良好，并会尊重robots.txt如果您需要完全或部分阻止访问，请使用排除指令。
此外：Amazonbot 支持链接级rel=nofollow指令。将这些指令包含在 HTML 中，以使 Amazonbot 能够跟踪而不是从您的网站抓取特定链接。
经常将 User-Agent 字符串记录到您的 Web 服务器日志中有助于更快地识别机器人和爬虫。