我正在尝试从 nginx 中提取在 60 秒窗口内加载我的网站页面超过 10 次的任何 IP 地址，但它必须仅用于GET / HTTPnginx 日志的主页部分/var/log/nginx/access.log，然后将这些 IP 地址转换为拒绝列表/etc/nginx/conf.d/includes-optional/deny.conf
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31822 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31826 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31826 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31714 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31822 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31774 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31748 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31774 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31720 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31820 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
2.2.2.2 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31834 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31822 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31826 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31826 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31714 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31822 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31774 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31748 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31774 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31720 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31820 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
1.1.1.1 - - [06/Sep/2021:21:02:58 +0200] "GET / HTTP/1.1" 200 31834 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/18B92 Safari/604.1 Puffin/8.0.2LP"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
3.3.3.3 - - [06/Sep/2021:21:13:36 +0200] "GET /image.php?xxxxxxxxxxxxxxxxxxxxxxxxx" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) GSA/175.0.393249130 Mobile/15E148 Safari/604.1"
由于这两个 IP 出现超过 10 次，并且在/命令运行时位于根目录中，因此应将以下内容添加到/etc/nginx/conf.d/includes-optional/deny.conf
deny 1.1.1.1;
deny 2.2.2.2;
这也必须适用于 IPv6 地址
答案1
我写了一个小项目TXR做这种事，名为txrban。我已经运行这个很多年了。不幸的是它没有记录。
程序apache.txr、exim.txr和ssh.txr是其中的入口点。这些程序扫描 Web 服务器日志、邮件服务器日志和auth.logSSH 活动。您很可能可以克隆这些程序之一，apache.txr并对其进行调整以执行您想要的操作。
这些程序加载主txrban.txr模块，主模块还加载utils.txr一个config.txr. config.txrgit 存储库中不存在；仅需要重命名和编辑的示例版本。
该startup.sh脚本是我用来在启动时启动三个进程的脚本。程序通过daemonTXR Lisp 中的函数对自身进行守护进程，并用于syslog报告禁止和取消禁止事件。
被禁止的地址不会持续存在；它只是保存在内存中的哈希表中。 （欢迎补丁。）启动时，程序会读取现有日志并执行所有隐含的禁止和取消禁止（这可能需要一些时间），然后切换到实时模式：跟踪日志并对新材料做出反应。
在该config.txr文件中，有各种参数。
用于*ban-duration*基于积分的禁止。日志扫描点可以报告 IP 地址以及非零分数。该地址立即被禁止一段时间，该时间是通过使用分数作为该*ban-duration*列表的索引来确定的。
参数*short-period*、*short-limit*和*short-ban*以及它们的long对应项决定基于时间的禁止。周期为感兴趣的访问周期，限制为该周期内的最大访问次数。该ban值是禁令的持续时间。如果某个 IP 地址的访问模式违反了长期禁止参数，则该 IP 地址将受到长期禁止。否则，如果违反短期禁令，则会相应被禁止。
主程序只需通过调用该report函数来扫描日志并报告有趣的事件，并使用三个参数：IP 地址、时间和点。 （SSH 模块使用第四个可选参数来报告从给定 IP 地址尝试登录的用户 ID。SSH 模块会对尝试使用多个用户 ID 登录失败的 IP 地址进行惩罚。）