我曾使用 Prometheus Blackbox Exporter http_2xx 检查监控各种 Web 服务器。然后，Web 托管服务提供商从 cPanel 迁移到 Stack CP。
从那时起，所有 http_2xx 都只返回 403（禁止）错误，我无法弄清楚原因，因为我可以从同一个 Prometheus 服务器/IP 运行成功wget和curl命令。我似乎无法使用其他命令重新创建 403 错误。
有人能对为什么会出现这种情况或如何进一步调试提出建议吗？
我已经检查了 StackCP 配置，似乎没有任何内容会禁止这些监控检查。
我在 Web 服务器日志中看不到有关 403 错误的任何内容。
blackbox.yml配置开始：
modules:
http_2xx:
prober: http
timeout: 10s
http:
method: GET
我也尝试过这个，认为这可能是 SSL/证书问题，但这并没有什么区别：
modules:
http_2xx:
prober: http
timeout: 10s
http:
method: GET
tls_config:
insecure_skip_verify: true
以下是来自 prometheus 服务器的 syslog 摘录：
Jan 31 18:08:00 Overwatch prometheus-blackbox-exporter[3362305]: ts=2024-01-31T18:08:00.356Z caller=main.go:180 module=http_2xx target=https://southcoastgroup.co.uk level=debug msg="Beginning probe" probe=http timeout_seconds=9.5
Jan 31 18:08:00 Overwatch prometheus-blackbox-exporter[3362305]: ts=2024-01-31T18:08:00.356Z caller=main.go:180 module=http_2xx target=https://southcoastgroup.co.uk level=debug msg="Resolving target address" ip_protocol=ip6
Jan 31 18:08:00 Overwatch prometheus-blackbox-exporter[3362305]: ts=2024-01-31T18:08:00.371Z caller=main.go:180 module=http_2xx target=https://southcoastgroup.co.uk level=debug msg="Resolving target address" ip_protocol=ip4
Jan 31 18:08:00 Overwatch prometheus-blackbox-exporter[3362305]: ts=2024-01-31T18:08:00.371Z caller=main.go:180 module=http_2xx target=https://southcoastgroup.co.uk level=debug msg="Resolved target address" ip=185.151.30.208
Jan 31 18:08:00 Overwatch prometheus-blackbox-exporter[3362305]: ts=2024-01-31T18:08:00.371Z caller=main.go:180 module=http_2xx target=https://southcoastgroup.co.uk level=debug msg="Making HTTP request" url=https://185.151.30.208 host=southcoastgroup.co.uk
Jan 31 18:08:00 Overwatch prometheus-blackbox-exporter[3362305]: ts=2024-01-31T18:08:00.406Z caller=main.go:180 module=http_2xx target=https://southcoastgroup.co.uk level=debug msg="Received HTTP response" status_code=403
Jan 31 18:08:00 Overwatch prometheus-blackbox-exporter[3362305]: ts=2024-01-31T18:08:00.406Z caller=main.go:180 module=http_2xx target=https://southcoastgroup.co.uk level=debug msg="Invalid HTTP response status code, wanted 2xx" status_code=403
Ubuntu 22.04.3 LTS / prometheus，版本 2.31.2+ds1 / blackbox_exporter，版本 0.19.0
按照@AlexD的建议编辑1，尽管输出对我来说意义不大。strace我可以看到它连接到网络服务器（185.151.30.208），但我看不到任何403错误或任何明显的问题原因。
tcpdump：
12:38:35.206371 IP (tos 0x0, ttl 64, id 10246, offset 0, flags [DF], proto TCP (6), length 60)
192.168.1.50.42780 > 185.151.30.208.443: Flags [S], cksum 0x9a70 (incorrect -> 0xce3d), seq 709527684, win 64240, options [mss 1460,sackOK,TS val 1145455779 ecr 0,nop,wscale 7], length 0
12:38:35.212490 IP (tos 0x0, ttl 64, id 10247, offset 0, flags [DF], proto TCP (6), length 52)
192.168.1.50.42780 > 185.151.30.208.443: Flags [.], cksum 0x9a68 (incorrect -> 0xd0f0), ack 1090810164, win 502, options [nop,nop,TS val 1145455785 ecr 1774061831], length 0
12:38:35.213068 IP (tos 0x0, ttl 64, id 10248, offset 0, flags [DF], proto TCP (6), length 351)
192.168.1.50.42780 > 185.151.30.208.443: Flags [P.], cksum 0x9b93 (incorrect -> 0x362b), seq 0:299, ack 1, win 502, options [nop,nop,TS val 1145455786 ecr 1774061831], length 299
12:38:35.219896 IP (tos 0x0, ttl 64, id 10249, offset 0, flags [DF], proto TCP (6), length 52)
192.168.1.50.42780 > 185.151.30.208.443: Flags [.], cksum 0x9a68 (incorrect -> 0xc71e), ack 2207, win 496, options [nop,nop,TS val 1145455792 ecr 1774061839], length 0
12:38:35.221063 IP (tos 0x0, ttl 64, id 10250, offset 0, flags [DF], proto TCP (6), length 116)
192.168.1.50.42780 > 185.151.30.208.443: Flags [P.], cksum 0x9aa8 (incorrect -> 0x5ff2), seq 299:363, ack 2207, win 501, options [nop,nop,TS val 1145455794 ecr 1774061839], length 64
12:38:35.221177 IP (tos 0x0, ttl 64, id 10251, offset 0, flags [DF], proto TCP (6), length 138)
192.168.1.50.42780 > 185.151.30.208.443: Flags [P.], cksum 0x9abe (incorrect -> 0xe1f8), seq 363:449, ack 2207, win 501, options [nop,nop,TS val 1145455794 ecr 1774061839], length 86
12:38:35.221291 IP (tos 0x0, ttl 64, id 10252, offset 0, flags [DF], proto TCP (6), length 125)
192.168.1.50.42780 > 185.151.30.208.443: Flags [P.], cksum 0x9ab1 (incorrect -> 0xdb93), seq 449:522, ack 2207, win 501, options [nop,nop,TS val 1145455794 ecr 1774061839], length 73
12:38:35.227011 IP (tos 0x0, ttl 64, id 10253, offset 0, flags [DF], proto TCP (6), length 52)
192.168.1.50.42780 > 185.151.30.208.443: Flags [.], cksum 0x9a68 (incorrect -> 0xc543), ack 2439, win 501, options [nop,nop,TS val 1145455800 ecr 1774061846], length 0
12:38:35.227070 IP (tos 0x0, ttl 64, id 10254, offset 0, flags [DF], proto TCP (6), length 83)
192.168.1.50.42780 > 185.151.30.208.443: Flags [P.], cksum 0x9a87 (incorrect -> 0x9012), seq 522:553, ack 2439, win 501, options [nop,nop,TS val 1145455800 ecr 1774061846], length 31
12:38:35.229169 IP (tos 0x0, ttl 64, id 10255, offset 0, flags [DF], proto TCP (6), length 76)
192.168.1.50.42780 > 185.151.30.208.443: Flags [P.], cksum 0x9a80 (incorrect -> 0xa0f1), seq 553:577, ack 2614, win 501, options [nop,nop,TS val 1145455802 ecr 1774061848], length 24
12:38:35.229208 IP (tos 0x0, ttl 64, id 10256, offset 0, flags [DF], proto TCP (6), length 52)
192.168.1.50.42780 > 185.151.30.208.443: Flags [F.], cksum 0x9a68 (incorrect -> 0xc458), seq 577, ack 2614, win 501, options [nop,nop,TS val 1145455802 ecr 1774061848], length 0
12:38:35.234976 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
192.168.1.50.42780 > 185.151.30.208.443: Flags [R], cksum 0xb7b6 (correct), seq 709528262, win 0, length 0
12:38:35.235003 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
192.168.1.50.42780 > 185.151.30.208.443: Flags [R], cksum 0xb7b6 (correct), seq 709528262, win 0, length 0
12:38:35.235026 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
192.168.1.50.42780 > 185.151.30.208.443: Flags [R], cksum 0xb7b5 (correct), seq 709528263, win 0, length 0
摘录自sudo strace -f -o file -p 3043635：
3043960 futex(0xc000046d48, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
3043636 <... nanosleep resumed>NULL) = 0
3043960 <... futex resumed>) = 1
3043638 <... futex resumed>) = 0
3043636 nanosleep({tv_sec=0, tv_nsec=20000}, <unfinished ...>
3043638 futex(0xc000046d48, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
3043960 socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP <unfinished ...>
3043636 <... nanosleep resumed>NULL) = 0
3043960 <... socket resumed>) = 8
3043636 nanosleep({tv_sec=0, tv_nsec=20000}, <unfinished ...>
3043960 connect(8, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("185.151.30.208")}, 16) = -1 EINPROGRESS (Operation now in progress)
3043636 <... nanosleep resumed>NULL) = 0
3043960 epoll_ctl(5, EPOLL_CTL_ADD, 8, {events=EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, data={u32=2636876800, u64=140362168110080}} <unfinished ...>
3043636 nanosleep({tv_sec=0, tv_nsec=20000}, <unfinished ...>
3043960 <... epoll_ctl resumed>) = 0
3043960 write(7, "\0", 1) = 1
3043636 <... nanosleep resumed>NULL) = 0
3043958 <... epoll_pwait resumed>[{events=EPOLLIN, data={u32=15774768, u64=15774768}}], 128, 9499, NULL, 2321375387197960) = 1
3043960 futex(0xc000046d48, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
3043636 nanosleep({tv_sec=0, tv_nsec=20000}, <unfinished ...>
3043960 <... futex resumed>) = 1
3043958 read(6, <unfinished ...>
3043638 <... futex resumed>) = 0
3043960 futex(0xc000081148, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
3043958 <... read resumed>"\0", 16) = 1
3043638 futex(0xc000046d48, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
3043636 <... nanosleep resumed>NULL) = 0
3043958 epoll_pwait(5, <unfinished ...>
3043636 nanosleep({tv_sec=0, tv_nsec=20000}, <unfinished ...>
3043958 <... epoll_pwait resumed>[], 128, 0, NULL, 2321375387197960) = 0
3043958 epoll_pwait(5, <unfinished ...>
3043636 <... nanosleep resumed>NULL) = 0
3043636 futex(0xedd178, FUTEX_WAIT_PRIVATE, 0, {tv_sec=9, tv_nsec=463217921} <unfinished ...>
3043958 <... epoll_pwait resumed>[{events=EPOLLOUT, data={u32=2636876800, u64=140362168110080}}], 128, 9463, NULL, 2321375387197773) = 1
3043958 futex(0xedd178, FUTEX_WAKE_PRIVATE, 1) = 1
3043958 getsockopt(8, SOL_SOCKET, SO_ERROR, <unfinished ...>
3043636 <... futex resumed>) = 0
3043958 <... getsockopt resumed>[0], [4]) = 0
3043636 nanosleep({tv_sec=0, tv_nsec=20000}, <unfinished ...>
3043958 getpeername(8, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("185.151.30.208")}, [112 => 16]) = 0
3043636 <... nanosleep resumed>NULL) = 0
3043958 futex(0xc000046d48, FUTEX_WAKE_PRIVATE, 1) = 1
3043638 <... futex resumed>) = 0
3043636 nanosleep({tv_sec=0, tv_nsec=20000}, <unfinished ...>
3043958 getsockname(8, <unfinished ...>
3043638 epoll_pwait(5, <unfinished ...>
编辑2正如@DazWilkin所注意到的，openssl命令在我的服务器上出现错误（Verify return code: 21 (unable to verify the first certificate)。检查网站后，https://www.ssllabs.com看起来认证路径中存在额外的下载，并且我的服务器上缺少几个中间证书。因此，我修复了这个问题，现在命令可以正常openssl工作，没有任何证书错误。
但是，Blackbox http_2xx 仍然返回 403
答案1
好吧，我终于搞明白了。出于某种原因，迁移的 Web 服务器根据 User-Agent 拒绝了（或者可能是缺少 User-Agent，因为我不确定 http2_xx 探测是否发送了 User-Agent）。但我需要做的就是在我的例子中添加一个类似于浏览器的假 User-Agent，如下所示：
modules:
http_2xx:
prober: http
timeout: 10s
http:
method: GET
headers:
User-Agent: "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0"