我的一台开发服务器（CentOS 6）突然发现入站网络流量大幅增加
它让速度变得非常慢，SSH 需要 10 秒以上才能登录，输入时有延迟，网站超时，Nagios 感到不安，因为 NRPE 检查不断超时（这是我的 Nagios 主机），所以似乎突然出现了巨大的网络流量风暴，但我不知道它来自哪里。该服务器有一个公共 IP，因此可以直接访问，它运行一个非常严格的 IPTables 规则集（仅允许 80、443 和几个其他实用程序端口用于 Jenkins 等）。我尝试使用类似的工具，iftop但它们没有显示任何异常。不确定这是因为 IPTables 阻止了连接，所以它们没有显示出来，但因为我不确定这些是试图连接到我的服务器的外部设备，还是其他原因。这似乎很奇怪，它使 SSH 变慢，其他服务无响应，但网络流量大约在同一时间开始，我第一次开始遇到问题。我应该在哪里查看以找出这些流量来自哪里以及如何阻止它？我无法直接访问任何路由器，但我可以在服务器上做任何我想做的事情。我查看了 /var/log/messages，发现有很多我从未见过的有关 DNS 的奇怪消息，但它们似乎不是错误，只是过于冗长的日志记录（见下文）。
标准有用的东西；
[sr@ns309372 ~]$ sudo uptime
23:51:41 up 6:30, 3 users, load average: 0.03, 0.12, 0.11
[sr@ns309372 ~]$ sudo free -m
total used free shared buffers cached
Mem: 3920 2197 1722 0 103 1060
-/+ buffers/cache: 1032 2887
Swap: 1019 0 1019
[sr@ns309372 ~]$ sudo tail -n 30 /var/log/messages
Apr 18 23:11:08 ns309372 named[2451]: success resolving 'ftp.halifax.rwth-aachen.de/A' (in 'rwth-aachen.de'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:11:10 ns309372 named[2451]: success resolving 'deneb.dfn.de/AAAA' (in 'dfn.de'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:11:10 ns309372 named[2451]: success resolving 'ns1.leaseweb.nl/AAAA' (in 'leaseweb.nl'?) after disabling EDNS
Apr 18 23:11:15 ns309372 named[2451]: success resolving 'ns4.leaseweb.net/AAAA' (in 'leaseweb.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:11:22 ns309372 named[2451]: success resolving 'pkg.jenkins-ci.org/A' (in 'jenkins-ci.org'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:11:30 ns309372 named[2451]: success resolving 'mirror.ovh.net/A' (in 'ovh.net'?) after disabling EDNS
Apr 18 23:11:30 ns309372 named[2451]: success resolving 'mirror.ovh.net/AAAA' (in 'ovh.net'?) after disabling EDNS
Apr 18 23:33:54 ns309372 named[2451]: success resolving 'vs1.nagios.org/A' (in 'nagios.org'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:34:36 ns309372 named[2451]: success resolving 'ns.ripe.net/A' (in 'ripe.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:34:36 ns309372 named[2451]: success resolving 'dns1.ntli.net/AAAA' (in 'ntli.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:34:37 ns309372 named[2451]: success resolving 'dns2.ntli.net/AAAA' (in 'ntli.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:34:37 ns309372 named[2451]: success resolving 'dns2.ntli.net/A' (in 'ntli.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:34:38 ns309372 named[2451]: success resolving 'sec1.apnic.net/AAAA' (in 'apnic.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:34:39 ns309372 named[2451]: success resolving 'sec3.apnic.net/AAAA' (in 'apnic.net'?) after disabling EDNS
Apr 18 23:34:40 ns309372 named[2451]: success resolving 'sec3.apnic.net/A' (in 'apnic.net'?) after disabling EDNS
Apr 18 23:34:40 ns309372 named[2451]: success resolving 'dns2.ntli.net/AAAA' (in 'ntli.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:35:02 ns309372 named[2451]: success resolving 'urlatron.com/AAAA' (in 'urlatron.com'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:35:03 ns309372 named[2451]: success resolving 'urlatron.com/A' (in 'urlatron.com'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:35:56 ns309372 named[2451]: success resolving 'bitbucket.org/A' (in 'bitbucket.org'?) after disabling EDNS
Apr 18 23:48:26 ns309372 named[2451]: success resolving '113.155.23.94.in-addr.arpa/PTR' (in '155.23.94.in-addr.arpa'?) after disabling EDNS
Apr 18 23:48:29 ns309372 named[2451]: success resolving '8.137.145.217.in-addr.arpa/PTR' (in '217.in-addr.arpa'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:48:29 ns309372 named[2451]: success resolving '10.169.216.196.in-addr.arpa/PTR' (in '169.216.196.in-addr.arpa'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:48:29 ns309372 named[2451]: success resolving 'ns2.lacnic.net/AAAA' (in 'lacnic.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:48:30 ns309372 named[2451]: success resolving 'ns2.dns.br/AAAA' (in 'br'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:48:30 ns309372 named[2451]: success resolving 'ns2.dns.br/A' (in 'br'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:48:34 ns309372 named[2451]: success resolving 'ns2.afrinic.net/A' (in 'afrinic.net'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:50:03 ns309372 named[2451]: success resolving 'urlatron.com/A' (in 'urlatron.com'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:50:04 ns309372 named[2451]: success resolving 'ns2.ecogeek.org/A' (in 'ecogeek.org'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:50:05 ns309372 named[2451]: success resolving 'ns1.ecogeek.org/AAAA' (in 'ecogeek.org'?) after reducing the advertised EDNS UDP packet size to 512 octets
Apr 18 23:50:05 ns309372 named[2451]: success resolving 'urlatron.com/AAAA' (in 'urlatron.com'?) after reducing the advertised EDNS UDP packet size to 512 octets
[sr@ns309372 ~]$ sudo ifconfig
eth0 Link encap:Ethernet HWaddr 00:27:0E:0B:86:51
inet addr:188.165.192.119 Bcast:188.165.192.255 Mask:255.255.255.0
inet6 addr: fe80::227:eff:fe0b:8651/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:456082 errors:0 dropped:91 overruns:0 frame:0
TX packets:821015 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:59793427 (57.0 MiB) TX bytes:1008283171 (961.5 MiB)
Interrupt:43 Base address:0xc000
eth0:0 Link encap:Ethernet HWaddr 00:27:0E:0B:86:51
inet addr:94.23.155.32 Bcast:94.23.155.32 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:43 Base address:0xc000
eth0:1 Link encap:Ethernet HWaddr 00:27:0E:0B:86:51
inet addr:94.23.155.113 Bcast:94.23.155.113 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:43 Base address:0xc000
eth0:2 Link encap:Ethernet HWaddr 00:27:0E:0B:86:51
inet addr:178.32.48.78 Bcast:178.32.48.78 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:43 Base address:0xc000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:169675 errors:0 dropped:0 overruns:0 frame:0
TX packets:169675 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:172646550 (164.6 MiB) TX bytes:172646550 (164.6 MiB)
[sr@ns309372 ~]$ sudo sar -n DEV 1 3
Linux 2.6.38.2-grsec-xxxx-grs-ipv6-64 (ns309372.ovh.net) 18/04/12 _x86_64_ (2 CPU)
23:57:35 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
23:57:36 lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:36 dummy0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:36 eth0 13.00 8.00 1.11 5.08 0.00 0.00 0.00
23:57:36 tunl0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:36 sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:36 ip6tnl0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:36 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
23:57:37 lo 10.00 10.00 2.92 2.92 0.00 0.00 0.00
23:57:37 dummy0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:37 eth0 11.00 8.00 0.91 3.47 0.00 0.00 0.00
23:57:37 tunl0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:37 sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:37 ip6tnl0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:37 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
23:57:38 lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:38 dummy0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:38 eth0 7.00 9.00 7.54 1.33 0.00 0.00 0.00
23:57:38 tunl0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:38 sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
23:57:38 ip6tnl0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
Average: lo 3.33 3.33 0.97 0.97 0.00 0.00 0.00
Average: dummy0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: eth0 10.33 8.33 3.19 3.30 0.00 0.00 0.00
Average: tunl0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: ip6tnl0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
我有多个“虚拟”接口来支持多个 IP，只有一个物理接口
答案1
我会做两件事：
在 iptables 中记录流量。您创建一条规则并使用
LOG（使用内核日志系统）和ULOG（将日志定向到套接字而不是内核系统）目标。因此，您可能对所有-A INPUT发送到的数据包感兴趣-j LOG，也许--log-prefix "incoming packets "还有--log-level 6（它遵循系统日志级别）监控您的进程以查看是否有任何进程接收了大量带宽。我建议使用网猪。流量可能会通过 iptables，但不会被写入磁盘，而是会被某个奇怪的进程立即丢弃。