我在 KVM 服务器上的几台计算机遇到了互联网连接问题，该服务器也遇到了同样的问题。Windows 客户机上的互联网连接工作正常。这种情况发生在多台机器上。网关是 pfsense 防火墙，根据其默认配置允许所有 LAN 端流量，192.168.1.1 路由器使用其 DMZ 将所有流量推送到防火墙。
我已经使用 ping 测试了连通性。
$ ping 216.58.220.100
PING 216.58.220.100 (216.58.220.100) 56(84) bytes of data.
64 bytes from 216.58.220.100: icmp_seq=1 ttl=56 time=49.3 ms
64 bytes from 216.58.220.100: icmp_seq=2 ttl=56 time=51.5 ms
64 bytes from 216.58.220.100: icmp_seq=3 ttl=56 time=49.8 ms
64 bytes from 216.58.220.100: icmp_seq=4 ttl=56 time=84.2 ms
64 bytes from 216.58.220.100: icmp_seq=5 ttl=56 time=49.1 ms
^C
--- 216.58.220.100 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 49.139/56.820/84.288/13.762 ms
Nslookup 工作正常
$ nslookup google.com
Server: 10.0.100.1
Address: 10.0.100.1#53
Non-authoritative answer:
Name: google.com
Address: 216.58.220.110
我可以通过 80 端口进行 telnet
$ telnet google.com 80
Trying 216.58.220.110...
Connected to google.com.
Escape character is '^]'.
但是，如果我尝试发送命令，例如 GET / HTTP/1.0 来查看它是否会返回 HTTP 标头，则什么也不会发生。
当我尝试 HTTP GET 命令时，我在防火墙上针对服务器 IP 运行了数据包捕获并发现了这一点，但我不太确定该如何处理它。
1 0.000000 IntelCor_93:8f:4b Broadcast ARP 60 Who has IP.IP.IP.10? Tell IP.IP.IP.200
2 2.130120 IP.IP.IP.10 216.58.220.100 TCP 66 55642?80 [FIN, ACK] Seq=1 Ack=1 Win=229 Len=0 TSval=19535544 TSecr=1347772624
3 3.816772 IP.IP.IP.10 IP.IP.IP.1 DNS 74 Standard query 0x13ec A www.google.com
4 3.816839 IP.IP.IP.10 IP.IP.IP.1 DNS 74 Standard query 0xf326 AAAA www.google.com
5 4.003369 IP.IP.IP.1 IP.IP.IP.10 DNS 102 Standard query response 0xf326 AAAA 2404:6800:4006:801::2004
6 4.035623 IP.IP.IP.1 IP.IP.IP.10 DNS 90 Standard query response 0x13ec A 216.58.220.100
7 4.036683 IP.IP.IP.10 216.58.220.100 TCP 74 55643?80 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=19536021 TSecr=0 WS=128
8 4.086164 216.58.220.100 IP.IP.IP.10 TCP 74 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347876636 TSecr=19536021 WS=128
9 4.086474 IP.IP.IP.10 216.58.220.100 TCP 66 55643?80 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=19536033 TSecr=1347876636
10 4.548559 216.58.220.100 IP.IP.IP.10 TCP 74 [TCP Spurious Retransmission] 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347877098 TSecr=19536021 WS=128
11 4.548808 IP.IP.IP.10 216.58.220.100 TCP 66 [TCP Dup ACK 9#1] 55643?80 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=19536149 TSecr=1347876636
12 6.547488 216.58.220.100 IP.IP.IP.10 TCP 74 [TCP Spurious Retransmission] 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347879098 TSecr=19536021 WS=128
13 6.547774 IP.IP.IP.10 216.58.220.100 TCP 66 [TCP Dup ACK 9#2] 55643?80 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=19536649 TSecr=1347876636
14 8.532081 IP.IP.IP.10 216.58.220.100 TCP 82 [TCP segment of a reassembled PDU]
15 8.781690 IP.IP.IP.10 216.58.220.100 TCP 82 [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19537208 TSecr=1347876636
16 9.033636 IP.IP.IP.10 216.58.220.100 TCP 82 [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19537271 TSecr=1347876636
17 9.453382 IP.IP.IP.10 216.58.220.100 TCP 82 [TCP Retransmission] [TCP segment of a reassembled PDU]
18 9.537284 IP.IP.IP.10 216.58.220.100 TCP 82 [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19537397 TSecr=1347876636
19 10.544897 IP.IP.IP.10 216.58.220.100 TCP 82 [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19537649 TSecr=1347876636
20 10.545286 216.58.220.100 IP.IP.IP.10 TCP 74 [TCP Spurious Retransmission] 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347883098 TSecr=19536021 WS=128
21 10.545542 IP.IP.IP.10 216.58.220.100 TCP 66 [TCP Dup ACK 19#1] 55643?80 [ACK] Seq=17 Ack=1 Win=29312 Len=0 TSval=19537649 TSecr=1347876636
22 12.563760 IP.IP.IP.10 216.58.220.100 TCP 82 [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19538154 TSecr=1347876636
23 16.601741 IP.IP.IP.10 216.58.220.100 TCP 82 [TCP Retransmission] 55643?80 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=16 TSval=19539164 TSecr=1347876636
24 18.540457 216.58.220.100 IP.IP.IP.10 TCP 74 [TCP Spurious Retransmission] 80?55643 [SYN, ACK] Seq=0 Ack=1 Win=42540 Len=0 MSS=1402 SACK_PERM=1 TSval=1347891098 TSecr=19536021 WS=128
但每当我尝试使用 wget 之类的东西时，请求就会失败。
$ wget -T60 google.com
--2015-09-03 11:28:31-- http://google.com/
Resolving google.com (google.com)... 216.58.220.110, 2404:6800:4006:801::200e
Connecting to google.com (google.com)|216.58.220.110|:80... connected.
HTTP request sent, awaiting response... Read error (Connection timed out) in headers.
Retrying.
我还收到了奇怪的跟踪路由响应
$ traceroute to google.com (216.58.220.110), 30 hops max, 60 byte packets
1 Firewall.domain.lan (Removed) 1.002 ms 0.994 ms 0.973 ms
2 192.168.1.1 (192.168.1.1) 2.018 ms 2.010 ms 1.991 ms
3 * * *
4 te1-1-6.sydgscore1.wireline.com.au (103.19.172.61) 103.634 ms 107.541 ms 109.968 ms
5 ge1-1-2.sydgsbdr2.wireline.com.au (103.19.172.34) 111.886 ms 115.329 ms 1 17.046 ms
6 as15169.nsw.ix.asn.au (218.100.52.3) 120.206 ms 121.862 ms 123.631 ms
7 209.85.242.124 (209.85.242.124) 128.192 ms 147.385 ms 129.917 ms
8 * * *
9 * * *
...
29 * * *
30 * * *
但在 Windows 上运行良好
$ tracert google.com
Tracing route to google.com [216.58.220.110] over a maximum of 30 hops:
1 1 ms 2 ms 2 ms Firewall.domain.lan [Removed]
2 2 ms 2 ms 2 ms 192.168.1.1
3 63 ms 62 ms 67 ms 1.1.1.1
4 99 ms 74 ms 55 ms te1-1-6.sydgscore1.wireline.com.au [103.19.172.61]
5 63 ms 51 ms 51 ms ge1-1-2.sydgsbdr2.wireline.com.au [103.19.172.34]
6 50 ms 55 ms 52 ms as15169.nsw.ix.asn.au [218.100.52.3]
7 51 ms 52 ms 62 ms 209.85.242.124
8 51 ms 51 ms 52 ms 209.85.142.11
9 52 ms 52 ms 64 ms syd10s01-in-f14.1e100.net [216.58.220.110]
Trace complete.
ip addr 的输出：
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:8d:d6:5a brd ff:ff:ff:ff:ff:ff
inet IP Removed/24 brd 10.0.100.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe8d:d65a/64 scope link
valid_lft forever preferred_lft forever
路线看起来都很好：
Destination Gateway Genmask Flags Metric Ref Use Iface
default Firewall.lan 0.0.0.0 UG 0 0 0 eth0
IP Removed * 255.255.255.0 U 0 0 0 eth0
KVM 主机上的 Netstat
$ netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
br0 1500 0 2704662 0 0 0 1815107 0 0 0 BMRU
eth0 1500 0 8757521 9 30 0 9220563 0 0 0 BMRU
eth1 1500 0 0 0 0 0 0 0 0 0 BMU
lo 65536 0 1778126 0 0 0 1778126 0 0 0 LRU
vnet0 1500 0 699 0 0 0 5860 0 0 0 BMRU
vnet2 1500 0 6525 0 0 0 432071 0 0 0 BMRU
vnet3 1500 0 3724358 0 0 0 6696965 0 0 0 BMRU
vnet4 1500 0 6956040 0 0 0 6418627 0 0 0 BMRU
vnet5 1500 0 3766203 0 0 0 2957162 0 1107 0 BMRU
vnet6 1500 0 2946274 0 0 0 4322828 0 0 0 BMRU
ARP 表
$ arp -e
Address HWtype HWaddress Flags Mask Iface
IP.IP.IP.10 ether 52:54:00:8d:d6:5a C eth0
IP.IP.IP.5 ether 52:54:00:bc:40:03 C eth0
firewall.domain.lan ether 52:54:00:b8:74:20 C eth0
ss -lt | grep 80没有返回任何内容，因此没有可能干扰端口 80 流量的服务。
export | grep -i proxy由于我没有配置代理，因此不返回任何内容
IPTables 配置：
$ iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
答案1
事实证明，VirtIO 的硬件 TX 校验和卸载功能存在问题，这会导致 PFSense 出现问题，表现为这种部分工作的连接。
$sudo ethtool -K eth0 tx off针对每个虚拟接口的运行解决了该问题。