在对 MPLS 电路上的 TCP 速度在一个方向上变慢的原因进行了大量故障排除后,我发现通过调整 TCP 窗口大小可以大大改善情况。也许你可以告诉我原因。
细节:
- 两台主机之间的 ping 时间为 7ms
- 两台机器都运行 Debian Wheezy,内核为 3.2.0-4
- 两台机器都有默认的 TCP 设置
- 两台机器都有 10 Gbps 上行链路
使用默认 TCP 设置的结果:
地点 A -> 地点 B: iperf -c $Location B --> ~2 Gbps
地点 B -> 地点 A:iperf -c $Location A --> 65 - 250 Mbps
应用以下更改后/etc/sysctl.conf我得到了更好的结果
net.ipv4.tcp_window_scaling = 1
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 16384 16777216
调整TCP设置后的结果:
地点 A -> 地点 B: iperf -c $Location B --> ~2 Gbps
地点 B -> 地点 A: iperf -c $Location A --> 150 Mbps - 2 Gbps
(大部分约为 1.8 Gbps)
我的问题是,什么可能会对 TCP 窗口缩放产生负面影响?以下是跟踪我已经记录下来了。也许这有助于找出问题所在。
感谢您的帮助!
问候,Leon Kramer
#
# Speed: ~2 Gbps
#
1 arg remaining, starting with 'tcpdump.lol'
Ostermann's tcptrace -- version 6.6.7 -- Thu Nov 4, 2004
260307 packets seen, 260307 TCP packets traced
elapsed wallclock time: 0:00:17.740455, 14673 pkts/sec analyzed
trace file elapsed time: 0:00:03.798052
TCP connection info:
2 TCP connections traced:
================================
TCP connection 2:
host c: 10.1.2.5:59242
host d: 10.1.2.2:5201
complete conn: RESET (SYNs: 2) (FINs: 0)
first packet: Thu Nov 13 00:05:18.256992 2014
last packet: Thu Nov 13 00:05:21.876883 2014
elapsed time: 0:00:03.619891
total packets: 260276
filename: tcpdump.lol
c->d: d->c:
total packets: 156463 total packets: 103813
resets sent: 0 resets sent: 154
ack pkts sent: 156462 ack pkts sent: 103660
pure acks sent: 1 pure acks sent: 103658
sack pkts sent: 0 sack pkts sent: 8716
dsack pkts sent: 0 dsack pkts sent: 6
max sack blks/ack: 0 max sack blks/ack: 1
unique bytes sent: 674405781 unique bytes sent: 0
actual data pkts: 156461 actual data pkts: 0
actual data bytes: 674415917 actual data bytes: 0
rexmt data pkts: 6 rexmt data pkts: 0
rexmt data bytes: 10136 rexmt data bytes: 0
zwnd probe pkts: 0 zwnd probe pkts: 0
zwnd probe bytes: 0 zwnd probe bytes: 0
outoforder pkts: 3784 outoforder pkts: 0
pushed data pkts: 28 pushed data pkts: 0
SYN/FIN pkts sent: 1/0 SYN/FIN pkts sent: 1/0
req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y
adv wind scale: 7 adv wind scale: 7
req sack: Y req sack: Y
sacks sent: 0 sacks sent: 8716
urgent data pkts: 0 pkts urgent data pkts: 0 pkts
urgent data bytes: 0 bytes urgent data bytes: 0 bytes
mss requested: 1460 bytes mss requested: 1460 bytes
max segm size: 14480 bytes max segm size: 0 bytes
min segm size: 37 bytes min segm size: 0 bytes
avg segm size: 4310 bytes avg segm size: 0 bytes
max win adv: 14720 bytes max win adv: 3145728 bytes
min win adv: 14720 bytes min win adv: 14592 bytes
zero win adv: 0 times zero win adv: 0 times
avg win adv: 14720 bytes avg win adv: 3123096 bytes
initial window: 37 bytes initial window: 0 bytes
initial window: 1 pkts initial window: 0 pkts
ttl stream length: NA ttl stream length: NA
missed data: NA missed data: NA
truncated data: 0 bytes truncated data: 0 bytes
truncated packets: 0 pkts truncated packets: 0 pkts
data xmit time: 3.612 secs data xmit time: 0.000 secs
idletime max: 43.3 ms idletime max: 43.3 ms
throughput: 186305549 Bps throughput: 0 Bps
====================================================================================
#
# Speed: ~65 Mbps
#
1 arg remaining, starting with 'tcpdump.lol'
Ostermann's tcptrace -- version 6.6.7 -- Thu Nov 4, 2004
652370 packets seen, 652370 TCP packets traced
elapsed wallclock time: 0:00:07.985123, 81698 pkts/sec analyzed
trace file elapsed time: 0:02:13.121598
TCP connection info:
2 TCP connections traced:
================================
TCP connection 2:
host c: 10.1.2.2:45259
host d: 10.1.2.5:5201
complete conn: RESET (SYNs: 2) (FINs: 1)
first packet: Wed Nov 12 22:42:06.606682 2014
last packet: Wed Nov 12 22:44:19.553859 2014
elapsed time: 0:02:12.947177
total packets: 652339
filename: tcpdump.lol
c->d: d->c:
total packets: 446774 total packets: 205565
resets sent: 0 resets sent: 22
ack pkts sent: 446773 ack pkts sent: 205543
pure acks sent: 1 pure acks sent: 205541
sack pkts sent: 0 sack pkts sent: 9322
dsack pkts sent: 0 dsack pkts sent: 2
max sack blks/ack: 0 max sack blks/ack: 3
unique bytes sent: 1058366405 unique bytes sent: 0
actual data pkts: 446772 actual data pkts: 0
actual data bytes: 1058379437 actual data bytes: 0
rexmt data pkts: 5 rexmt data pkts: 0
rexmt data bytes: 13032 rexmt data bytes: 0
zwnd probe pkts: 0 zwnd probe pkts: 0
zwnd probe bytes: 0 zwnd probe bytes: 0
outoforder pkts: 1577 outoforder pkts: 0
pushed data pkts: 5733 pushed data pkts: 0
SYN/FIN pkts sent: 1/0 SYN/FIN pkts sent: 1/1
req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y
adv wind scale: 7 adv wind scale: 7
req sack: Y req sack: Y
sacks sent: 0 sacks sent: 9322
urgent data pkts: 0 pkts urgent data pkts: 0 pkts
urgent data bytes: 0 bytes urgent data bytes: 0 bytes
mss requested: 1460 bytes mss requested: 1460 bytes
max segm size: 18824 bytes max segm size: 0 bytes
min segm size: 37 bytes min segm size: 0 bytes
avg segm size: 2368 bytes avg segm size: 0 bytes
max win adv: 14720 bytes max win adv: 561920 bytes
min win adv: 14720 bytes min win adv: 14592 bytes
zero win adv: 0 times zero win adv: 0 times
avg win adv: 14720 bytes avg win adv: 533768 bytes
initial window: 37 bytes initial window: 0 bytes
initial window: 1 pkts initial window: 0 pkts
ttl stream length: NA ttl stream length: 0 bytes
missed data: NA missed data: 0 bytes
truncated data: 0 bytes truncated data: 0 bytes
truncated packets: 0 pkts truncated packets: 0 pkts
data xmit time: 132.940 secs data xmit time: 0.000 secs
idletime max: 37.8 ms idletime max: 37.8 ms
throughput: 7960804 Bps throughput: 0 Bps