位于不同数据中心的两台服务器之间的吞吐量问题,scp速度慢

位于不同数据中心的两台服务器之间的吞吐量问题,scp速度慢

基于网络支持,我们在DC之间拥有10Gbps的网络链路。

在这两台基于网络的服务器之间,我们应该获得 100MB/秒或 > 50MB/秒的速度,但我们得到了 6MB/秒,所以寻找原因。

两台服务器都是 Linux (RedHat Linux 7)

SeverA.Domain1.com(连接到10Gbps网络)

ethtool em1
Settings for em1:
    Supported ports: [ TP ]
    Supported link modes:   1000baseT/Full 
                            10000baseT/Full 
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  1000baseT/Full 
                            10000baseT/Full 
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Speed: 10000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 13
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: Unknown
    Supports Wake-on: g
    Wake-on: g
    Current message level: 0x00000000 (0)
                   
    Link detected: yes

Sever2.Domain2.com(连接至 1000Mb/秒)

ethtool em1
Settings for em1:
    Supported ports: [ TP ]
    Supported link modes:   1000baseT/Full 
                            10000baseT/Full 
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  1000baseT/Full 
                            10000baseT/Full 
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Link partner advertised link modes:  Not reported
    Link partner advertised pause frame use: Transmit-only
    Link partner advertised auto-negotiation: No
    Link partner advertised FEC modes: Not reported
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 13
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: Unknown
    Supports Wake-on: g
    Wake-on: g
    Current message level: 0x00000000 (0)
                   
    Link detected: yes

从服务器 A 到服务器 B 的 scp 速度 < 6 MB/秒

运行 iperf 和 qperf

qperf 显示 15MB/秒

qperf -t 60 -v ServerB.domain2.com tcp_bw tcp_lat
tcp_bw:
    bw              =  15.4 MB/sec
    msg_rate        =   234 /sec
    time            =    60 sec
    send_cost       =  20.1 sec/GB
    recv_cost       =  14.8 sec/GB
    send_cpus_used  =    31 % cpus
    recv_cpus_used  =  22.7 % cpus
tcp_lat:
    latency        =  17.3 ms
    msg_rate       =  57.7 /sec
    time           =    60 sec
    loc_cpus_used  =  29.9 % cpus
    rem_cpus_used  =  16.4 % cpus

iperf shows

iperf -c ServerB.domain2.com
------------------------------------------------------------
Client connecting to ServerB.domain2.com, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 10.xx.xxx.xx port 29404 connected with 10.xx.xxx.xxx port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  48.6 MBytes  40.7 Mbits/sec

为什么两台服务器之间的吞吐量很慢?服务器配置或网络方面有问题。

奇怪的是,如果我们反向测试方向,即从 Server2 到 Server1,我们的 scp 速度约为 50MB/秒,因此我们在反向方向上的表现会更好。

请提出建议?我正在寻找问题是否出在 Linux 服务器配置中或网络方面出现问题。

相关内容