信息::
我的刀片机箱中有 6 台 HP ProLiant BL460c G7 服务器。安装了 RHEL6.2 64 位。每台刀片服务器 NIC1 都在使用中,机箱 LAN 电缆仅与交换机连接。交换机配置为自动协商模式。
问题::
当 TCP 网络负载中等时,所有服务器都会出现数百个 RX 错误。六台服务器中有两台是生产服务器(主备模式),并且 Rx 错误流量最大。
我在相同的生产服务器上运行 org 网站(非商业项目)。通常,站点会有 <500 个集中建立会话,在高峰时段会达到 > 500,我观察到 ESTABLISH 会话<100,然后 RX 错误中没有动静,我正在分享活动生产服务器的详细信息
下面是在我的服务器上设置的用于扩展 TCP 缓冲区值的内核参数:
[root@ ~]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
net.ipv4.ip_local_port_range = 10000 65500
fs.aio-max-nr = 1048576
fs.file-max = 100000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.optmem_max = 40960
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.core.netdev_max_backlog = 50000
net.ipv4.tcp_max_syn_backlog = 32768
net.core.somaxconn = 2048
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 20
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.udp_rmem_min = 8192
net.ipv4.udp_wmem_min = 8192
net.ipv4.conf.all.send_redirects = 0
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 60
net.ipv4.tcp_keepalive_probes = 3
SYSTEM Detail is here ::
[root@ ~]# uname -a
Linux mohit 2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
[root@ ~]# netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0 1500 0 43061491 2394 0 0 39363458 0 0 0 BMRU
lo 16436 0 12792095 0 0 0 12792095 0 0 0 LRU
You have mail in /var/spool/mail/root
[root@ ~]# ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: g
Wake-on: d
Link detected: yes
[root@ ~]# ethtool -S eth0|grep error
rx_errors: 2394
tx_errors: 0
rx_crc_errors: 0
rx_alignment_symbol_errors: 0
rx_in_range_errors: 0
rx_out_range_errors: 0
rx_address_match_errors: 6932