以太网链路断开并在服务器重启时恢复

以太网链路断开并在服务器重启时恢复

我们有一台 DELL R610 服务器连接到 DELL PowerConnect 5424 交换机。此交换机连接到 DELL Equallogic SAN。DELL R610 用作 MySQL 数据库服务器,SAN 提供数据目录,作为 iSCSI 驱动器安装。

根据我之前提出的一个问题这里,我们观察到重启后,MySQL 无法自行启动。这也是一种间歇性行为。调查后我们发现 iSCSI 启动器服务在启动时无法执行命令。检查日志后,我们发现网络接口启动时会出现一个小故障。使用 grep 命令查看 bnx2 日志:

 bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.2.4 (Aug 05, 2013)
 bnx2 0000:01:00.0 eth0: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem d6000000, IRQ 36, node addr 5c:f9:dd:f1:8a:ea
 bnx2 0000:01:00.1 eth1: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem d8000000, IRQ 48, node addr 5c:f9:dd:f1:8a:ec
 bnx2 0000:02:00.0 eth2: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem da000000, IRQ 32, node addr 5c:f9:dd:f1:8a:ee
 bnx2 0000:02:00.1 eth3: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem dc000000, IRQ 42, node addr 5c:f9:dd:f1:8a:f0
 bnx2 0000:02:00.0: irq 78 for MSI/MSI-X
 bnx2 0000:02:00.0: irq 79 for MSI/MSI-X
 bnx2 0000:02:00.0: irq 80 for MSI/MSI-X
 bnx2 0000:02:00.0: irq 81 for MSI/MSI-X
 bnx2 0000:02:00.0: irq 82 for MSI/MSI-X
 bnx2 0000:02:00.0: irq 83 for MSI/MSI-X
 bnx2 0000:02:00.0: irq 84 for MSI/MSI-X
 bnx2 0000:02:00.0: irq 85 for MSI/MSI-X
 bnx2 0000:02:00.0: irq 86 for MSI/MSI-X
 bnx2 0000:02:00.0 em3: using MSIX
 bnx2 0000:01:00.0: irq 87 for MSI/MSI-X
 bnx2 0000:01:00.0: irq 88 for MSI/MSI-X
 bnx2 0000:01:00.0: irq 89 for MSI/MSI-X
 bnx2 0000:01:00.0: irq 90 for MSI/MSI-X
 bnx2 0000:01:00.0: irq 91 for MSI/MSI-X
 bnx2 0000:01:00.0: irq 92 for MSI/MSI-X
 bnx2 0000:01:00.0: irq 93 for MSI/MSI-X
 bnx2 0000:01:00.0: irq 94 for MSI/MSI-X
 bnx2 0000:01:00.0: irq 95 for MSI/MSI-X
 bnx2 0000:01:00.0 em1: using MSIX
 bnx2 0000:01:00.1: irq 96 for MSI/MSI-X
 bnx2 0000:01:00.1: irq 97 for MSI/MSI-X
 bnx2 0000:01:00.1: irq 98 for MSI/MSI-X
 bnx2 0000:01:00.1: irq 99 for MSI/MSI-X
 bnx2 0000:01:00.1: irq 100 for MSI/MSI-X
 bnx2 0000:01:00.1: irq 101 for MSI/MSI-X
 bnx2 0000:01:00.1: irq 102 for MSI/MSI-X
 bnx2 0000:01:00.1: irq 103 for MSI/MSI-X
 bnx2 0000:01:00.1: irq 104 for MSI/MSI-X
 bnx2 0000:01:00.1 em2: using MSIX
 bnx2 0000:01:00.0 em1: NIC Copper Link is Up, 100 Mbps full duplex
 bnx2 0000:02:00.0 em3: NIC Copper Link is Up, 1000 Mbps full duplex
 bnx2 0000:01:00.1 em2: NIC Copper Link is Up, 1000 Mbps full duplex
 **bnx2 0000:01:00.1 em2: NIC Copper Link is Down**
 bnx2 0000:01:00.1 em2: NIC Copper Link is Up, 1000 Mbps full duplex

我们目前的解决方法是重启服务器。到目前为止,重启服务器后,一切都顺利进行,没有观察到上面日志中显示的故障。

有人能帮忙解决故障吗?我已经提到这里但这很可能不是我的情况,因为我们的问题只发生在重启时。除此之外,ifconfig 中没有 NIC 错误,NIC 中也没有数据包丢失。服务器启动后,我们从未遇到过任何网络问题。

DELL R610 运行的是 Ubuntu 14.04。

@Dom 建议的更多日志:

 $ cat logfile.mysql.withoutdate |grep -B 15 -A 15 "NIC Copper Link is Down"
 Loading iSCSI transport class v2.0-870.
 bnx2 0000:01:00.1 em2: using MSIX
 IPv6: ADDRCONF(NETDEV_UP): em2: link is not ready
 iscsi: registered transport (tcp)
 iscsi: registered transport (iser)
 multipathd (2470): /proc/2470/oom_adj is deprecated, please use /proc/2470/oom_score_adj instead.
 bnx2 0000:01:00.0 em1: NIC Copper Link is Up, 100 Mbps full duplex

 IPv6: ADDRCONF(NETDEV_CHANGE): em1: link becomes ready
 bnx2 0000:02:00.0 em3: NIC Copper Link is Up, 1000 Mbps full duplex
 , receive & transmit flow control ON
 IPv6: ADDRCONF(NETDEV_CHANGE): em3: link becomes ready
 bnx2 0000:01:00.1 em2: NIC Copper Link is Up, 1000 Mbps full duplex
 , receive & transmit flow control ON
 IPv6: ADDRCONF(NETDEV_CHANGE): em2: link becomes ready
 bnx2 0000:01:00.1 em2: NIC Copper Link is Down
 bnx2 0000:01:00.1 em2: NIC Copper Link is Up, 1000 Mbps full duplex
 , receive & transmit flow control ON

相关内容