我有一个相当复杂的问题，涉及一台旧的 Cisco 5509 交换机和一些基于 kvm/qemu 的虚拟机。首先，系统设置如下：
|-------------------------------------------- -------------|
||----------| VMHOST | |5509 |
||VM1 | | | |
|| ------| | | 101|-------Juniper
|| |vmnic|---vnet0<->br0<->eth3.101--eth3|-----|Trunk |
||----------| | | |
| | | |
|-------------------------------------------- --------------
因此，虚拟机不了解 vlan，但通过桥接器进行连接，该桥接器使用 vlan 101 对流量进行标记，然后此 vlan 在 5509 上的中继端口上发送，该端口在端口 (101) 上发送流量，删除标记并将其传送给 Juniper。
问题是，此设置对于广播流量很有效。我可以在 vm1 和 Juniper 之间进行 arping。但是，在 juniper 和 eth3 之间的某个地方，单播会丢失，但只有从 vm1 到 juniper 的流量会丢失！
一些日志：从 vm1 到主机的 Arping 和 ping
sudo arping 192.168.0.2
ARPING 192.168.0.2
60 bytes from 00:05:85:cc:f2:10 (192.168.0.2): index=0 time=3.354 msec
60 bytes from 00:05:85:cc:f2:10 (192.168.0.2): index=1 time=3.739 msec
60 bytes from 00:05:85:cc:f2:10 (192.168.0.2): index=2 time=1.511 msec
^C
--- 192.168.0.2 statistics ---
3 packets transmitted, 3 packets received, 0% unanswered (0 extra)
PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
^C
--- 192.168.0.2 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2004ms
运行这两个命令时 eth3 的转储
sudo tcpdump -ei eth3
tcpdump: WARNING: eth3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 65535 bytes
19:25:06.871102 00:16:3e:3e:02:11 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 192.168.0.2 tell 192.168.0.1, length 28
19:25:06.872563 00:05:85:cc:f2:10 (oui Unknown) > 00:16:3e:3e:02:11 (oui Unknown), ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP, Reply 192.168.0.2 is-at 00:05:85:cc:f2:10 (oui Unknown), length 46
19:25:07.871848 00:16:3e:3e:02:11 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 192.168.0.2 tell 192.168.0.1, length 28
19:25:07.874369 00:05:85:cc:f2:10 (oui Unknown) > 00:16:3e:3e:02:11 (oui Unknown), ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP, Reply 192.168.0.2 is-at 00:05:85:cc:f2:10 (oui Unknown), length 46
19:25:08.872454 00:16:3e:3e:02:11 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 192.168.0.2 tell 192.168.0.1, length 28
19:25:09.028734 00:05:85:cc:f2:10 (oui Unknown) > 00:16:3e:3e:02:11 (oui Unknown), ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP, Reply 192.168.0.2 is-at 00:05:85:cc:f2:10 (oui Unknown), length 46
19:25:13.686148 00:16:3e:3e:02:11 (oui Unknown) > 00:05:85:cc:f2:10 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.1 > 192.168.0.2: ICMP echo request, id 1002, seq 1, length 64
19:25:14.690923 00:16:3e:3e:02:11 (oui Unknown) > 00:05:85:cc:f2:10 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.1 > 192.168.0.2: ICMP echo request, id 1002, seq 2, length 64
19:25:15.690788 00:16:3e:3e:02:11 (oui Unknown) > 00:05:85:cc:f2:10 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.1 > 192.168.0.2: ICMP echo request, id 1002, seq 3, length 64
在另一个方向执行 ping 操作：
run ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
^C
--- 192.168.0.1 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
以及来自 eth3 的相关数据包转储，显示数据包从 Juniper 到达 vm1 并一直返回 eth3 然后才消失。
sudo tcpdump -ei eth3
tcpdump: WARNING: eth3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 65535 bytes
19:27:46.960138 00:05:85:cc:f2:10 (oui Unknown) > 00:16:3e:3e:02:11 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.2 > 192.168.0.1: ICMP echo request, id 61736, seq 0, length 64
19:27:46.970773 00:16:3e:3e:02:11 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 192.168.0.2 tell 192.168.0.1, length 28
19:27:46.972689 00:05:85:cc:f2:10 (oui Unknown) > 00:16:3e:3e:02:11 (oui Unknown), ethertype 802.1Q (0x8100), length 64: vlan 101, p 0, ethertype ARP, Reply 192.168.0.2 is-at 00:05:85:cc:f2:10 (oui Unknown), length 46
19:27:46.973052 00:16:3e:3e:02:11 (oui Unknown) > 00:05:85:cc:f2:10 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.1 > 192.168.0.2: ICMP echo reply, id 61736, seq 0, length 64
19:27:47.959952 00:05:85:cc:f2:10 (oui Unknown) > 00:16:3e:3e:02:11 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.2 > 192.168.0.1: ICMP echo request, id 61736, seq 1, length 64
19:27:47.960300 00:16:3e:3e:02:11 (oui Unknown) > 00:05:85:cc:f2:10 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.1 > 192.168.0.2: ICMP echo reply, id 61736, seq 1, length 64
19:27:49.048280 00:05:85:cc:f2:10 (oui Unknown) > 00:16:3e:3e:02:11 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.2 > 192.168.0.1: ICMP echo request, id 61736, seq 2, length 64
19:27:49.048618 00:16:3e:3e:02:11 (oui Unknown) > 00:05:85:cc:f2:10 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 101, p 0, ethertype IPv4, 192.168.0.1 > 192.168.0.2: ICMP echo reply, id 61736, seq 2, length 64
8 packets captured
8 packets received by filter
0 packets dropped by kernel
一些相关的配置。第一个vm1：
eth1 Link encap:Ethernet HWaddr 00:16:3e:3e:02:11
inet addr:192.168.0.1 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fe3e:211/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1953 errors:0 dropped:0 overruns:0 frame:0
TX packets:3933 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:153032 (153.0 KB) TX bytes:315162 (315.1 KB)
Interrupt:10 Base address:0x6000
然后是 vmhost：
brctl show
bridge name bridge id STP enabled interfaces
br0 8000.001e68a9b341 no eth3.101
vnet0
以及接口和桥梁（不相关的内容已删除）
eth3 Link encap:Ethernet HWaddr 00:1e:68:a9:b3:41
inet6 addr: fe80::21e:68ff:fea9:b341/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:4306 errors:0 dropped:0 overruns:0 frame:0
TX packets:4870 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:329486 (329.4 KB) TX bytes:419680 (419.6 KB)
Interrupt:47 Base address:0xc000
eth3.101 Link encap:Ethernet HWaddr 00:1e:68:a9:b3:41
inet6 addr: fe80::21e:68ff:fea9:b341/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:2082 errors:0 dropped:0 overruns:0 frame:0
TX packets:3697 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:159118 (159.1 KB) TX bytes:306482 (306.4 KB)
br0 Link encap:Ethernet HWaddr 00:1e:68:a9:b3:41
inet6 addr: fe80::490:41ff:fea8:25bd/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:4006 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:296858 (296.8 KB) TX bytes:468 (468.0 B)
vnet0 Link encap:Ethernet HWaddr fe:16:3e:3e:02:11
inet6 addr: fe80::fc16:3eff:fe3e:211/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:3940 errors:0 dropped:0 overruns:0 frame:0
TX packets:2004 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:315680 (315.6 KB) TX bytes:184138 (184.1 KB)
5509配置的相关部分
set vlan 100-104
set spantree disable all
set trunk 3/8 on dot1q 101-104,201-204,301-304,401-404,501-504,1002-1005
set vlan 101 4/1
最后是杜松箱
fe-0/0/0 {
unit 0 {
family inet {
address 192.168.0.2/24;
}
}
}
此时，我开始强烈怀疑 5509 或桥接设置中存在配置问题，但我甚至无法想象是什么原因导致了这个问题。有网络经验的人能建议一种解决此问题的方法吗？如果您需要更多信息，请直接询问。
编辑：
进一步的调试表明这与 Juniper 盒中的过滤器有关。我仍然不知道发生了什么，但是“输入 DA 拒绝”计数器会随着每个数据包的丢失而增加。
run show interfaces fe-0/0/0 extensive
Physical interface: fe-0/0/0, Enabled, Physical link is Up
Interface index: 129, SNMP ifIndex: 118, Generation: 130
Link-level type: Ethernet, MTU: 1514, Link-mode: Half-duplex, Speed: 100mbps,
MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled,
Flow control: Enabled
Device flags : Present Running
Interface flags: SNMP-Traps Internal: 0x4000
CoS queues : 8 supported, 8 maximum usable queues
Hold-times : Up 0 ms, Down 0 ms
Current address: 00:05:85:cc:f2:10, Hardware address: 00:05:85:cc:f2:10
Last flapped : 2011-01-24 19:03:05 CET (16:10:25 ago)
Statistics last cleared: Never
Traffic statistics:
Input bytes : 109620 0 bps
Output bytes : 331366 0 bps
Input packets: 2035 0 pps
Output packets: 5611 0 pps
Input errors:
Errors: 1, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0,
L3 incompletes: 1, L2 channel errors: 0, L2 mismatch timeouts: 0,
FIFO errors: 0, Resource errors: 0
Output errors:
Carrier transitions: 7, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0,
FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0
Egress queues: 8 supported, 4 in use
Queue counters: Queued packets Transmitted packets Dropped packets
0 best-effort 5611 5611 0
1 expedited-fo 0 0 0
2 assured-forw 0 0 0
3 network-cont 0 0 0
Active alarms : None
Active defects : None
MAC statistics: Receive Transmit
Total octets 0 331926
Total packets 0 5611
Unicast packets 0 3234
Broadcast packets 0 2377
Multicast packets 0 0
CRC/Align errors 0 0
FIFO errors 0 0
MAC control frames 0 0
MAC pause frames 0 0
Oversized frames 0
Jabber frames 0
Fragment frames 0
VLAN tagged frames 0
Code violations 0
Filter statistics:
Input packet count 64407
Input packet rejects 62371
Input DA rejects 62371
Input SA rejects 0
Output packet count 0
Output packet pad count 0
Output packet error count 0
CAM destination filters: 1, CAM source filters: 0
Autonegotiation information:
Negotiation status: Complete
Link partner:
Link mode: Full-duplex, Flow control: None, Remote fault: OK,
Link partner Speed: 100 Mbps
Packet Forwarding Engine configuration:
Destination slot: 0
Direction : Output
CoS transmit queue Bandwidth Buffer Priority Limit
% bps % usec
0 best-effort 95 95000000 95 0 low none
3 network-control 5 5000000 5 0 low none
Logical interface fe-0/0/0.0 (Index 68) (SNMP ifIndex 136) (Generation 133)
Flags: SNMP-Traps Encapsulation: ENET2
Traffic statistics:
Input bytes : 222600
Output bytes : 331366
Input packets: 2035
Output packets: 5611
Local statistics:
Input bytes : 112980
Output bytes : 328006
Input packets: 1995
Output packets: 5571
Transit statistics:
Input bytes : 109620 0 bps
Output bytes : 3360 0 bps
Input packets: 40 0 pps
Output packets: 40 0 pps
Protocol inet, MTU: 1500, Generation: 139, Route table: 0
Flags: None
Addresses, Flags: Is-Preferred Is-Primary
Destination: 192.168.0/24, Local: 192.168.0.2, Broadcast: 192.168.0.255,
Generation: 140
答案1
问题出在 Juniper 机箱中的 CompactFlash 卡损坏。存储系统映像的闪存卡已损坏，可能是由于写入次数过多造成的。最有可能的是，损坏的映像在线路卡上加载了损坏的代码，从而导致线路卡行为异常。
用新的闪存替换旧的，在其上载入新的图像，然后恢复配置，一切便恢复正常。