veth 上收到的 UDP 数据包,被 tcpdump 捕获,被 iptables 接受,但未转发到 netcat

veth 上收到的 UDP 数据包,被 tcpdump 捕获,被 iptables 接受,但未转发到 netcat

我有两个命名空间 srv1 和 srv2,通过软交换机 (p4 bmv2) 与 veth 对互连。软交换只做简单的转发。命名空间内的 veth 接口分配有 IP 地址(分别为 192.168.1.1 和 192.168.1.2)。我可以使用这些 IP 地址在两个命名空间之间进行 ping 操作:

sudo ip netns exec srv1 ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=1.03 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=1.04 ms

但是当我尝试 netcat 时,我在服务器端没有收到消息:

客户:

sudo ip netns exec srv1 netcat 192.168.1.2 80 -u
 hello!

服务器:

sudo ip netns exec srv2 netcat -l 80 -u

接口接收格式正确的报文。我在两个命名空间上使用 tcpdump 进行了验证,并且看到数据包正确发送和接收:

客户:

sudo ip netns exec srv1  tcpdump -XXvv -i srv1p 
tcpdump: listening on srv1p, link-type EN10MB (Ethernet), capture size 262144 bytes
^C06:09:41.088601 IP (tos 0x0, ttl 64, id 14169, offset 0, flags [DF], proto UDP (17), length 35)
    192.168.1.1.55080 > 192.168.1.2.http: [bad udp cksum 0x8374 -> 0x5710!] UDP, length 7
    0x0000:  00aa bbcc dd02 00aa bbcc dd01 0800 4500  ..............E.
    0x0010:  0023 3759 4000 4011 801d c0a8 0101 c0a8  .#7Y@.@.........
    0x0020:  0102 d728 0050 000f 8374 6865 6c6c 6f21  ...(.P...thello!
    0x0030:  0a                                       .

1 packet captured
1 packet received by filter
0 packets dropped by kernel

服务器:

sudo ip netns exec srv2  tcpdump -XXvv -i srv2p
tcpdump: listening on srv2p, link-type EN10MB (Ethernet), capture size 262144 bytes
^C06:09:41.089232 IP (tos 0x0, ttl 64, id 14169, offset 0, flags [DF], proto UDP (17), length 35)
    192.168.1.1.55080 > 192.168.1.2.http: [bad udp cksum 0x8374 -> 0x5710!] UDP, length 7
    0x0000:  00aa bbcc dd02 00aa bbcc dd01 0800 4500  ..............E.
    0x0010:  0023 3759 4000 4011 801d c0a8 0101 c0a8  .#7Y@.@.........
    0x0020:  0102 d728 0050 000f 8374 6865 6c6c 6f21  ...(.P...thello!
    0x0030:  0a                                       .

1 packet captured
1 packet received by filter
0 packets dropped by kernel

我在 srv2 iptable 规则上添加了接受端口 80 上的 udp 数据包并记录:

sudo ip netns exec srv2 iptables -t filter -A INPUT -p udp --dport 80 -j ACCEPT
sudo ip netns exec srv2 iptables -I INPUT -p udp --dport 80 -j LOG --log-prefix " IPTABLES " --log-level=debug

我可以看到条目上的统计数据不断增加,并且数据包正在记录在 /var/log/kern.log 上,但该消息从未到达 netcat 的侦听套接字。

sudo ip netns exec srv2 iptables -L -n -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    1    33 LOG        udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:80 LOG flags 0 level 7 prefix " IPTABLES "
    4   133 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:80

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination 

内核日志:

kernel: [581970.306032]  IPTABLES IN=srv2p OUT= MAC=00:aa:bb:cc:dd:02:00:aa:bb:cc:dd:01:08:00 SRC=192.168.1.1 DST=192.168.1.2 LEN=33 TOS=0x00 PREC=0x00 TTL=64 ID=51034 DF PROTO=UDP SPT=48784 DPT=80 LEN=13 

当我用桥接器替换软交换机时,netcat 就可以工作了。我认为软交换机可能损坏了数据包,但 tcpdump 显示了正确的格式。 UDP 校验和不正确,但它是从源服务器生成的,无论如何使用 linux 桥时都是一样的,但在这种情况下它可以工作。有没有办法知道这些数据包未到达 netcat 服务器的原因?

编辑:以下是评论中要求的更多信息:

$ sudo ip netns exec srv1 ip -br link
lo               DOWN           00:00:00:00:00:00 <LOOPBACK> 
srv1p@if2948     UP             00:aa:bb:cc:dd:01 <BROADCAST,MULTICAST,UP,LOWER_UP> 

$ sudo ip netns exec srv2 ip -br link
lo               DOWN           00:00:00:00:00:00 <LOOPBACK> 
srv2p@if2944     UP             00:aa:bb:cc:dd:02 <BROADCAST,MULTICAST,UP,LOWER_UP>



$ sudo ip netns exec srv1 ip route
192.168.1.0/24 dev srv1p  proto kernel  scope link  src 192.168.1.1 

$ sudo ip netns exec srv2 ip route
192.168.1.0/24 dev srv2p  proto kernel  scope link  src 192.168.1.2 

$ sudo ip netns exec srv2 ip neighbour
192.168.1.7 dev srv2p lladdr 00:aa:bb:cc:dd:0d STALE
192.168.1.4 dev srv2p lladdr 00:aa:bb:cc:dd:06 STALE
192.168.1.8 dev srv2p lladdr 00:aa:bb:cc:dd:0e STALE
192.168.1.6 dev srv2p lladdr 00:aa:bb:cc:dd:0a STALE
192.168.1.1 dev srv2p lladdr 00:aa:bb:cc:dd:01 STALE
192.168.1.3 dev srv2p lladdr 00:aa:bb:cc:dd:05 STALE
192.168.1.5 dev srv2p lladdr 00:aa:bb:cc:dd:09 STALE

$ sudo ip netns exec srv1 ip neighbour
192.168.1.8 dev srv1p lladdr 00:aa:bb:cc:dd:0e STALE
192.168.1.4 dev srv1p lladdr 00:aa:bb:cc:dd:06 STALE
192.168.1.2 dev srv1p lladdr 00:aa:bb:cc:dd:02 STALE
192.168.1.7 dev srv1p lladdr 00:aa:bb:cc:dd:0d STALE
192.168.1.5 dev srv1p lladdr 00:aa:bb:cc:dd:09 STALE
192.168.1.3 dev srv1p lladdr 00:aa:bb:cc:dd:05 STALE
192.168.1.6 dev srv1p lladdr 00:aa:bb:cc:dd:0a STALE

$ /sbin/sysctl -ar '\.rp_filter'
net.ipv4.conf.all.rp_filter = 2
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.docker0.rp_filter = 0
net.ipv4.conf.eth0.rp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.veth0b5517e.rp_filter = 2
net.ipv4.conf.veth5c69974.rp_filter = 2
net.ipv4.conf.veth6c63fa9.rp_filter = 2
net.ipv4.conf.veth87b4442.rp_filter = 2
net.ipv4.conf.vethaecf041.rp_filter = 2
net.ipv4.conf.vethc9014e5.rp_filter = 2
net.ipv4.conf.vethf280de4.rp_filter = 2
net.ipv4.conf.vethf442944.rp_filter = 2

答案1

该问题与 udp 校验和有关。这些数据包是由 netcat 生成的,UDP 校验和错误,并且在内核中设置了一个标志来忽略它。它与 Linux 桥配合使用,因为数据包保留在内核空间中,但是当它们通过软交换机时,数据包进入用户空间并重新插入内核中,而没有忽略错误校验和的初始标志,因此它们被丢弃。

这是通过接口上的校验和卸载解决的:

/sbin/ethtool --offload $intf tx off
/sbin/ethtool --offload $intf rx off

相关内容