我正在使用以下命令设置流量测试环境。我使用 bash shell 作为流量源,但最终将是 VM 或容器。
我不明白流量在哪里下降,希望有人能帮忙。
echo 1 > /proc/sys/net/ipv4/conf/all/log_martians
brctl addbr br0
brctl addbr br1
ip netns add nstest
ip link add veth-a type veth peer name veth-b
ip link add veth-c type veth peer name veth-d
ip link set veth-b netns nstest
ip netns exec nstest ip addr add 172.20.0.2/24 dev veth-b
ip netns exec nstest ip route add default via 172.20.0.1
ip netns exec nstest ip link set dev veth-b up
brctl addif br0 veth-a
brctl addif br0 veth-d
brctl addif br1 veth-c
ip addr add 172.20.0.1/24 dev br1
ip link set dev br0 up
ip link set dev br1 up
ip link set dev veth-a up
ip link set dev veth-c up
ip link set dev veth-d up
ip route flush cache
find /proc/sys -name rp_filter -exec sh -c "echo 0 > {}" \;
find /proc/sys -name rp_filter -print -exec sh -c "cat {}" \;
iptables -t nat -I POSTROUTING -s 172.20.0.0/16 ! -d 172.20.0.0/16 -j MASQUERADE
ip netns exec nstest bash
$ ping -c 1 172.20.0.1/24
$ ping -c 1 8.8.8.8
两座桥通过 veth 接口连接,并将 GW 放置在 BR1 上,以使流量穿越两座桥。
172.20.0.2[veth-b]----[veth-a][br0][veth-d]-----[veth-c][br1][172.20.0.1]
从 bash shell 我可以 ping GW 172.20.0.1
OK 但是如果我尝试 ping 公共地址,例如,8.8.8.8
我没有得到回复。
conntrack 显示流量
icmp 1 28 src=172.20.0.2 dst=8.8.8.8 type=8 code=0 id=6085 [UNREPLIED] src=8.8.8.8 dst=10.0.2.15 type=0 code=0 id=6085 mark=0 use=1
tcpdump 有点奇怪。MACe2:1c:84:b1:a3:5f
被分配给 nstest 网络命名空间中的 veth-d 接口。
12:37:53.547319 P e2:1c:84:b1:a3:5f ethertype IPv4 (0x0800), length 100: 172.20.0.2 > 8.8.8.8: ICMP echo request, id 6017, seq 1, length 64
12:37:53.547435 Out e2:1c:84:b1:a3:5f ethertype IPv4 (0x0800), length 100: 10.0.2.15 > 8.8.8.8: ICMP echo request, id 6017, seq 1, length 64
12:37:53.547437 In e2:1c:84:b1:a3:5f ethertype IPv4 (0x0800), length 100: 10.0.2.15 > 8.8.8.8: ICMP echo request, id 6017, seq 1, length 64
12:37:53.547437 In e2:1c:84:b1:a3:5f ethertype IPv4 (0x0800), length 100: 10.0.2.15 > 8.8.8.8: ICMP echo request, id 0, seq 1, length 64
这是 iptables TRACE
TRACE: raw:PREROUTING:policy:2 IN=br0 OUT= PHYSIN=veth-a MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: mangle:PREROUTING:policy:1 IN=br0 OUT= PHYSIN=veth-a MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: nat:PREROUTING:policy:2 IN=br0 OUT= PHYSIN=veth-a MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: mangle:FORWARD:policy:1 IN=br0 OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: filter:FORWARD:rule:1 IN=br0 OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: filter:DOCKER-ISOLATION:return:7 IN=br0 OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: filter:FORWARD:policy:16 IN=br0 OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: mangle:POSTROUTING:policy:2 IN= OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: nat:POSTROUTING:rule:1 IN= OUT=br0 PHYSIN=veth-a PHYSOUT=veth-d SRC=172.20.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: raw:PREROUTING:policy:2 IN=br1 OUT= PHYSIN=veth-c MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: mangle:PREROUTING:policy:1 IN=br1 OUT= PHYSIN=veth-c MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
TRACE: nat:PREROUTING:policy:2 IN=br1 OUT= PHYSIN=veth-c MAC=02:e4:cc:1e:06:cc:e2:1c:84:b1:a3:5f:08:00 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=33737 DF PROTO=ICMP TYPE=8 CODE=0 ID=4182 SEQ=1
IPv4: martian source 8.8.8.8 from 10.0.2.15, on dev br1
ll header: 00000000: 02 e4 cc 1e 06 cc e2 1c 84 b1 a3 5f 08 00 ..........._..
自从我禁用了 rp_filter 之后,我不知道火星的事情是否相关。
谢谢你 fLo
更新更多信息
1)是的,/proc/sys/net/ipv4/ip_forward
已启用。
2)enp0s3
默认网络空间中主主机适配器的地址为10.0.2.15/24
3)这是ip route
默认命名空间的输出
default via 10.0.2.2 dev enp0s3
10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.15
172.20.0.0/24 dev br1 proto kernel scope link src 172.20.0.1
并从testns
命名空间
default via 172.20.0.1 dev veth-b
172.20.0.0/24 dev veth-b proto kernel scope link src 172.20.0.2
4)ip a show
如果有用的话,这是来自主人的。
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:3b:e4:70 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global enp0s3
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe3b:e470/64 scope link
valid_lft forever preferred_lft forever
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 4a:0a:90:fc:18:53 brd ff:ff:ff:ff:ff:ff
inet6 fe80::480a:90ff:fefc:1853/64 scope link
valid_lft forever preferred_lft forever
4: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:e4:cc:1e:06:cc brd ff:ff:ff:ff:ff:ff
inet 172.20.0.1/24 scope global br1
valid_lft forever preferred_lft forever
inet6 fe80::e4:ccff:fe1e:6cc/64 scope link
valid_lft forever preferred_lft forever
5: veth-a@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
link/ether 4a:0a:90:fc:18:53 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::480a:90ff:fefc:1853/64 scope link
valid_lft forever preferred_lft forever
6: veth-d@veth-c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
link/ether de:5d:d5:85:08:ee brd ff:ff:ff:ff:ff:ff
inet6 fe80::dc5d:d5ff:fe85:8ee/64 scope link
valid_lft forever preferred_lft forever
7: veth-c@veth-d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br1 state UP group default qlen 1000
link/ether 02:e4:cc:1e:06:cc brd ff:ff:ff:ff:ff:ff
inet6 fe80::e4:ccff:fe1e:6cc/64 scope link
valid_lft forever preferred_lft forever