我正在将一些服务（SMTP、IMAP 等）移到 systemd-nspawn 容器中以实现更好的隔离，因此我需要将一些来自外部的连接重定向到容器。我从提供商处仅获得了此虚拟机上的 16 个 IPv6 地址，因此我决定使用 iptables 将一些端口重定向到容器中。对于 IPv4，它运行正常，但我无法让它在 IPv6 上工作。我已将问题归结为丢失虚拟以太网对等 MAC 的邻居表条目，但我不知道如何继续。为了简化问题，我创建了一个仅包含 socat 的测试容器，在端口 5555 上执行 echo。我已为它提供了一个主机和容器之间的虚拟以太网接口，该接口具有一个私有 IPv4 地址和一个 ULA IPv6 地址。
root@bonclok:~# machinectl status listenbox
listenbox(8c403969837546a7ad6a342da35fdf49)
Since: Thu 2020-09-17 10:42:50 CEST; 4s ago
Leader: 7319 ((sd-stubinit))
Service: systemd-nspawn; class container
Root: /chroot/listenbox
Iface: ve-listenbox
Address: 10.30.0.1
fdc9:c654:8207:17cb::1%91
fe80::ecaa:95ff:fe9a:1cff%91
Unit: [email protected]
├─payload
│ ├─7319 (sd-stubinit)
│ └─7324 socat -ly -d -d -d TCP6-LISTEN:5555,fork PIPE
└─supervisor
└─7317 /usr/bin/systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --network-veth -U --set
Sep 17 10:42:50 bonclok.hasiok.net systemd[1]: Started Container listenbox.
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I socat by Gerhard Rieger and contributors - see www.dest-unreach.org
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I This product includes software developed by the OpenSSL Project for use in t
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I This product includes software written by Tim Hudson ([email protected])
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I setting option "fork" to 1
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I socket(10, 1, 6) -> 6
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I starting accept loop
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: N listening on AF=10 [0000:0000:0000:0000:0000:0000:0000:0000]:5555
以下是虚拟接口两端的配置
root@bonclok:~# ip a sh dev ve-listenbox # on host
91: ve-listenbox@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 9e:81:a0:6a:93:be brd ff:ff:ff:ff:ff:ff link-netnsid 2
inet 10.30.0.2/24 brd 10.30.0.255 scope global ve-listenbox
valid_lft forever preferred_lft forever
inet6 fdc9:c654:8207:17cb::2/64 scope global
valid_lft forever preferred_lft forever
root@bonclok:~# nsenter -a -t 7324 ip a sh dev host0 # in container
2: host0@if91: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ee:aa:95:9a:1c:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.30.0.1/24 scope global host0
valid_lft forever preferred_lft forever
inet6 fdc9:c654:8207:17cb::1/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::ecaa:95ff:fe9a:1cff/64 scope link
以及路由表：
root@bonclok:~# ip -6 r|grep ve-listenbox # on host
fdc9:c654:8207:17cb::/64 dev ve-listenbox proto kernel metric 256 pref medium
root@bonclok:~# nsenter -a -t 7324 ip -6 r # in container
::1 dev lo proto kernel metric 256 pref medium
fdc9:c654:8207:17cb::/64 dev host0 proto kernel metric 256 pref medium
fe80::/64 dev host0 proto kernel metric 256 pref medium
default via fdc9:c654:8207:17cb::2 dev host0 metric 1024 pref medium
我已经通过 iptables 重定向了端口 555
root@bonclok:~# ip6tables -n -L -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DNAT tcp !fdc9:c654:8207:17cb::1 ::/0 tcp dpt:5555 to:[fdc9:c654:8207:17cb::1]:5555
现在，当我从主机连接到 fdc9:c654:8207:17cb::1 时，它可以正常工作。但如果我从外部连接，则只有当我最近从主机连接并且存在当前邻居表条目时，它才有效。我在 ve-listenbox 接口上运行了 tcpdump，当我从主机连接时：
root@bonclok:~# socat - TCP:[fdc9:c654:8207:17cb::1]:5555
我看到 ICMPv6 数据包来回传输以及 TCP 握手：
root@bonclok:~# tcpdump -t -n -vvv -i ve-listenbox
tcpdump: listening on ve-listenbox, link-type EN10MB (Ethernet), capture size 262144 bytes
IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::ecaa:95ff:fe9a:1cff > ff02::2: [icmp6 sum ok] ICMP6, router solicitation, length 16
source link-address option (1), length 8 (1): ee:aa:95:9a:1c:ff
0x0000: eeaa 959a 1cff
IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) fdc9:c654:8207:17cb::2 > ff02::1:ff00:1: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has fdc9:c654:8207:17cb::1
source link-address option (1), length 8 (1): 9e:81:a0:6a:93:be
0x0000: 9e81 a06a 93be
IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) fdc9:c654:8207:17cb::1 > fdc9:c654:8207:17cb::2: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is fdc9:c654:8207:17cb::1, Flags [solicited, override]
destination link-address option (2), length 8 (1): ee:aa:95:9a:1c:ff
0x0000: eeaa 959a 1cff
IP6 (flowlabel 0x2e8e4, hlim 64, next-header TCP (6) payload length: 40) fdc9:c654:8207:17cb::2.38176 > fdc9:c654:8207:17cb::1.5555: Flags [S], cksum 0xbc13 (incorrect -> 0xe27d), seq 3531392912, win 64800, options [mss 1440,sackOK,TS val 2219762528 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0x78e11, hlim 64, next-header TCP (6) payload length: 40) fdc9:c654:8207:17cb::1.5555 > fdc9:c654:8207:17cb::2.38176: Flags [S.], cksum 0xbc13 (incorrect -> 0x151d), seq 3651211421, ack 3531392913, win 64260, options [mss 1440,sackOK,TS val 1555204218 ecr 2219762528,nop,wscale 7], length 0
IP6 (flowlabel 0x2e8e4, hlim 64, next-header TCP (6) payload length: 32) fdc9:c654:8207:17cb::2.38176 > fdc9:c654:8207:17cb::1.5555: Flags [.], cksum 0xbc0b (incorrect -> 0x3cdf), seq 1, ack 1, win 507, options [nop,nop,TS val 2219762528 ecr 1555204218], length 0
我看到了邻居表条目。
root@bonclok:~# ip ne sh dev ve-listenbox
fdc9:c654:8207:17cb::1 lladdr ee:aa:95:9a:1c:ff REACHABLE
然后，我关闭本地连接并从外部连接后：
piotras@red:~$ telnet -6 2a03:b0c0:3:f0::36:7000 5555
Trying 2a03:b0c0:3:f0::36:7000...
Connected to 2a03:b0c0:3:f0::36:7000.
我在 eth0 上看到了原始数据包：
root@bonclok:~# tcpdump -n -t -vvv -i eth0 tcp port 5555
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
IP6 (flowlabel 0xcad87, hlim 51, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43314 > 2a03:b0c0:3:f0::36:7000.5555: Flags [S], cksum 0x7dcc (correct), seq 3406297189, win 64800, options [mss 1440,sackOK,TS val 2466613775 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0xcc342, hlim 63, next-header TCP (6) payload length: 40) 2a03:b0c0:3:f0::36:7000.5555 > 2a02:2780:9000:50:fb:ff:fe02:2.43314: Flags [S.], cksum 0x2ded (incorrect -> 0x701a), seq 1345388437, ack 3406297190, win 64260, options [mss 1440,sackOK,TS val 290303657 ecr 2466613775,nop,wscale 7], length 0
IP6 (flowlabel 0xcad87, hlim 51, next-header TCP (6) payload length: 32) 2a02:2780:9000:50:fb:ff:fe02:2.43314 > 2a03:b0c0:3:f0::36:7000.5555: Flags [.], cksum 0x97c6 (correct), seq 1, ack 1, win 507, options [nop,nop,TS val 2466613797 ecr 290303657], length 0
并在VE接口上翻译数据包。
IP6 (flowlabel 0xcad87, hlim 50, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43314 > fdc9:c654:8207:17cb::1.5555: Flags [S], cksum 0x6bc7 (correct), seq 3406297189, win 64800, options [mss 1440,sackOK,TS val 2466613775 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0xcc342, hlim 64, next-header TCP (6) payload length: 40) fdc9:c654:8207:17cb::1.5555 > 2a02:2780:9000:50:fb:ff:fe02:2.43314: Flags [S.], cksum 0x3ff2 (incorrect -> 0x5e15), seq 1345388437, ack 3406297190, win 64260, options [mss 1440,sackOK,TS val 290303657 ecr 2466613775,nop,wscale 7], length 0
IP6 (flowlabel 0xcad87, hlim 50, next-header TCP (6) payload length: 32) 2a02:2780:9000:50:fb:ff:fe02:2.43314 > fdc9:c654:8207:17cb::1.5555: Flags [.], cksum 0x85c1 (correct), seq 1, ack 1, win 507, options [nop,nop,TS val 2466613797 ecr 290303657], length 0
但一段时间后邻居表条目过期
root@bonclok:~# ip ne sh dev ve-listenbox
fdc9:c654:8207:17cb::1 INCOMPLETE
ve-listenbox 上不再有流量。当我尝试重新连接时，我看到 eth0 上的 SYN 数据包被 ICMP 目标无法访问响应
IP6 (flowlabel 0x09321, hlim 51, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43532 > 2a03:b0c0:3:f0::36:7000.5555: Flags [S], cksum 0x53f1 (correct), seq 3900363577, win 64800, options [mss 1440,sackOK,TS val 2468133554 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0xa154e, hlim 51, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43532 > 2a03:b0c0:3:f0::36:7000.5555: Flags [S], cksum 0x4feb (correct), seq 3900363577, win 64800, options [mss 1440,sackOK,TS val 2468134584 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0x7c56a, hlim 51, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43532 > 2a03:b0c0:3:f0::36:7000.5555: Flags [S], cksum 0x480b (correct), seq 3900363577, win 64800, options [mss 1440,sackOK,TS val 2468136600 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0xd1899, hlim 64, next-header ICMPv6 (58) payload length: 88) 2a03:b0c0:3:f0::36:7000 > 2a02:2780:9000:50:fb:ff:fe02:2: [icmp6 sum ok] ICMP6, destination unreachable, unreachable address 2a03:b0c0:3:f0::36:7000
IP6 (flowlabel 0xd1899, hlim 64, next-header ICMPv6 (58) payload length: 88) 2a03:b0c0:3:f0::36:7000 > 2a02:2780:9000:50:fb:ff:fe02:2: [icmp6 sum ok] ICMP6, destination unreachable, unreachable address 2a03:b0c0:3:f0::36:7000
IP6 (flowlabel 0xd1899, hlim 64, next-header ICMPv6 (58) payload length: 88) 2a03:b0c0:3:f0::36:7000 > 2a02:2780:9000:50:fb:ff:fe02:2: [icmp6 sum ok] ICMP6, destination unreachable, unreachable address 2a03:b0c0:3:f0::36:7000
但虚拟以太网上没有任何内容 - NDP 查询未在 VE 接口上发送。iptables 规则的计数增加了 - 因此它肯定正在将新的传入连接 NAT 到端口 5555，但由于没有邻居表条目，它不知道如何发送转换后的数据包。
为什么当有数据包从外部传来时它不会尝试重新查询，而当连接来自主机时却很乐意重新查询？我不明白。
当我向邻居表添加静态条目时，它可以工作，但我不喜欢这样。
我究竟做错了什么？
答案1
听起来你需要一个 NDP 代理。我使用ndppd为此目的。它的配置非常简单，只要您输入外部接口和 IPv6 块规则即可工作。