问题:我在 Docker 中有一个可用的 WireGuard 设置(请参阅指南:关联),但在将配置移植到具有主机网络的 Kubernetes 时,很难实现客户端的互联网访问。我可以进行握手,甚至可以 ping 主机的 LAN IP,但似乎无法到达默认网关。
请注意,我使用 21421 作为外部端口,并将流量转发到 51820。我的 wireguard 子网是 10.14.14.0/24 和 2601:204:xxxx:xxxc::/64;我的 LAN 子网是 10.0.0.0/24 和 2601:204:xxxx:xxx0::/64。
配置映射.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: wireguard-config
data:
PUID: "1000"
PGID: "1000"
TZ: "America/Los_Angeles"
SERVERURL: my.website.addr
SERVERPORT: "21421"
PEERS: pphone,wphone,tablet,laptop,trouter
PEERDNS: 75.75.75.75,75.75.76.76,2001:558:feed::1,2001:558:feed::2
INTERNAL_SUBNET: 10.14.14.0/24
ALLOWEDIPS: 0.0.0.0/0, ::/0
PERSISTENTKEEPALIVE_PEERS: all
部署.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: wireguard
spec:
selector:
matchLabels:
app: wireguard
replicas: 1
template:
metadata:
labels:
app: wireguard
spec:
nodeSelector:
kubernetes.io/hostname: obsidiana
hostNetwork: true
containers:
- name: wireguard
image: linuxserver/wireguard:latest
securityContext:
privileged: true
capabilities:
add:
- NET_ADMIN
- SYS_MODULE
volumeMounts:
- name: wireguard-configfiles
mountPath: /config
- name: lib-modules
mountPath: /lib/modules
envFrom:
- configMapRef:
name: wireguard-config
volumes:
- name: wireguard-configfiles
hostPath:
path: /srv/wireguard/config
- name: lib-modules
hostPath:
path: /lib/modules
此外,以下是主机上的 IP 路由(请注意 wireguard 子网 10.14.14.0/24 和 2601:204:xxxx:xxxc::/64 的存在):
atom@obsidiana [10:53:18] [/srv/wireguard]
-> % ip -c route
default via 10.0.0.1 dev enp3s0
default via 10.0.0.1 dev enp3s0 proto dhcp src 10.0.0.238 metric 100
10.0.0.0/24 dev enp3s0 proto kernel scope link src 10.0.0.238 metric 100
10.0.0.1 dev enp3s0 proto dhcp scope link src 10.0.0.238 metric 100
10.14.14.2 dev wg0 scope link
10.14.14.3 dev wg0 scope link
10.14.14.4 dev wg0 scope link
10.14.14.5 dev wg0 scope link
10.14.14.6 dev wg0 scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev br-1b4d200d1cbb proto kernel scope link src 172.18.0.1 linkdown
172.19.0.0/16 dev br-a1be084c54c9 proto kernel scope link src 172.19.0.1 linkdown
172.21.0.0/16 dev br-4d301d3707dd proto kernel scope link src 172.21.0.1
172.25.0.0/16 dev br-8745f19da673 proto kernel scope link src 172.25.0.1
172.26.0.0/16 dev br-d9ec277ec93b proto kernel scope link src 172.26.0.1
172.27.0.0/16 dev br-8a6e7b3004eb proto kernel scope link src 172.27.0.1
192.168.48.0/20 dev br-45b26225ad0a proto kernel scope link src 192.168.48.1 linkdown
192.168.67.0/24 dev br-2fe8a6223784 proto kernel scope link src 192.168.67.1 linkdown
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
blackhole 192.168.139.128/26 proto 80
192.168.139.154 dev cali151eafd1c9f scope link
192.168.139.160 dev calia50db85314e scope link
192.168.139.164 dev calia28aed46668 scope link
192.168.139.166 dev calib00d4512918 scope link
192.168.139.167 dev cali2018d45df2e scope link
192.168.139.168 dev cali339a2a73fab scope link
192.168.139.169 dev calia8fc0d7cff4 scope link
192.168.139.170 dev cali5d667b293c0 scope link
192.168.139.172 dev calic7ba6791d16 scope link
192.168.139.173 dev calif47c6967706 scope link
192.168.139.174 dev caliaeb0ffaab04 scope link
192.168.139.175 dev caliaf5a7cc0076 scope link
192.168.139.176 dev cali4497ec7f2ec scope link
192.168.176.0/20 dev br-3606b1dbef9e proto kernel scope link src 192.168.176.1
192.168.190.64/26 via 10.0.0.1 dev enp3s0 proto 80 onlink
atom@obsidiana [10:57:51] [/srv/wireguard]
-> % ip -c -6 route
::1 dev lo proto kernel metric 256 pref medium
2601:204:xxxx:xxx0::/64 dev enp3s0 proto ra metric 100 expires 3588sec pref medium
2601:204:xxxx:xxxc::1 dev wg0 proto kernel metric 256 pref medium
2601:204:xxxx:xxxc::2 dev wg0 metric 1024 pref medium
2601:204:xxxx:xxxc::3 dev wg0 metric 1024 pref medium
2601:204:xxxx:xxxc::4 dev wg0 metric 1024 pref medium
2601:204:xxxx:xxxc::5 dev wg0 metric 1024 pref medium
2601:204:xxxx:xxxc::6 dev wg0 metric 1024 pref medium
fd2b:938d:7743:1::/64 proto ra metric 100 expires 1655sec pref medium
nexthop via fe80::d358:7828:fa79:4a97 dev enp3s0 weight 1
nexthop via fe80::d9c7:c6cc:58c8:1181 dev enp3s0 weight 1
fe80::/64 dev enp3s0 proto kernel metric 256 pref medium
fe80::/64 dev br-45b26225ad0a proto kernel metric 256 linkdown pref medium
fe80::/64 dev br-4d301d3707dd proto kernel metric 256 pref medium
fe80::/64 dev br-8745f19da673 proto kernel metric 256 pref medium
fe80::/64 dev vethca97195 proto kernel metric 256 pref medium
fe80::/64 dev br-d9ec277ec93b proto kernel metric 256 pref medium
fe80::/64 dev veth3e9a2b2 proto kernel metric 256 pref medium
fe80::/64 dev br-3606b1dbef9e proto kernel metric 256 pref medium
fe80::/64 dev veth5f2e53f proto kernel metric 256 pref medium
fe80::/64 dev br-8a6e7b3004eb proto kernel metric 256 pref medium
fe80::/64 dev veth42b0ce5 proto kernel metric 256 pref medium
fe80::/64 dev veth4730c27 proto kernel metric 256 pref medium
fe80::/64 dev cali151eafd1c9f proto kernel metric 256 pref medium
fe80::/64 dev calia50db85314e proto kernel metric 256 pref medium
fe80::/64 dev calib00d4512918 proto kernel metric 256 pref medium
fe80::/64 dev cali2018d45df2e proto kernel metric 256 pref medium
fe80::/64 dev cali339a2a73fab proto kernel metric 256 pref medium
fe80::/64 dev calia28aed46668 proto kernel metric 256 pref medium
fe80::/64 dev cali5d667b293c0 proto kernel metric 256 pref medium
fe80::/64 dev calia8fc0d7cff4 proto kernel metric 256 pref medium
fe80::/64 dev calif47c6967706 proto kernel metric 256 pref medium
fe80::/64 dev caliaeb0ffaab04 proto kernel metric 256 pref medium
fe80::/64 dev caliaf5a7cc0076 proto kernel metric 256 pref medium
fe80::/64 dev cali4497ec7f2ec proto kernel metric 256 pref medium
fe80::/64 dev calic7ba6791d16 proto kernel metric 256 pref medium
fe80::/64 dev veth3c7f6d9 proto kernel metric 256 pref medium
default via fe80::6cf2:67ff:fed0:9b95 dev enp3s0 proto ra metric 100 expires 1788sec pref medium
我已经调整了主机上的防火墙规则以适应主机网络(请注意 wg0 的存在,以及 wireguard 子网 10.14.14.0/24、2601:204:xxxx:xxxc::/64)。
trusted (active)
target: ACCEPT
icmp-block-inversion: no
interfaces: enp3s0 wg0
sources: 2601:204:xxxx:xxx0::/64 2601:204:xxxx:xxxc::/64 10.14.14.0/24 10.0.0.0/24 192.168.0.0/16
services:
ports:
protocols:
forward: yes
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
在具有活动客户端的网关/路由器上运行tcpdump -i br0 udp and port 51820
显示双向流量(br0 是 LAN iface,obsidiana 是托管 WireGuard 的 PC):
listening on br0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:10:52.858477 IP obsidiana.51820 > 172.56.168.229.41909: UDP, length 32
16:10:52.858919 IP obsidiana.51820 > 172.56.168.229.41909: UDP, length 148
16:10:53.810684 IP 172.56.168.229.41909 > obsidiana.51820: UDP, length 92
16:10:53.810900 IP obsidiana.51820 > 172.56.168.229.41909: UDP, length 32
16:10:55.867321 IP 108.147.99.17.35334 > obsidiana.51820: UDP, length 148
16:10:55.867700 IP obsidiana.51820 > 108.147.99.17.35334: UDP, length 92
16:10:55.948070 IP 108.147.99.17.35334 > obsidiana.51820: UDP, length 96
16:10:55.948476 IP 108.147.99.17.35334 > obsidiana.51820: UDP, length 96
16:10:56.272068 IP 108.147.99.17.35334 > obsidiana.51820: UDP, length 128
我还可以看到来自路由器的双向流量tcpdump -i enp10s0 udp and port 21421
(enp10s0 是 WAN,21421 是 wireguard 的外部端口):
18:03:54.241853 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 112
18:03:54.248918 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 112
18:03:54.669307 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:54.679954 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:55.269114 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 96
18:03:55.285552 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 96
18:03:55.758942 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:55.774862 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:55.835307 IP c-73-151-158-xxx.hsd1.ca.comcast.net.21421 > 172.56.168.229.41909: UDP, length 32
18:03:56.769571 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:56.774526 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:56.859496 IP c-73-151-158-xxx.hsd1.ca.comcast.net.21421 > 108.147.99.18.60458: UDP, length 32
18:03:57.688746 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:57.691103 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:58.776023 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:58.776023 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:59.791058 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
18:03:59.791058 IP 108.147.99.18.60458 > c-73-151-158-xxx.hsd1.ca.comcast.net.21421: UDP, length 128
最后但同样重要的一点是,以下是默认网关(firewalld)的相关防火墙设置:
➜ ~ sudo firewall-cmd --list-all --zone=home
home (active)
target: default
icmp-block-inversion: no
interfaces: br0 wg0
sources: 192.168.0.0/16 10.0.0.0/24 2601:204:xxxx:xxx0::/64 2601:204:xxxx:xxxc::/64
services: dhcp dhcpv6-client dns dropbox-lansync elasticsearch grafana http iperf kibana kube-apiserver kube-repo kubelet mdns netbootxyz plex remote-wireguard samba-client ssh upnp wireguard
ports: 6667/udp 49152/tcp 9101/tcp 9093/tcp 5353/udp
protocols: igmp
forward: yes
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
➜ ~ sudo firewall-cmd --list-all --zone=external
external (active)
target: default
icmp-block-inversion: no
interfaces: enp10s0
sources:
services: dhcpv6-client shadowsocks
ports:
protocols:
forward: yes
masquerade: no
forward-ports:
port=21421:proto=udp:toport=51820:toaddr=10.0.0.238
source-ports:
icmp-blocks:
rich rules:
有什么想法可能出现什么问题吗?