我的标题可能不太清楚
我有一台运行 CentOS 8 的服务器：
eth0连接到子网10.0.3.0/24，具有 IP10.0.3.15
eth1连接到子网10.0.103.0/24，具有 IP10.0.103.15
由于它们有不同的网关，因此我有 IP 规则：
$ ip rule list
0: from all lookup local
1024: from all to 172.16.5.0/24 lookup main
5000: from 10.0.3.0/24 lookup t1
6000: from 10.0.103.0/24 lookup t2
32766: from all lookup main
32767: from all lookup default
和路由表：
$ ip route show table t1
default via 10.0.3.1 dev eth0 proto static metric 100
10.0.3.1 dev eth0 proto static scope link metric 100
$ ip route show table t2
default via 10.0.103.1 dev eth1 proto static metric 101
10.0.103.1 dev eth1 proto static scope link metric 101
上述配置至少对于主机网络来说应该可以正常工作，因为我可以从另一台服务器 ping10.0.3.15两者10.0.103.15。
但是当我使用 podman 创建两个 redis 容器时：
podman run -d --name redis1 -p 10.0.3.15:6379:6379 redis:apline
podman run -d --name redis2 -p 10.0.103.15:6379:6379 redis:apline
只有 redis1 可以通过 访问telnet [IP] 6379，
telnet 10.0.103.15 6379无法访问任何地方，如果我在容器 redis2 中使用 tcpdump，则看不到任何内容进入。但是，如果我使用telnet 10.88.0.3 637910.88.0.3 作为 redis2 容器的内部 IP，则我可以从主机连接到 redis。
所以我怀疑 iptables NAT 规则可能有问题，但这是我的 iptables：
$ iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
CNI-HOSTPORT-DNAT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
CNI-HOSTPORT-MASQ all -- 0.0.0.0/0 0.0.0.0/0 /* CNI portfwd requiring masquerade */
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
CNI-97d3d333b7a91bb85f60a8e8 all -- 10.88.0.3 0.0.0.0/0 /* name: "podman" id: "f39d5e41593bc7f949db7bae5d06baa6d72606edec97a818a0842640ac557af5" */
CNI-0131634b11335acac74ae092 all -- 10.88.0.4 0.0.0.0/0 /* name: "podman" id: "65616612a5e5db3ef42cdae3cb40e65390bba2a826f0c7f097e3ed577e38590a" */
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
CNI-HOSTPORT-DNAT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain KUBE-MARK-DROP (1 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x8000
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
RETURN all -- 0.0.0.0/0 0.0.0.0/0 mark match ! 0x4000/0x4000
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK xor 0x4000
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ random-fully
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-LOAD-BALANCER all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes service lb portal */ match-set KUBE-LOAD-BALANCER dst,dst
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP src,dst
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-LOAD-BALANCER dst,dst
Chain KUBE-FIREWALL (0 references)
target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port with externalTrafficPolicy=local */ match-set KUBE-NODE-PORT-LOCAL-TCP dst
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-LOAD-BALANCER (1 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes service load balancer ip + port with externalTrafficPolicy=local */ match-set KUBE-LOAD-BALANCER-LOCAL dst,dst
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain CNI-HOSTPORT-SETMARK (4 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 /* CNI portfwd masquerade mark */ MARK or 0x2000
Chain CNI-HOSTPORT-MASQ (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 mark match 0x2000/0x2000
Chain CNI-HOSTPORT-DNAT (2 references)
target prot opt source destination
CNI-DN-97d3d333b7a91bb85f60a tcp -- 0.0.0.0/0 0.0.0.0/0 /* dnat name: "podman" id: "f39d5e41593bc7f949db7bae5d06baa6d72606edec97a818a0842640ac557af5" */ multiport dports 6379
CNI-DN-0131634b11335acac74ae tcp -- 0.0.0.0/0 0.0.0.0/0 /* dnat name: "podman" id: "65616612a5e5db3ef42cdae3cb40e65390bba2a826f0c7f097e3ed577e38590a" */ multiport dports 6379
Chain CNI-97d3d333b7a91bb85f60a8e8 (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.88.0.0/16 /* name: "podman" id: "f39d5e41593bc7f949db7bae5d06baa6d72606edec97a818a0842640ac557af5" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "podman" id: "f39d5e41593bc7f949db7bae5d06baa6d72606edec97a818a0842640ac557af5" */
Chain CNI-DN-97d3d333b7a91bb85f60a (1 references)
target prot opt source destination
CNI-HOSTPORT-SETMARK tcp -- 10.88.0.0/16 10.0.103.15 tcp dpt:6379
CNI-HOSTPORT-SETMARK tcp -- 127.0.0.1 10.0.103.15 tcp dpt:6379
LOG tcp -- 0.0.0.0/0 10.0.103.15 tcp dpt:6379 LOG flags 0 level 4 prefix "iptables: "
DNAT tcp -- 0.0.0.0/0 10.0.103.15 tcp dpt:6379 to:10.88.0.3:6379
Chain CNI-0131634b11335acac74ae092 (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.88.0.0/16 /* name: "podman" id: "65616612a5e5db3ef42cdae3cb40e65390bba2a826f0c7f097e3ed577e38590a" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "podman" id: "65616612a5e5db3ef42cdae3cb40e65390bba2a826f0c7f097e3ed577e38590a" */
Chain CNI-DN-0131634b11335acac74ae (1 references)
target prot opt source destination
CNI-HOSTPORT-SETMARK tcp -- 10.88.0.0/16 10.0.3.15 tcp dpt:6379
CNI-HOSTPORT-SETMARK tcp -- 127.0.0.1 10.0.3.15 tcp dpt:6379
LOG tcp -- 0.0.0.0/0 10.0.3.15 tcp dpt:6379 LOG flags 0 level 4 prefix "iptables: "
DNAT tcp -- 0.0.0.0/0 10.0.3.15 tcp dpt:6379 to:10.88.0.4:6379
如您所见，10.88.0.3（redis2，存在问题）和 10.88.0.4（redis1，可以正常工作）是相同的
过滤表中损坏的容器没有什么特别之处：
$ iptables -n -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- 169.254.0.0/16 0.0.0.0/0 tcp dpt:10250
ACCEPT tcp -- 192.168.0.0/16 0.0.0.0/0 tcp dpt:10250
ACCEPT tcp -- 172.16.0.0/12 0.0.0.0/0 tcp dpt:10250
ACCEPT tcp -- 30.0.0.0/8 0.0.0.0/0 tcp dpt:10250
ACCEPT tcp -- 11.0.0.0/8 0.0.0.0/0 tcp dpt:10250
ACCEPT tcp -- 9.0.0.0/8 0.0.0.0/0 tcp dpt:10250
ACCEPT tcp -- 10.0.0.0/8 0.0.0.0/0 tcp dpt:10250
ACCEPT tcp -- 127.0.0.0/8 0.0.0.0/0 tcp dpt:10250
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes health check rules */
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:10250
Chain FORWARD (policy ACCEPT)
target prot opt source destination
CNI-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* CNI firewall plugin rules */
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
DROP all -- !127.0.0.0/8 127.0.0.0/8 /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes health check node port */ match-set KUBE-HEALTH-CHECK-NODE-PORT dst
Chain CNI-FORWARD (1 references)
target prot opt source destination
CNI-ADMIN all -- 0.0.0.0/0 0.0.0.0/0 /* CNI firewall plugin admin overrides */
ACCEPT all -- 0.0.0.0/0 10.88.0.3 ctstate RELATED,ESTABLISHED
ACCEPT all -- 10.88.0.3 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 10.88.0.4 ctstate RELATED,ESTABLISHED
ACCEPT all -- 10.88.0.4 0.0.0.0/0
Chain CNI-ADMIN (1 references)
target prot opt source destination
可能出了什么问题？
答案1
您是否看到了以下规则的日志dmesg？
LOG tcp -- 0.0.0.0/0 10.0.103.15 tcp dpt:6379 LOG flags 0 level 4 prefix "iptables: "