![使用 iptables 将流量路由到 Kubernetes 集群](https://linux22.com/image/782618/%E4%BD%BF%E7%94%A8%20iptables%20%E5%B0%86%E6%B5%81%E9%87%8F%E8%B7%AF%E7%94%B1%E5%88%B0%20Kubernetes%20%E9%9B%86%E7%BE%A4.png)
我的根服务器 (netcup) 上运行着一个 kubernetes 集群(目前只有 1 个节点),并尝试将流量路由到它。我的网络技术很差,所以希望有人能帮助我。
在集群上金属已安装 calico 作为 CNI。Metallb 负责将 IP 地址分配并通告给 K8s-LoadBalancers,它在内部运行良好。因此我可以从服务器内部访问负载平衡服务。我只公开一个具有固定内部 IP(反向代理)的服务
现在我想通过使用以下方式将其暴露给外部:将流量从 metallb 直接路由到 IP 地址iptables
iptables 中应用了以下规则:
iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 80 -m comment --comment Redirect web traffic to cluster (80)
iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 443 -m comment --comment Redirect web traffic to cluster (443)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 80 -m comment --comment Redirect web traffic from cluster (80)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 443 -m comment --comment Redirect web traffic from cluster (443)
(这些命令可能不是 100% 正确,因为我是从我的 ansible 脚本中提取的。你可以在底部找到 ansible 部分)
据我所知,这应该使用 DNAT 将所有传入的 TCP 流量发送到内部 IP,而所有传出的流量将使用 SNAT 获得公共服务器地址。那么对于连接的客户端来说,它看起来应该像是在直接与公共 IP/端口对话?
不幸的是,尝试从互联网访问该服务会导致以下结果:
$> curl <public-server-ip>
curl: (7) Failed to connect to <public-server-ip> port 80 after 647 ms: Network is down
从服务器访问的工作原理:
$> curl <metallb-ip>
404 page not found # which the expected answer since its a blank Traefik reverse proxy
nmap 端口扫描当前会抛出以下错误:
$> nmap -Pn -p 80,443 <public-server-ip>
Starting Nmap 7.93 ( https://nmap.org ) at 2022-12-15 15:57 CET
Strange SO_ERROR from connection to <public-server-ip> (50 - 'Network is down') -- bailing scan
QUITTING!
但由于我通过 SSH 连接,它已经启动并运行。
目前没有ufw
启用。
我非常感谢任何帮助,因为正如我所说,我不太擅长社交。;)
Ansible 脚本:
- name: DNAT port 80 to cluster
iptables:
table: nat
chain: PREROUTING
in_interface: eth0
protocol: tcp
match: tcp
jump: DNAT # REDIRECT
to_destination: <metallb-ip>
destination_port: 80
comment: Redirect web traffic to cluster (80)
become: yes
- name: DNAT port 443 to cluster
iptables:
table: nat
chain: PREROUTING
in_interface: eth0
protocol: tcp
match: tcp
destination_port: 443
jump: DNAT # REDIRECT
to_destination: <metallb-ip>
comment: Redirect web traffic to cluster (443)
become: yes
- name: SNAT port 80 from cluster
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
protocol: tcp
match: tcp
destination_port: 80
jump: SNAT # REDIRECT
to_source: <public-server-ip>
comment: Redirect web traffic from cluster (443)
become: yes
- name: SNAT port 443 from cluster
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
protocol: tcp
match: tcp
destination_port: 443
jump: SNAT # REDIRECT
to_source: <public-server-ip>
comment: Redirect web traffic from cluster (443)
become: yes
- name: Configure IP Masquerading
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
jump: MASQUERADE
更新: 在...的帮助下这文章中我尝试了解流程。添加了日志语句来调查哪里出了问题。以下是我目前发现的:
为了检查数据包丢失的位置,我添加了以下规则:
iptables -t raw -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[raw:PREROUTING]: "
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[nat:PREROUTING]: "
iptables -t mangle -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[nat:PREROUTING]: "
/var/log/kern.log
仅出现raw
和mangle
日志语句。因此,似乎nat
未触发 -step。使用以下方法检查 mangle 规则:
sudo iptables -t mangle -L
但无法在此处发现任何问题。
IP 转发已启用:
$> sudo sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
那么为什么数据包没有到达nat
步骤呢?
$> sudo iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
cali-PREROUTING all -- anywhere anywhere /* cali:6gwbT8clXdHdC1b1 */
LOG tcp -- anywhere anywhere tcp dpt:http LOG level warning prefix "[mangle:PREROUTING]: "
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
cali-POSTROUTING all -- anywhere anywhere /* cali:O3lYWMrLQYEMJtB5 */
Chain KUBE-IPTABLES-HINT (0 references)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-PROXY-CANARY (0 references)
target prot opt source destination
Chain cali-from-host-endpoint (1 references)
target prot opt source destination
Chain cali-to-host-endpoint (1 references)
target prot opt source destination
Chain cali-PREROUTING (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere /* cali:6BJqBjBC7crtA-7- */ ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere /* cali:KX7AGNd6rMcDUai6 */ mark match 0x10000/0x10000
cali-from-host-endpoint all -- anywhere anywhere /* cali:wNH7KsA3ILKJBsY9 */
ACCEPT all -- anywhere anywhere /* cali:Cg96MgVuoPm7UMRo */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000
Chain cali-POSTROUTING (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere /* cali:NX-7roTexQ3fGRfU */ mark match 0x10000/0x10000
MARK all -- anywhere anywhere /* cali:nnqPh8lh2VOogSzX */ MARK and 0xfff0ffff
cali-to-host-endpoint all -- anywhere anywhere /* cali:nquN8Jw8Tz72pcBW */ ctstate DNAT
RETURN all -- anywhere anywhere /* cali:jWrgvDQ0xEZHmta3 */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000