使用 iptables 将流量路由到 Kubernetes 集群

使用 iptables 将流量路由到 Kubernetes 集群

我的根服务器 (netcup) 上运行着一个 kubernetes 集群(目前只有 1 个节点),并尝试将流量路由到它。我的网络技术很差,所以希望有人能帮助我。

在集群上金属已安装 calico 作为 CNI。Metallb 负责将 IP 地址分配并通告给 K8s-LoadBalancers,它在内部运行良好。因此我可以从服务器内部访问负载平衡服务。我只公开一个具有固定内部 IP(反向代理)的服务

现在我想通过使用以下方式将其暴露给外部:将流量从 metallb 直接路由到 IP 地址iptables

iptables 中应用了以下规则:

iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 80 -m comment --comment Redirect web traffic to cluster (80)
iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 443 -m comment --comment Redirect web traffic to cluster (443)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 80 -m comment --comment Redirect web traffic from cluster (80)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 443 -m comment --comment Redirect web traffic from cluster (443)

(这些命令可能不是 100% 正确,因为我是从我的 ansible 脚本中提取的。你可以在底部找到 ansible 部分)

据我所知,这应该使用 DNAT 将所有传入的 TCP 流量发送到内部 IP,而所有传出的流量将使用 SNAT 获得公共服务器地址。那么对于连接的客户端来说,它看起来应该像是在直接与公共 IP/端口对话?

不幸的是,尝试从互联网访问该服务会导致以下结果:

$> curl <public-server-ip>
curl: (7) Failed to connect to <public-server-ip> port 80 after 647 ms: Network is down

从服务器访问的工作原理:

$> curl <metallb-ip>
404 page not found # which the expected answer since its a blank Traefik reverse proxy

nmap 端口扫描当前会抛出以下错误:

$> nmap -Pn -p 80,443 <public-server-ip>
Starting Nmap 7.93 ( https://nmap.org ) at 2022-12-15 15:57 CET
Strange SO_ERROR from connection to <public-server-ip> (50 - 'Network is down') -- bailing scan
QUITTING!

但由于我通过 SSH 连接,它已经启动并运行。

目前没有ufw启用。

我非常感谢任何帮助,因为正如我所说,我不太擅长社交。;)

Ansible 脚本:

- name: DNAT port 80 to cluster
  iptables:
    table: nat
    chain: PREROUTING
    in_interface: eth0
    protocol: tcp
    match: tcp
    jump: DNAT # REDIRECT
    to_destination: <metallb-ip>
    destination_port: 80
    comment: Redirect web traffic to cluster (80)
  become: yes

- name: DNAT port 443 to cluster
  iptables:
    table: nat
    chain: PREROUTING
    in_interface: eth0
    protocol: tcp
    match: tcp
    destination_port: 443
    jump: DNAT # REDIRECT
    to_destination: <metallb-ip>
    comment: Redirect web traffic to cluster (443)
  become: yes 

- name: SNAT port 80 from cluster
  iptables:
    table: nat
    chain: POSTROUTING
    out_interface: eth0
    protocol: tcp
    match: tcp
    destination_port: 80
    jump: SNAT # REDIRECT
    to_source: <public-server-ip>
    comment: Redirect web traffic from cluster (443)
  become: yes

- name: SNAT port 443 from cluster
  iptables:
    table: nat
    chain: POSTROUTING
    out_interface: eth0
    protocol: tcp
    match: tcp
    destination_port: 443
    jump: SNAT # REDIRECT
    to_source: <public-server-ip>
    comment: Redirect web traffic from cluster (443)
  become: yes

- name: Configure IP Masquerading
  iptables: 
    table: nat
    chain: POSTROUTING
    out_interface: eth0
    jump: MASQUERADE

更新: 在...的帮助下文章中我尝试了解流程。添加了日志语句来调查哪里出了问题。以下是我目前发现的:

为了检查数据包丢失的位置,我添加了以下规则:

iptables -t raw -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[raw:PREROUTING]: "
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[nat:PREROUTING]: "
iptables -t mangle -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[nat:PREROUTING]: "

/var/log/kern.log仅出现rawmangle日志语句。因此,似乎nat未触发 -step。使用以下方法检查 mangle 规则: sudo iptables -t mangle -L但无法在此处发现任何问题。

IP 转发已启用:

$> sudo sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

那么为什么数据包没有到达nat步骤呢?

$> sudo iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
cali-PREROUTING  all  --  anywhere             anywhere             /* cali:6gwbT8clXdHdC1b1 */
LOG        tcp  --  anywhere             anywhere             tcp dpt:http LOG level warning prefix "[mangle:PREROUTING]: "

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
cali-POSTROUTING  all  --  anywhere             anywhere             /* cali:O3lYWMrLQYEMJtB5 */

Chain KUBE-IPTABLES-HINT (0 references)
target     prot opt source               destination

Chain KUBE-KUBELET-CANARY (0 references)
target     prot opt source               destination

Chain KUBE-PROXY-CANARY (0 references)
target     prot opt source               destination

Chain cali-from-host-endpoint (1 references)
target     prot opt source               destination

Chain cali-to-host-endpoint (1 references)
target     prot opt source               destination

Chain cali-PREROUTING (1 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             /* cali:6BJqBjBC7crtA-7- */ ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere             /* cali:KX7AGNd6rMcDUai6 */ mark match 0x10000/0x10000
cali-from-host-endpoint  all  --  anywhere             anywhere             /* cali:wNH7KsA3ILKJBsY9 */
ACCEPT     all  --  anywhere             anywhere             /* cali:Cg96MgVuoPm7UMRo */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000

Chain cali-POSTROUTING (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere             /* cali:NX-7roTexQ3fGRfU */ mark match 0x10000/0x10000
MARK       all  --  anywhere             anywhere             /* cali:nnqPh8lh2VOogSzX */ MARK and 0xfff0ffff
cali-to-host-endpoint  all  --  anywhere             anywhere             /* cali:nquN8Jw8Tz72pcBW */ ctstate DNAT
RETURN     all  --  anywhere             anywhere             /* cali:jWrgvDQ0xEZHmta3 */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000

相关内容