proxmox 和 pfsense VM 之间的 IPTables 问题

proxmox 和 pfsense VM 之间的 IPTables 问题

我对 IPTables 问题感到有点绝望。

我开始设置自己的专用服务器,即 proxmox 服务器。我有一个 pfSense VM,所有流量都路由到该 VM。我通过构建 VPN 来保护 LAN 访问。一切都很好,每台机器都能互相 ping 通,但我有我的主机有问题:proxmox 服务器。我可以 ping 通它,但无法访问 Proxmox Web UI(端口 8006)或任何其他端口(我尝试了端口 8000 的 http.server)。

这是我设置 iptables 的脚本:

#!/bin/sh

    # ---------
    # VARIABLES
    # ---------

## Proxmox bridge holding Public IP
PrxPubVBR="vmbr0"
## Proxmox bridge on VmWanNET (PFSense WAN side)
PrxVmWanVBR="vmbr1"

## Network/Mask of VmWanNET
VmWanNET="192.168.0.0/30"

## Public IP => Your own public IP address
PublicIP="1.2.3.4"
## Proxmox IP on the same network than PFSense WAN (VmWanNET)
ProxVmWanIP="192.168.0.1"
## PFSense IP used by the firewall (inside VM)
PfsVmWanIP="192.168.0.2"

    # ---------------------
    # CLEAN ALL & DROP IPV6
    # ---------------------

### Delete all existing rules.
iptables -F
iptables -t nat -F
iptables -t mangle -F
iptables -X
### This policy does not handle IPv6 traffic except to drop it.
ip6tables -P INPUT DROP
ip6tables -P OUTPUT DROP
ip6tables -P FORWARD DROP

    # --------------
    # DEFAULT POLICY
    # --------------

### Block ALL !
iptables -P OUTPUT DROP
iptables -P INPUT DROP
iptables -P FORWARD DROP

    # ------
    # CHAINS
    # ------

### Creating chains
iptables -N TCP
iptables -N UDP

# UDP = ACCEPT / SEND TO THIS CHAIN
iptables -A INPUT -p udp -m conntrack --ctstate NEW -j UDP
# TCP = ACCEPT / SEND TO THIS CHAIN
iptables -A INPUT -p tcp --syn -m conntrack --ctstate NEW -j TCP

    # ------------
    # GLOBAL RULES
    # ------------

# Allow localhost
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Don't break the current/active connections
iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow Ping - Comment this to return timeout to ping request
iptables -A INPUT -p icmp --icmp-type 8 -m conntrack --ctstate NEW -j ACCEPT

    # --------------------
    # RULES FOR PrxPubVBR
    # --------------------

### INPUT RULES
# ---------------

# Allow SSH server
iptables -A TCP -i $PrxPubVBR -d $PublicIP -p tcp --dport 56361 -j ACCEPT
# Allow Proxmox WebUI
#iptables -A TCP -i $PrxPubVBR -d $PublicIP -p tcp --dport 8006 -j ACCEPT

### OUTPUT RULES
# ---------------

# Allow ping out
iptables -A OUTPUT -p icmp -j ACCEPT

### Proxmox Host as CLIENT
# Allow HTTP/HTTPS
iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --dport 80 -j ACCEPT
iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --dport 443 -j ACCEPT
# Allow DNS
iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p udp --dport 53 -j ACCEPT

### Proxmox Host as SERVER
# Allow SSH
iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --sport 56361 -j ACCEPT
# Allow PROXMOX WebUI
#iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --sport 8006 -j ACCEPT

### FORWARD RULES
# ----------------

### Redirect (NAT) traffic from internet
# All tcp to PFSense WAN except 56361, 8006
iptables -A PREROUTING -t nat -i $PrxPubVBR -p tcp --match multiport ! --dport 56361 -j DNAT --to $PfsVmWanIP
# All udp to PFSense WAN
iptables -A PREROUTING -t nat -i $PrxPubVBR -p udp -j DNAT --to $PfsVmWanIP

# Allow request forwarding to PFSense WAN interface
iptables -A FORWARD -i $PrxPubVBR -d $PfsVmWanIP -o $PrxVmWanVBR -p tcp -j ACCEPT
iptables -A FORWARD -i $PrxPubVBR -d $PfsVmWanIP -o $PrxVmWanVBR -p udp -j ACCEPT

# Allow request forwarding from LAN
iptables -A FORWARD -i $PrxVmWanVBR -s $VmWanNET -j ACCEPT

### MASQUERADE MANDATORY
# Allow WAN network (PFSense) to use vmbr0 public adress to go out
iptables -t nat -A POSTROUTING -s $VmWanNET -o $PrxPubVBR -j MASQUERADE

#Allow WAN network (PFSense) to use vmbr1
iptables -A TCP -i $PrxVmWanVBR -d $ProxVmWanIP -p tcp --dport 8006 -j ACCEPT

    # --------------------
    # RULES FOR PrxVmWanVBR
    # --------------------

### Allow being a client for the VMs
#iptables -A OUTPUT -o $PrxVmWanVBR -s $ProxVmWanIP -p tcp -j ACCEPT

#Debug
iptables -A OUTPUT -o vmbr1 -s 192.168.0.1 -p tcp -j LOG

iptables-L在 proxmox 服务器上 (192.168.0.1)

Chain INPUT (policy DROP)
target     prot opt source               destination
UDP        udp  --  anywhere             anywhere             ctstate NEW
TCP        tcp  --  anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN ctstate NEW
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     icmp --  anywhere             anywhere             icmp echo-request ctstate NEW

Chain FORWARD (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             192.168.0.2
ACCEPT     udp  --  anywhere             192.168.0.2
ACCEPT     all  --  192.168.0.0/30       anywhere

Chain OUTPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere
ACCEPT     tcp  --  proxhost             anywhere             tcp dpt:http
ACCEPT     tcp  --  proxhost             anywhere             tcp dpt:https
ACCEPT     udp  --  proxhost             anywhere             udp dpt:domain
ACCEPT     tcp  --  proxhost             anywhere             tcp spt:56361
LOG        tcp  --  192.168.0.1          anywhere             LOG level warning

Chain TCP (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             proxhost             tcp dpt:56361
ACCEPT     tcp  --  anywhere             192.168.0.1          tcp dpt:8006

Chain UDP (1 references)
target     prot opt source               destination

tcpdump -i vmbr1 -p tcp port 8006样本在 PROXMOX 服务器上 (192.168.0.1)

21:55:20.469413 IP 192.168.0.2.19382 > 192.168.0.1.8006: Flags [S], seq 2361462968, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:51.271929 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:51.525784 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:52.279653 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:52.536924 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:54.293975 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:54.550895 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:58.308703 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:56:58.551754 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:57:06.313913 IP 192.168.0.2.41994 > 192.168.0.1.8006: Flags [S], seq 3056927626, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0
21:57:06.552666 IP 192.168.0.2.61212 > 192.168.0.1.8006: Flags [S], seq 3653062990, win 64240, options [mss 1361,nop,wscale 8,nop,nop,sackOK], length 0

我肯定错过了什么,但我真的看不出在哪里。谢谢 :)

编辑:

iptables-save -c

# Generated by iptables-save v1.8.2 on Thu Dec  9 20:23:23 2021
*mangle
:PREROUTING ACCEPT [38261:1392282]
:INPUT ACCEPT [24532:758223]
:FORWARD ACCEPT [10902:503215]
:OUTPUT ACCEPT [21470:682727]
:POSTROUTING ACCEPT [29151:1040069]
COMMIT
# Completed on Thu Dec  9 20:23:23 2021
# Generated by iptables-save v1.8.2 on Thu Dec  9 20:23:23 2021
*nat
:PREROUTING ACCEPT [6492:234198]
:INPUT ACCEPT [2:104]
:OUTPUT ACCEPT [227:14700]
:POSTROUTING ACCEPT [2486:123267]
[2311:104588] -A PREROUTING -i vmbr0 -p tcp -m multiport ! --dports 56361 -j DNAT --to-destination 192.168.0.2
[2967:147367] -A PREROUTING -i vmbr0 -p udp -j DNAT --to-destination 192.168.0.2
[307:23332] -A POSTROUTING -s 192.168.0.0/30 -o vmbr0 -j MASQUERADE
COMMIT
# Completed on Thu Dec  9 20:23:23 2021
# Generated by iptables-save v1.8.2 on Thu Dec  9 20:23:23 2021
*raw
:PREROUTING ACCEPT [7393057:7076961567]
:OUTPUT ACCEPT [135130:41526439]
COMMIT
# Completed on Thu Dec  9 20:23:23 2021
# Generated by iptables-save v1.8.2 on Thu Dec  9 20:23:23 2021
*filter
:INPUT DROP [3256:115951]
:FORWARD DROP [2988:97686]
:OUTPUT DROP [233:48187]
:TCP - [0:0]
:UDP - [0:0]
[0:0] -A INPUT -p udp -m conntrack --ctstate NEW -j UDP
[2:104] -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j TCP
[3:156] -A INPUT -i lo -j ACCEPT
[21271:642012] -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
[1189:95194] -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
[6277:270428] -A FORWARD -d 192.168.0.2/32 -i vmbr0 -o vmbr1 -p tcp -j ACCEPT
[140:16523] -A FORWARD -d 192.168.0.2/32 -i vmbr0 -o vmbr1 -p udp -j ACCEPT
[308:23384] -A FORWARD -s 192.168.0.0/30 -i vmbr1 -j ACCEPT
[3:156] -A OUTPUT -o lo -j ACCEPT
[21055:611502] -A OUTPUT -p icmp -j ACCEPT
[0:0] -A OUTPUT -s 1.2.3.4/32 -o vmbr0 -p tcp -m tcp --dport 80 -j ACCEPT
[0:0] -A OUTPUT -s 1.2.3.4/32 -o vmbr0 -p tcp -m tcp --dport 443 -j ACCEPT
[70:4312] -A OUTPUT -s 1.2.3.4/32 -o vmbr0 -p udp -m udp --dport 53 -j ACCEPT
[115:19514] -A OUTPUT -s 1.2.3.4/32 -o vmbr0 -p tcp -m tcp --sport 56361 -j ACCEPT
[2:104] -A TCP -d 1.2.3.4/32 -i vmbr0 -p tcp -m tcp --dport 56361 -j ACCEPT
[0:0] -A TCP -d 192.168.0.1/32 -i vmbr1 -p tcp -m tcp --dport 8006 -j ACCEPT
COMMIT
# Completed on Thu Dec  9 20:23:23 2021

ip -br link:

lo               UNKNOWN        00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP>
eno1             UP             ac:1f:6b:71:2c:f6 <BROADCAST,MULTICAST,UP,LOWER_UP>
eno2             DOWN           ac:1f:6b:71:2c:f7 <BROADCAST,MULTICAST>
vmbr0            UP             ac:1f:6b:71:2c:f6 <BROADCAST,MULTICAST,UP,LOWER_UP>
vmbr1            UP             fe:b6:c7:0f:cb:04 <BROADCAST,MULTICAST,UP,LOWER_UP>
vmbr2            UP             82:6f:f1:04:65:39 <BROADCAST,MULTICAST,UP,LOWER_UP>

ip -br address:

lo               UNKNOWN        127.0.0.1/8 ::1/128
eno1             UP
eno2             DOWN
vmbr0            UP             1.2.3.4/32 fe80::ae1f:6bff:fe71:2cf6/64
vmbr1            UP             192.168.0.1/30 fe80::2c70:1ff:fe15:c679/64
vmbr2            UP             192.168.1.1/24 fe80::be:fcff:fea0:2bca/64

ip route:

default via 1.2.3.4 dev vmbr0 proto kernel onlink
10.2.2.0/24 via 192.168.0.2 dev vmbr1
192.168.0.0/30 dev vmbr1 proto kernel scope link src 192.168.0.1
192.168.1.0/24 via 192.168.0.2 dev vmbr1

答案1

您的规则集在转发/输出链中缺少允许回复流量的状态规则。

添加此内容应该可以通过允许有状态回复来解决问题:

iptables -I OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

请注意,在iptables脚本中,此允许回复(使用无状态规则)的命令无效,并且不会重新出现在规则集中,因为它已被注释掉:

# Allow PROXMOX WebUI
#iptables -A OUTPUT -o $PrxPubVBR -s $PublicIP -p tcp --sport 8006 -j ACCEPT

相关内容