☰

“ip rule to” 有效，但是“ip rule fwmark”失败 - 为什么？

tag-icon • tag-icon iptables iproute2
“ip rule to” 有效，但是“ip rule fwmark”失败 - 为什么？

我有一个 CentOS 6（内核 2.6.32）路由器，上面有可运行的 OpenVPN 客户端，我想通过 VPN 服务器重定向一些流量。

客户端 ( 192.168.60.159) 向路由器 ( ) 发送请求192.168.60.6:1443，路由器通过 VPN 连接 ( 10.200.0.54) 将其重定向至服务器 ( 185.61.149.21:443)。

我创建了一个特定的路由表tunde和一个规则，用于在 iptables 标记的数据包上使用此表。此规则在 VPN 上有默认网关，而主路由表有真实网关。当我尝试使用时ip rule add fwmark 0x64 lookup tunde，它失败了。请求成功传递到服务器，服务器回复，路由器收到回复但未将其传递给客户端。

但如果我补充一点ip rule add to 185.61.149.21 lookup tunde——一切完美. （但这条规则不能满足我的需求，我需要每个端口的路由）

看起来 iptables 无法以某种方式揭开回复的伪装。

有什么想法吗？谢谢！

#ip rule ls
32765:  from all fwmark 0x64 lookup tunde

该路由表在 VPN 上有默认网关：

#ip route ls table tunde
10.200.0.53 dev tunde  proto kernel  scope link  src 10.200.0.54
192.168.60.0/24 dev eth0  proto kernel  scope link  src 192.168.60.6
default via 10.200.0.53 dev tunde  src 10.200.0.54

而主路由表有真正的默认网关：

# ip route ls
10.200.0.53 dev tunde  proto kernel  scope link  src 10.200.0.54
192.168.60.0/24 dev eth0  proto kernel  scope link  src 192.168.60.6
default via 192.168.60.1 dev eth0

Tcpdump 显示 TUN 接口上的请求和回复：

# tcpdump -i tunde -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tunde, link-type RAW (Raw IP), capture size 65535 bytes
15:50:37.136708 IP 10.200.0.54.51409 > 185.61.149.21.443: Flags [S], seq 302961260, win 65280, options [mss 1360,nop,wscale 8,nop,nop,sackOK], length 0
15:50:37.209278 IP 185.61.149.21.443 > 10.200.0.54.51409: Flags [S.], seq 390829204, ack 302961261, win 29200, options [mss 1357,nop,nop,sackOK,nop,wscale 9], length 0
15:50:38.219458 IP 185.61.149.21.443 > 10.200.0.54.51409: Flags [S.], seq 390829204, ack 302961261, win 29200, options [mss 1357,nop,nop,sackOK,nop,wscale 9], length 0
15:50:40.136933 IP 10.200.0.54.51409 > 185.61.149.21.443: Flags [S], seq 302961260, win 65280, options [mss 1360,nop,wscale 8,nop,nop,sackOK], length 0
15:50:40.182989 IP 185.61.149.21.443 > 10.200.0.54.51409: Flags [S.], seq 390829204, ack 302961261, win 29200, options [mss 1357,nop,nop,sackOK,nop,wscale 9], length 0
15:50:42.191772 IP 185.61.149.21.443 > 10.200.0.54.51409: Flags [S.], seq 390829204, ack 302961261, win 29200, options [mss 1357,nop,nop,sackOK,nop,wscale 9], length 0
15:50:42.892051 IP 185.61.149.21.443 > 10.200.0.54.51391: Flags [S.], seq 528990609, ack 3345061728, win 29200, options [mss 1357,nop,nop,sackOK,nop,wscale 9], length 0

在 iptables 的日志中，我也看到了回复。所有数据包都已标记并使用正确的路由表：

# tail -f /var/log/messages
Oct 10 15:50:37 toy2 kernel: INPUT from Cli:  IN=eth0 OUT= MAC=00:15:5d:3c:bc:03:1c:39:47:f0:74:87:08:00 SRC=192.168.60.159 DST=192.168.60.6 LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=19050 DF PROTO=TCP SPT=51409 DPT=1443 WINDOW=65280 RES=0x00 SYN URGP=0 MARK=0x64
Oct 10 15:50:37 toy2 kernel: Forward To TUN:   IN=eth0 OUT=tunde SRC=192.168.60.159 DST=185.61.149.21 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=19050 DF PROTO=TCP SPT=51409 DPT=443 WINDOW=65280 RES=0x00 SYN URGP=0 MARK=0x64
Oct 10 15:50:37 toy2 kernel: OUTPUT To TUN:  IN= OUT=tunde SRC=192.168.60.159 DST=185.61.149.21 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=19050 DF PROTO=TCP SPT=51409 DPT=443 WINDOW=65280 RES=0x00 SYN URGP=0 MARK=0x64
Oct 10 15:50:37 toy2 kernel: INPUT from TUN:  IN=tunde OUT= MAC= SRC=185.61.149.21 DST=10.200.0.54 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=443 DPT=51409 WINDOW=29200 RES=0x00 ACK SYN URGP=0 MARK=0x64
Oct 10 15:50:38 toy2 kernel: INPUT from TUN:  IN=tunde OUT= MAC= SRC=185.61.149.21 DST=10.200.0.54 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=443 DPT=51409 WINDOW=29200 RES=0x00 ACK SYN URGP=0 MARK=0x64
Oct 10 15:50:40 toy2 kernel: INPUT from Cli:  IN=eth0 OUT= MAC=00:15:5d:3c:bc:03:1c:39:47:f0:74:87:08:00 SRC=192.168.60.159 DST=192.168.60.6 LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=19063 DF PROTO=TCP SPT=51409 DPT=1443 WINDOW=65280 RES=0x00 SYN URGP=0 MARK=0x64
Oct 10 15:50:40 toy2 kernel: Forward To TUN:   IN=eth0 OUT=tunde SRC=192.168.60.159 DST=185.61.149.21 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=19063 DF PROTO=TCP SPT=51409 DPT=443 WINDOW=65280 RES=0x00 SYN URGP=0 MARK=0x64
Oct 10 15:50:40 toy2 kernel: INPUT from TUN:  IN=tunde OUT= MAC= SRC=185.61.149.21 DST=10.200.0.54 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=443 DPT=51409 WINDOW=29200 RES=0x00 ACK SYN URGP=0 MARK=0x64
Oct 10 15:50:42 toy2 kernel: INPUT from TUN:  IN=tunde OUT= MAC= SRC=185.61.149.21 DST=10.200.0.54 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=443 DPT=51409 WINDOW=29200 RES=0x00 ACK SYN URGP=0 MARK=0x64
Oct 10 15:50:42 toy2 kernel: INPUT from TUN:  IN=tunde OUT= MAC= SRC=185.61.149.21 DST=10.200.0.54 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=443 DPT=51391 WINDOW=29200 RES=0x00 ACK SYN URGP=0 MARK=0x64

（查看传出和传入数据包上的 MARK=0x64）

现在，iptables 规则：

# iptables-save

*mangle
:PREROUTING ACCEPT [6278:3182515]
:INPUT ACCEPT [4169:3043489]
:FORWARD ACCEPT [9:468]
:OUTPUT ACCEPT [677:98865]
:POSTROUTING ACCEPT [703:101438]
-A PREROUTING -d 192.168.60.6/32 -i eth0 -p tcp -m tcp --dport 1443 -m state --state NEW -j CONNMARK --set-xmark 0x64/0xffffffff
-A PREROUTING -m state --state RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A PREROUTING -m connmark --mark 0x64 -j MARK --set-xmark 0x64/0xffffffff
-A PREROUTING -m state --state NEW -m connmark ! --mark 0x0 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A PREROUTING -i tunde -j LOG --log-prefix " INPUT from TUN:  "
-A PREROUTING -s 192.168.60.159/32 -i eth0 -p tcp -m tcp --dport 1443 -j LOG --log-prefix " INPUT from Cli:  "
COMMIT

*nat
:PREROUTING ACCEPT [3487:307264]
:POSTROUTING ACCEPT [57:13668]
:OUTPUT ACCEPT [57:13668]
-A PREROUTING -d 192.168.60.6/32 -i eth0 -p tcp -m tcp --dport 1443 -j DNAT --to-destination 185.61.149.21:443
-A POSTROUTING -o tunde -j LOG --log-prefix " OUTPUT To TUN:  "
-A POSTROUTING -d 192.168.60.159/32 -o eth0 -p tcp -m tcp --sport 1443 -j LOG --log-prefix " OUTPUT To Cli:  "
-A POSTROUTING -o tun+ -j MASQUERADE
COMMIT

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [680:99605]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m multiport --dports 22,23 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 1443 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -i tunde -j LOG --log-prefix " Forward From TUN: "
-A FORWARD -o tunde -j LOG --log-prefix " Forward To TUN:   "
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -p icmp -j ACCEPT
-A FORWARD -i eth0 -o tun+ -j ACCEPT
-A FORWARD -i tun+ -o eth0 -j ACCEPT
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT

答案1

解决了！

通过详细的日志记录，我发现数据包在mangle/预路由并且不进入nat/预路由。这是因为内核默认处理“不需要的”数据包。

一个

# echo 0 >/proc/sys/net/ipv4/conf/$interface/rp_filter

禁用此过滤机制，一切恢复正常运行。

相关内容