使用 fwmark 将子网路由到特定路由表,直接到 ISP 和 VPN

使用 fwmark 将子网路由到特定路由表,直接到 ISP 和 VPN

介绍

我正在尝试在路由器上设置 VPN,以便拥有两个路由表。我希望所有过滤都由 iptables 完成,而 iproute 只对特定 fwmark(即通用规则)起作用。

这是将特定子网路由到 VPN 隧道

可以找到此完整项目的当前工作配置

http://wiki.alpinelinux.org/wiki/Linux_Router_with_VPN_on_a_Raspberry_Pi#VPN_Tunnel_on_specific_subnet

该解决方案的主要问题是,虽然它有效并且我目前正在使用它,但它需要在 iptables 和 ip 规则中进行过滤,因此不是很灵活。

其思想是,标记为 1 的数据包直接通过 ppp0,而标记为 2 的数据包则通过 tun0。

概括

  • ISP 表将来自 192.168.1.0/24 的所有流量路由出 ppp0
  • VPN 表将来自 192.168.2.0/24 的所有流量路由出 tun0

网络图

https://i.stack.imgur.com/xQncJ.png

首先我添加了两个路由表:

gateway:~# cat /etc/iproute2/rt_tables
1 ISP
2 VPN

路由脚本

在启动时 ppp0 启动时添加的规则。注意,我使用 pppd 钩子来保持通用性https://ppp.samba.org/pppd.html#sect13

gateway:~# cat /etc/ppp/ip-up 
#!/bin/sh
#
# This script is run by pppd when there's a successful ppp connection.
#

# Flush out any old routes when ppp0 goes down
/sbin/ip route flush table ISP

# Add a route for this subnet to the ISP table
/sbin/ip route add 192.168.1.0/24 dev eth0 table ISP prio 1

# Add a route from the ISP table
/sbin/ip rule add from ${IPLOCAL} table ISP prio 1

# Set default route to ppp0
/sbin/ip route add table ISP default via ${IPLOCAL} prio 1

VPN 启动时添加的规则。OpenVPN 还有环境变量:https://openvpn.net/index.php/open-source/documentation/manuals/65-openvpn-20x-manpage.html#lbAS

# cat /etc/openvpn/route-up-fwmark.sh 
#!/bin/sh
#
# This script is run by OpenVPN when there's a successful VPN connection.
#

# Flush out any old routes when ppp0 goes down
/sbin/ip route flush table VPN

# Add a route for this subnet to the VPN table
/sbin/ip route add 192.168.2.0/24 dev eth0 table VPN prio 2

# Add a route from the VPN table
/sbin/ip rule add from ${route_vpn_gateway} table VPN prio 2

# Set default route to tun0
/sbin/ip route add default via ${route_vpn_gateway} dev ${dev} table VPN prio 2

路由表

主要的:

gateway:~# ip route sh table main
default dev ppp0  scope link  metric 300
172.16.32.0/20 dev tun0  proto kernel  scope link  src 172.16.39.64
192.168.0.0/30 dev eth1  proto kernel  scope link  src 192.168.0.2
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.1
${IPREMOTE} dev ppp0  proto kernel  scope link  src ${IPLOCAL}

ISP:

gateway:~# ip route sh table ISP
default via ${IPLOCAL} dev ppp0
192.168.1.0/24 dev eth0  scope link

VPN:

gateway:~# ip route sh table VPN
default via 172.16.32.1 dev tun0
192.168.2.0/24 dev eth0  scope link

知识产权规则

在 /etc/network/interfaces 中我在其中一个接口下添加了以下内容:

post-up /etc/network/fwmark_2_0_subnet_rules

其中包含:

gateway:~#  cat /etc/network/fwmark_2_0_subnet_rules
#!/bin/sh

/sbin/ip rule add fwmark 1 table ISP prio 1
/sbin/ip rule add fwmark 2 table VPN prio 2

最后,上面的所有 IP 规则:

gateway:~# ip rule
0: from all lookup local
1: from all fwmark 0x1 lookup ISP
1: from <PPP IP ADDRESS> lookup ISP
2: from all fwmark 0x2 lookup VPN
2: from 172.16.32.1 lookup VPN
32766: from all lookup main
32767: from all lookup default

IPTables 规则

#########################################################################
# Advanced routing rule set
# Uses 192.168.1.0 via ISP
#      192.168.2.0 via VPN
#
# Packets to/from 192.168.1.0/24 are marked with 0x1 and routed to ISP
# Packets to/from 192.168.2.0/24 are marked with 0x2 and routed to VPN
#
# http://nerdboys.com/2006/05/05/conning-the-mark-multiwan-connections-using-iptables-mark-connmark-and-iproute2/
# http://nerdboys.com/2006/05/08/multiwan-connections-addendum/
#########################################################################

# Set up the mangle table
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]

# Restore CONNMARK to the MARK (If one doesn't exist then no mark is set
-A PREROUTING -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff

# If packet MARK is 2, then it means there is already a connection mark and the original packet came in on VPN
-A PREROUTING -s 192.168.2.0/24 -m mark --mark 0x2 -j ACCEPT

# Else MARK packet as 2
#-A PREROUTING -i tun0 -j MARK --set-xmark 0x2/0xffffffff
-A PREROUTING -i tun0 -m conntrack --ctstate NEW -m mark --mark 0x0 -j MARK --set-xmark 0x2/0xffffffff

# If packet MARK is 1, then it means there is already a connection mark and the original packet came in on ISP
-A PREROUTING -s 192.168.1.0/24 -m mark --mark 0x1 -j ACCEPT

# Else MARK packet as 1
#-A PREROUTING -i ppp0 -j MARK --set-xmark 0x1/0xffffffff
-A PREROUTING -i ppp0 -m conntrack --ctstate NEW -m mark --mark 0x0 -j MARK --set-xmark 0x1/0xffffffff

# Save MARK to CONNMARK
-A PREROUTING -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
COMMIT

# Set up the filter table
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]

# Create rule chain per input interface for forwarding packets
:FWD_ETH0 - [0:0]
:FWD_ETH1 - [0:0]
:FWD_PPP0 - [0:0]
:FWD_TUN0 - [0:0]

# Create rule chain per input interface for input packets (for host itself)
:IN_ETH0 - [0:0]
:IN_ETH1 - [0:0]
:IN_PPP0 - [0:0]
:IN_TUN0 - [0:0]

# Pass input packet to corresponded rule chain
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth0 -j IN_ETH0
-A INPUT -i eth1 -j IN_ETH1
-A INPUT -i ppp0 -j IN_PPP0
-A INPUT -i tun0 -j IN_TUN0

# TCP flag checks - block invalid flags
-A INPUT -m conntrack --ctstate INVALID -j DROP

# Log packets that are dropped in INPUT chain (useful for debugging)
-A INPUT -j LOG --log-prefix "iptables/filter/INPUT end"

# Pass forwarded packet to corresponded rule chain
-A FORWARD -i eth0 -j FWD_ETH0
-A FORWARD -i eth1 -j FWD_ETH1
-A FORWARD -i ppp0 -j FWD_PPP0
-A FORWARD -i tun0 -j FWD_TUN0

# Log packets that are dropped in FORWARD chain (useful for debugging)
-A FORWARD -j LOG --log-prefix "iptables/filter/FORWARD end"

# Forward traffic to LAN
-A FWD_ETH0 -s 192.168.1.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# Forward traffic to VPN
-A FWD_ETH0 -s 192.168.2.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# Forward SSH packets from network to modem
-A FWD_ETH1 -s 192.168.0.0/30 -d 192.168.1.0/24 -p tcp -m tcp --sport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A FWD_ETH1 -s 192.168.0.0/30 -d 192.168.2.0/24 -p tcp -m tcp --sport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# Forward traffic to ppp0 WAN port
-A FWD_PPP0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

# Forward ICMP from VPN, (breaks traceroute through VPN if you don't have this)
-A FWD_TUN0 -d 192.168.2.0/24 -p icmp -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

# Forward traffic to tun0 VPN port
-A FWD_TUN0 -d 192.168.2.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# SSH to Router
-A IN_ETH0 -s 192.168.1.0/24 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A IN_ETH0 -s 192.168.2.0/24 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# DNS to Router
-A IN_ETH0 -s 192.168.1.0/24 -p udp -m udp --dport 53 -m conntrack --ctstate NEW -j ACCEPT
-A IN_ETH0 -s 192.168.2.0/24 -p udp -m udp --dport 53 -m conntrack --ctstate NEW -j ACCEPT

# FreeRadius Client
-A IN_ETH0 -s 192.168.1.0/24 -p tcp -m tcp --dport 1812 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A IN_ETH0 -s 192.168.1.0/24 -p udp -m udp --dport 1812 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# Ubiquiti UAP Device Discovery Broadcast
-A IN_ETH0 -s 192.168.1.0/24 -p udp -m udp --dport 10001 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# NTP
-A IN_ETH0 -s 192.168.1.0/24 -p udp -m udp --dport 123 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A IN_ETH0 -s 192.168.2.0/24 -p udp -m udp --dport 123 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# Accept traffic to router on both subnets
-A IN_ETH0 -s 192.168.1.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A IN_ETH0 -s 192.168.2.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# Prevent leakages from 192.168.2.0/24 hosts when VPN goes down for some reason
-A IN_ETH0 -s 192.168.2.0/24 -o ppp0 -j REJECT --reject-with icmp-port-unreachable

# SSH To Modem from Router
-A IN_ETH1 -s 192.168.0.0/30 -d 192.168.0.0/30 -p tcp -m tcp --sport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

# Accept incoming tracked PPP0 connections
-A IN_PPP0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

# Incoming ICMP from VPN, (breaks traceroute through VPN if you don't have this)
-A IN_TUN0 -d 192.168.2.0/24 -p icmp -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

# Accept incoming tracked connections from 192.168.2.0/24 to VPN
-A IN_TUN0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
COMMIT

*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]

# Bittorrent forwarded to Linux Workstation through VPN
-A PREROUTING -i tun0 -p tcp -m tcp --dport 6881:6889 -j DNAT --to-destination 192.168.2.20
-A PREROUTING -i tun0 -p udp -m udp --dport 6881:6889 -j DNAT --to-destination 192.168.2.20

# Allows for network hosts to access the internet via VPN tunnel
-A POSTROUTING -s 192.168.2.0/24 -o tun0 -j MASQUERADE

# Allows for network hosts to access the internet via WAN port
-A POSTROUTING -s 192.168.1.0/24 -o ppp0 -j MASQUERADE
COMMIT

我怀疑问题出在我的 mangle 表上。似乎有些数据包被标记了:

gateway:~# iptables -L --line-numbers -n -v -t mangle
Chain PREROUTING (policy ACCEPT 1577 packets, 139K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     1577  139K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore
2        0     0 ACCEPT     all  --  *      *       192.168.2.0/24       0.0.0.0/0            mark match 0x2
3        0     0 MARK       all  --  tun0   *       0.0.0.0/0            0.0.0.0/0            ctstate NEW mark match 0x0 MARK set 0x2
4        0     0 ACCEPT     all  --  *      *       192.168.1.0/24       0.0.0.0/0            mark match 0x1
5      112  6720 MARK       all  --  ppp0   *       0.0.0.0/0            0.0.0.0/0            ctstate NEW mark match 0x0 MARK set 0x1
6     1577  139K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save

Chain INPUT (policy ACCEPT 758 packets, 68909 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 819 packets, 69715 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 620 packets, 99208 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 1380 packets, 166K bytes)
num   pkts bytes target     prot opt in     out     source               destination

欢迎任何其他改进,我计划用这些更改更新 AlpineLinux wiki 文章。

答案1

我终于找到了解决这个问题的方法。

首先,与上一个问题一样,编辑并添加到/etc/iproute2/rt_tables

1 ISP
2 VPN

接下来添加到/etc/网络/接口

auto eth0:2
iface eth0:2 inet static
    address 192.168.2.1
    netmask 255.255.255.0
    post-up /etc/network/fwmark_rules

创建文件/etc/network/fwmark_rules这里我们列出 fwmark 规则:

#!/bin/sh

# Normal packets to go direct out WAN
/sbin/ip rule add fwmark 1 table ISP prio 100

# Put packets destined into VPN when VPN is up
/sbin/ip rule add fwmark 2 table VPN prio 200

# Prevent packets from being routed out when VPN is down.
# This prevents packets from falling back to the main table
# that has a priority of 32766
/sbin/ip rule add prohibit fwmark 2 prio 300

如果 VPN 因优先级低于 32766 而发生故障,禁止行可防止表 VPN 中的事物回退到主服务器。您可以在4.9. 路由策略数据库(RPDB)

添加/etc/ppp/ip-up。这是我们放置 PPP 连接上线时的 IP 规则的地方。

#!/bin/sh
#
# This script is run by pppd when there's a successful ppp connection.
#

# Flush out any old rules that might be there
/sbin/ip route flush table ISP

# Add route to table from subnets on LAN
/sbin/ip route add 192.168.1.0/24 dev eth0 table ISP prio 100
/sbin/ip route add 192.168.2.0/24 dev eth0 table ISP prio 200

# Add route from IP given by ISP to the table
/sbin/ip rule add from ${IPLOCAL} table ISP prio 100

# Add a default route
/sbin/ip route add table ISP default via ${IPLOCAL} prio 100

确保添加/etc/ppp/ip-down这是我们在接口关闭时删除规则的地方,这样我们就不会得到重复。您可以在pppd man 文件 - 脚本

#!/bin/sh
#
# This script is run by pppd after the connection has ended.
#

# Delete the rules when we take the interface down
/sbin/ip rule del from ${IPLOCAL} table ISP prio 100

现在我们需要为 OpenVPN 创建路由脚本。因此编辑/etc/openvpn/route-up-fwmark.shOpenVPN也有特殊的环境变量OpenVPN-环境变量)

#!/bin/sh
#
# This script is run by OpenVPN when there's a successful VPN connection.
#

# Flush out any old rules that might be there
/sbin/ip route flush table VPN

# Add route to table from 192.168.2.0/24 subnet on LAN
/sbin/ip route add 192.168.2.0/24 dev eth0 table VPN prio 200

# Add route from VPN interface IP to the VPN table
/sbin/ip rule add from ${ifconfig_local} table VPN prio 200

# Add a default route
/sbin/ip route add default via ${ifconfig_local} dev ${dev} table VPN prio 200

在接口关闭之前删除规则/etc/openvpn/route-pre-down-fwmark.sh

#!/bin/sh
#
# This script is run by OpenVPN after the connection has ended
#

# Delete the rules when we take the interface down
/sbin/ip rule del from ${ifconfig_local} table VPN prio 200

最后你的 mangle 表应该如下所示:

*mangle

# Set default policies for table
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]

# Restore CONNMARK to the MARK (If one doesn't exist then no mark is set)
-A PREROUTING -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff

# If packet MARK is 2, then it means there is already a connection mark and the original packet came in on VPN
-A PREROUTING -s 192.168.2.0/24 -m mark --mark 0x2 -j ACCEPT

# Check exception (this is a server which when accessed on a 192.168.2.0/24 address will go out the ISP table) are 0x1
#-A PREROUTING -s 192.168.2.0/24 -d <IP_OF_EXCEPTED_SERVER>/32 -m mark --mark 0x1 -j ACCEPT

# Check packets coming from 192.168.2.0/24 are 0x2
-A PREROUTING -s 192.168.2.0/24 -j MARK --set-xmark 0x2/0xffffffff

# If packet MARK is 1, then it means there is already a connection mark and the original packet came in on ISP
-A PREROUTING -s 192.168.1.0/24 -m mark --mark 0x1 -j ACCEPT

# Check packets coming from 192.168.1.0/24 are 0x1
-A PREROUTING -s 192.168.1.0/24 -j MARK --set-xmark 0x1/0xffffffff

# Mark exception (this is a server which when accessed on a 192.168.2.0/24 address will go out the ISP table) as 0x1
#-A PREROUTING -s 192.168.2.0/24 -d <IP_OF_EXCEPTED_SERVER>/32 -j MARK --set-xmark 0x1/0xffffffff

# Save MARK to CONNMARK (remember iproute can't see CONNMARKs)
-A PREROUTING -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
COMMIT

您还需要编辑/etc/openvpn/openvpn.conf并添加

# Prevents default gateway from being set on the default routing table
route-noexec

# Allows route-up script to be executed
script-security 2

# Calls custom shell script after connection to add necessary routes
route-up /etc/openvpn/route-up-fwmark.sh
route-pre-down /etc/openvpn/route-pre-down-fwmark.sh

相关内容