首先我要说的是，我知道我在这里结合了大量的东西，但有时事情就是这样的。
因此场景是：
我正在使用 squid 和 tproxy 透明地代理 HTTP 流量，我还使用 tun2socks 将此 HTTP 流量发送到上游代理，我正在使用基于策略的路由来确定将哪些流量发送到 tun2socks
各个组件都是独立工作的，但组合起来却不行。具体来说，如下表所示，似乎路由决策在 inet 路由输出挂钩和 uip nat 后路由挂钩之间发生了某种变化。我越来越相信我以某种方式触发了某种非常奇怪的错误，但我很高兴发现我做错了。
任何帮助将不胜感激！
致以诚挚问候 Nicolai Moore
正在使用的 nftables 规则：
table inet filter {
set internet_adapters {
type ifname
elements = { "ens224","tunU-ens224" }
}
set internet_ipv4_addrs {
type ipv4_addr
elements = { 10.34.1.121, 203.0.113.2 }
}
set internet_ipv4_networks {
type ipv4_addr
flags interval
auto-merge
elements = { 10.34.0.0/16, 203.0.113.2 }
}
set local_adapters {
type ifname
elements = { "ens256" }
}
set local_ipv4_addrs {
type ipv4_addr
elements = { 10.111.1.1 }
}
set local_ipv4_networks {
type ipv4_addr
flags interval
auto-merge
elements = { 10.111.1.0/24 }
}
chain prerouting {
type filter hook prerouting priority mangle; policy accept;
ct state new ip saddr 10.111.1.2 meta nftrace set 1
jump pakete-prerouting
}
chain input {
type filter hook input priority security; policy drop;
iif "lo" accept
ip protocol icmp accept
ip6 nexthdr ipv6-icmp accept
meta mark & 0x00010000 != 0x00010000 ct state established,related accept
meta mark & 0x00010000 != 0x00010000 accept
meta mark & 0x00010000 == 0x00010000 jump pakete-input
}
chain forward {
type filter hook forward priority security; policy drop;
jump pakete-forward
}
chain output {
type filter hook output priority security; policy accept;
meta mark & 0x00020000 == 0x00020000 jump pakete-output
}
chain pakete-prerouting {
counter packets 3743 bytes 1577550
meta mark set ct mark & 0xfff4fff7
meta l4proto tcp socket transparent 1
meta mark set meta mark or 0x10000
meta mark set meta mark or 0x6
iifname @internet_adapters ip daddr @internet_ipv4_networks accept
iifname @local_adapters ip daddr @local_ipv4_networks accept
iifname @local_adapters tcp dport 80 meta
mark set meta mark | 0x00010000 tproxy to :31080 accept
}
chain pakete-output {
counter packets 0 bytes 0
meta mark & 0x00000002 == 0x00000002 counter packets 0
bytes 0 accept
meta mark & 0x00000001 == 0x00000001 counter packets 0
bytes 0 accept
counter packets 0 bytes 0 reject
}
chain pakete-input {
counter packets 0 bytes 0
ct mark set meta mark
meta mark & 0x00000002 == 0x00000002 counter packets 0
bytes 0 accept
meta mark & 0x00000001 == 0x00000001 counter packets 0
bytes 0 accept
counter packets 0 bytes 0 drop
}
chain pakete-forward {
counter packets 0 bytes 0
ct mark set meta mark
meta mark & 0x00000002 == 0x00000002 counter packets 0
bytes 0 accept
counter packets 0 bytes 0 drop
}
}
table inet route {
chain output {
type route hook output priority mangle; policy accept;
meta mark set ct mark & 0xfff4fff7
ct state new ip saddr 10.111.1.2 meta nftrace set 1
}
}
table ip nat {
set internet_adapters {
type ifname
elements = { "ens224", "tunU-ens224" }
}
set local_ipv4_networks {
type ipv4_addr
flags interval
auto-merge
elements = { 10.111.1.0/24 }
}
chain postrouting {
type nat hook postrouting priority srcnat; policy accept;
jump connectionconf-postrouting
}
chain connectionconf-postrouting {
counter packets 60 bytes 3892
ip saddr @local_ipv4_networks oifname @internet_adapters
counter packets 0 bytes 0 masquerade
}
}
ip 规则
0: from all lookup local
31900: from all fwmark 0x10000/0x10000 lookup squid_tproxy
32000: from all lookup main
32766: from all ipproto tcp dport 80 lookup upstream_proxy_default
32767: from all lookup default
nft 监控 curling 的跟踪输出http://1.1.1.1
trace id 34202808 inet filter prerouting packet: iif "ens256" ether
saddr 00:50:56:a1:22:4e ether daddr 00:50:56:a1:21:94 ip saddr
10.111.1.2 ip daddr 1.1.1.1 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id
3223 ip protocol tcp ip length 60 tcp sport 50952 tcp dport 80 tcp
flags == syn tcp window 29200
trace id 34202808 inet filter prerouting rule ct state new ip saddr
10.111.1.2 meta nftrace set 1 (verdict continue)
trace id 34202808 inet filter prerouting rule jump pakete-prerouting
(verdict jump pakete-prerouting)
trace id 34202808 inet filter pakete-prerouting rule counter packets
4054 bytes 1626001 (verdict continue)
trace id 34202808 inet filter pakete-prerouting rule meta mark set ct
mark & 0xfff4fff7 (verdict continue)
trace id 34202808 inet filter pakete-prerouting rule meta mark set
meta mark | 0x00000006 (verdict continue)
trace id 34202808 inet filter pakete-prerouting rule iifname
@local_adapters tcp dport 80 meta mark set meta mark | 0x00010000
tproxy to :31080 counter packets 0 bytes 0 accept (verdict accept)
trace id 34202808 inet filter input packet: iif "ens256" ether saddr
00:50:56:a1:22:4e ether daddr 00:50:56:a1:21:94 ip saddr 10.111.1.2 ip
daddr 1.1.1.1 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 3223 ip
protocol tcp ip length 60 tcp sport 50952 tcp dport 80 tcp flags ==
syn tcp window 29200
trace id 34202808 inet filter input rule meta mark & 0x00010000 ==
0x00010000 jump pakete-input (verdict jump pakete-input)
trace id 34202808 inet filter pakete-input rule counter packets 0
bytes 0 (verdict continue)
trace id 34202808 inet filter pakete-input rule ct mark set meta mark
(verdict continue)
trace id 34202808 inet filter pakete-input rule meta mark & 0x00000002
== 0x00000002 counter packets 0 bytes 0 accept (verdict accept)
trace id 3368ad3d inet route output packet: oif "tunU-ens224" ip saddr
10.111.1.2 ip daddr 1.1.1.1 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id
41644 ip protocol tcp ip length 60 tcp sport 40955 tcp dport 80 tcp
flags == syn tcp window 64240
trace id 3368ad3d inet route output rule ct state new ip saddr
10.111.1.2 meta nftrace set 1 (verdict continue)
trace id 3368ad3d inet route output verdict continue
trace id 3368ad3d inet route output policy accept
trace id 3368ad3d inet filter output packet: oif "tunU-ens224" ip
saddr 10.111.1.2 ip daddr 1.1.1.1 ip dscp cs0 ip ecn not-ect ip ttl 64
ip id 41644 ip protocol tcp ip length 60 tcp sport 40955 tcp dport 80
tcp flags == syn tcp window 64240
trace id 3368ad3d inet filter output verdict continue
trace id 3368ad3d inet filter output policy accept
trace id 3368ad3d ip nat postrouting packet: oif "ens224" ip saddr
10.111.1.2 ip daddr 1.1.1.1 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id
41644 ip length 60 tcp sport 40955 tcp dport 80 tcp flags == syn tcp
window 64240
trace id 3368ad3d ip nat postrouting rule jump
connectionconf-postrouting (verdict jump connectionconf-postrouting)
trace id 3368ad3d ip nat connectionconf-postrouting rule counter
packets 73 bytes 4891 (verdict continue)
trace id 3368ad3d ip nat connectionconf-postrouting rule ip saddr
@local_ipv4_networks oifname @internet_adapters counter packets 0
bytes 0 masquerade (verdict accept)
希望相关版本：
- nftables 1.0.5
- 内核 5.10.105
- 鱿鱼 4.11
- iproute 5.11
答案1
经过大量内核源代码挖掘后，我发现这确实是一个内核错误。好消息是它在较新版本的内核中已修复。不确定它的具体发布时间，但修复已在最新的 5.15 版内核中出现。
如果您在阅读这篇文章时遇到了同样的问题，并且升级内核对您来说不是一个可行的选择，那么解决方法是不依赖规则中的 dport 和 ipproto。相反，在找到补丁之前，我的计划是将规则替换为与 fwmark 匹配的规则，然后在 nftables 规则中设置 fwmark。