我正在尝试使用 OpenSWAN 和 Amazon VGW 将多个 Amazon VPC（跨区域）连接在一起。路由器实例可以 ping 通两个 VPC 中的主机，并且流量正在尝试穿过路由器，但被丢弃。
编辑：当 ping 未被转发时，我看到计数器 XfrmInNoPols 递增。
在这种情况下，有两个 VPC 连接在一起，恰好位于第三个 VPC 中的实例正在执行路由并充当枢纽。我正在尝试从本质上重新实现 Transit VPC 功能（https://aws.amazon.com/blogs/aws/aws-solution-transit-vpc/)，无需 Cisco CSR 和自动化 lambda 配置。
我的问题是，集线器能够到达东边和西边，但是来自任一端的数据包都到达集线器但无法再到达更远的地方。
拓扑：
West (172.19.0.0/16) - (hub) - East (172.18.0.0/16)。Hub 通过 VGW 连接到两端，因此东/西的明文数据包永远不会离开 Hub。根据正常的 VGW 行为，两端和 HUB 之间存在两个隧道。
此配置的基础是https://github.com/patrickbcullen/Openswan-VPC，修改为支持第二组隧道。这个脚本的一个奇怪之处是它设置了一个“网络命名空间”（http://man7.org/linux/man-pages/man8/ip-netns.8.html) 来处理所有的 ipsec 和路由。
集线器可以通过 IPSEC 隧道 ping 东部和西部的节点。VGW 同意 ipsec 和 BGP 已启动，东部/西部子网可以看到传播的路由。集线器有通往东部和西部的路由。Iptables 完全打开。rp_filter 设置为 0，而 forwarding / ip_forward 在 sysctl 中设置为 1。
我在西部设置了一个 ping 生成器，它正在尝试 ping 东部。数据包到达 hub 中的 openswan 网络命名空间：
16:38:49.311665 IP 35.163.220.45 > 169.254.255.3: ESP(spi=0x0a790d98,seq=0x4f5), length 132
16:38:49.311665 IP 172.19.58.64 > 172.18.57.207: ICMP echo request, id 411, seq 1113, length 64
我在 iptables 中设置了 NFLOG / ulogd2。它显示：
RAW-PREROUTING IN=eth0 OUT= MAC=d6:fd:61:4b:73:42:6a:3a:bb:e2:33:75:08:00 SRC=172.19.58.64 DST=172.18.57.207 LEN=84 TOS=00 PREC=0x00 TTL=254 ID=49803 DF PROTO=ICMP TYPE=8 CODE=0 ID=411 SEQ=1155 MARK=0
NAT-PREROUTING IN=eth0 OUT= MAC=d6:fd:61:4b:73:42:6a:3a:bb:e2:33:75:08:00 SRC=172.19.58.64 DST=172.18.57.207 LEN=84 TOS=00 PREC=0x00 TTL=254 ID=49803 DF PROTO=ICMP TYPE=8 CODE=0 ID=411 SEQ=1155 MARK=0
但是数据包从未到达 FORWARD iptables 链：
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
从东向西 ping 也同样失败。
集线器可以 ping 通源和目标：
# ping -c 1 172.18.57.207
64 bytes from 172.18.57.207: icmp_seq=1 ttl=254 time=1.74 ms
# ping -c 1 172.19.58.64
64 bytes from 172.19.58.64: icmp_seq=1 ttl=254 time=94.3 ms
对于可能阻止来自中转集线器的数据包的任何建议？
主机是AWS EC2 AMI，最新版本：
Linux version 4.4.30-32.54.amzn1.x86_64 (mockbuild@gobi-build-60008) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Thu Nov 10 15:52:05 UTC 2016ux
Linux Openswan U2.6.37/K4.4.30-32.54.amzn1.x86_64 (netkey)
我的 iptables 规则（全部接受，仅 NFLOG）：
# Generated by iptables-save v1.4.18 on Fri Nov 18 16:40:41 2016
*mangle
:PREROUTING ACCEPT [3648:404080]
:INPUT ACCEPT [2490:306808]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1802:171212]
:POSTROUTING ACCEPT [1802:171212]
-A INPUT -j NFLOG --nflog-prefix MAN-INPUT --nflog-group 5
-A OUTPUT -j NFLOG --nflog-prefix MAN-OUTPUT --nflog-group 5
-A POSTROUTING -j NFLOG --nflog-prefix MAN-POSTROUTING --nflog-group 5
COMMIT
# Completed on Fri Nov 18 16:40:41 2016
# Generated by iptables-save v1.4.18 on Fri Nov 18 16:40:41 2016
*filter
:INPUT ACCEPT [2490:306808]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1802:171212]
-A INPUT -j NFLOG --nflog-prefix FLT-INPUT --nflog-group 5
-A OUTPUT -j NFLOG --nflog-prefix FLT-OUTPUT --nflog-group 5
COMMIT
# Completed on Fri Nov 18 16:40:41 2016
# Generated by iptables-save v1.4.18 on Fri Nov 18 16:40:41 2016
*raw
:PREROUTING ACCEPT [3648:404080]
:OUTPUT ACCEPT [1802:171212]
-A PREROUTING -j NFLOG --nflog-prefix RAW-PREROUTING --nflog-group 5
-A OUTPUT -j NFLOG --nflog-prefix RAW-OUTPUT --nflog-group 5
COMMIT
# Completed on Fri Nov 18 16:40:41 2016
# Generated by iptables-save v1.4.18 on Fri Nov 18 16:40:41 2016
*nat
:PREROUTING ACCEPT [1158:97272]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -j NFLOG --nflog-prefix NAT-PREROUTING --nflog-group 5
-A POSTROUTING -j NFLOG --nflog-prefix NAT-POSTROUTING --nflog-group 5
COMMIT
IPSec 配置：
# /etc/ipsec.conf - Openswan IPsec configuration file
#
# Manual: ipsec.conf.5
#
# Please place your own config files in /etc/ipsec.d/ ending in .conf
version 2.0 # conforms to second version of ipsec.conf specification
# basic configuration
config setup
protostack=netkey
nat_traversal=no
virtual_private=
oe=off
conn awstunnel1
authby=secret
auto=start
left=169.254.255.2
leftid=169.254.255.2
right=35.163.197.247
rightid=35.163.197.247
type=tunnel
ikelifetime=8h
keylife=1h
phase2alg=aes128-sha1;modp1024
ike=aes128-sha1
auth=esp
keyingtries=%forever
aggrmode=no
keyexchange=ike
ikev2=never
leftsubnet=169.254.12.53/30
rightsubnet=0.0.0.0/0
dpddelay=10
dpdtimeout=30
dpdaction=restart_by_peer
conn awstunnel2
authby=secret
auto=start
left=169.254.255.3
leftid=169.254.255.3
right=35.163.220.45
rightid=35.163.220.45
type=tunnel
ikelifetime=8h
keylife=1h
phase2alg=aes128-sha1;modp1024
ike=aes128-sha1
auth=esp
keyingtries=%forever
aggrmode=no
keyexchange=ike
ikev2=never
leftsubnet=169.254.12.221/30
rightsubnet=0.0.0.0/0
dpddelay=10
dpdtimeout=30
dpdaction=restart_by_peer
conn awstunnel3
authby=secret
auto=start
left=169.254.255.4
leftid=169.254.255.4
right=52.45.134.147
rightid=52.45.134.147
type=tunnel
ikelifetime=8h
keylife=1h
phase2alg=aes128-sha1;modp1024
ike=aes128-sha1
auth=esp
keyingtries=%forever
aggrmode=no
keyexchange=ike
ikev2=never
leftsubnet=169.254.47.13/30
rightsubnet=0.0.0.0/0
dpddelay=10
dpdtimeout=30
dpdaction=restart_by_peer
conn awstunnel4
authby=secret
auto=start
left=169.254.255.5
leftid=169.254.255.5
right=52.45.232.151
rightid=52.45.232.151
type=tunnel
ikelifetime=8h
keylife=1h
phase2alg=aes128-sha1;modp1024
ike=aes128-sha1
auth=esp
keyingtries=%forever
aggrmode=no
keyexchange=ike
ikev2=never
leftsubnet=169.254.47.1/30
rightsubnet=0.0.0.0/0
dpddelay=10
dpdtimeout=30
dpdaction=restart_by_peer
（我省略了秘密）
BGP 配置：
# cat /etc/quagga/bgpd.conf
hostname ip-172-28-10-214
password xx
enable password xx
!
log file /var/log/quagga/bgpd.log
debug bgp events
debug bgp zebra
debug bgp updates
debug bgp filters
debug bgp fsm
!
router bgp 65001
bgp router-id 52.55.78.109
network 169.254.12.54/30
neighbor 169.254.12.53 remote-as 7224
neighbor 169.254.12.53 soft-reconfiguration inbound
neighbor 169.254.12.53 route-map rm_peer_1_out out
network 169.254.12.222/30
neighbor 169.254.12.221 remote-as 7224
neighbor 169.254.12.221 soft-reconfiguration inbound
neighbor 169.254.12.221 route-map rm_peer_1_out out
network 169.254.47.14/30
neighbor 169.254.47.13 remote-as 7224
neighbor 169.254.47.13 soft-reconfiguration inbound
neighbor 169.254.47.13 route-map rm_peer_1_out out
network 169.254.47.2/30
neighbor 169.254.47.1 remote-as 7224
neighbor 169.254.47.1 soft-reconfiguration inbound
neighbor 169.254.47.1 route-map rm_peer_1_out out
line vty
!
ip prefix-list localprefix seq 5 permit 172.18.0.0/16
ip prefix-list remoteprefix seq 5 permit any
! Suppress the AWS AS
route-map rm_peer_1_out permit 5
match ip address prefix-list localprefix
set as-path exclude 7224
! Suppress the AWS AS, synthetically extend the AS PATH
! For any vpc that isn't in the same region
route-map rm_peer_1_out permit 6
match ip address prefix-list remoteprefix
set as-path prepend 65001
set as-path exclude 7224
! Suppress advertisement for non-VPC addresses
access-list vpcprefixes permit 172.0.0.0/8
!
路由表：
default via 169.254.255.1 dev eth0
169.254.12.52/30 dev eth0 proto kernel scope link src 169.254.12.54
169.254.12.220/30 dev eth0 proto kernel scope link src 169.254.12.222
169.254.47.0/30 dev eth0 proto kernel scope link src 169.254.47.2
169.254.47.12/30 dev eth0 proto kernel scope link src 169.254.47.14
169.254.255.0/28 dev eth0 proto kernel scope link src 169.254.255.2
172.18.0.0/16 via 169.254.47.13 dev eth0 proto zebra metric 100
172.19.0.0/16 via 169.254.12.221 dev eth0 proto zebra metric 100
系统控制：
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.eth0.forwarding = 1
net.ipv4.conf.lo.forwarding = 1
net.ipv4.conf.veth1.forwarding = 1
net.ipv4.ip_forward = 1
ipsec 自动--状态：
000 using kernel interface: netkey
000 interface lo/lo ::1
000 interface lo/lo 127.0.0.1
000 interface eth0/eth0 169.254.255.2
000 interface eth0/eth0 169.254.12.54
000 interface eth0/eth0 169.254.12.222
000 interface eth0/eth0 169.254.47.14
000 interface eth0/eth0 169.254.47.2
000 interface eth0/eth0 169.254.255.3
000 interface eth0/eth0 169.254.255.4
000 interface eth0/eth0 169.254.255.5
000 %myid = (none)
000 debug none
000
000 virtual_private (%priv):
000 - allowed 0 subnets:
000 - disallowed 0 subnets:
000 WARNING: Either virtual_private= is not specified, or there is a syntax
000 error in that line. 'left/rightsubnet=vhost:%priv' will not work!
000 WARNING: Disallowed subnets in virtual_private= is empty. If you have
000 private address space in internal use, it should be excluded!
000
[SNIP algorithms]
000
000 stats db_ops: {curr_cnt, total_cnt, maxsz} :context={0,8,64} trans={0,8,3072} attrs={0,8,2048}
000
000 "awstunnel1": 169.254.12.52/30===169.254.255.2<169.254.255.2>[+S=C]...35.163.197.247<35.163.197.247>[+S=C]===0.0.0.0/0; erouted; eroute owner: #8
000 "awstunnel1": myip=unset; hisip=unset;
000 "awstunnel1": ike_life: 28800s; ipsec_life: 3600s; rekey_margin: 540s; rekey_fuzz: 100%; keyingtries: 0
000 "awstunnel1": policy: PSK+ENCRYPT+TUNNEL+PFS+UP+SAREFTRACK+lKOD+rKOD; prio: 30,0; interface: eth0;
000 "awstunnel1": dpd: action:restart_by_peer; delay:10; timeout:30;
000 "awstunnel1": newest ISAKMP SA: #1; newest IPsec SA: #8;
000 "awstunnel1": IKE algorithms wanted: AES_CBC(7)_128-SHA1(2)_000-MODP1536(5), AES_CBC(7)_128-SHA1(2)_000-MODP1024(2); flags=-strict
000 "awstunnel1": IKE algorithms found: AES_CBC(7)_128-SHA1(2)_160-MODP1536(5), AES_CBC(7)_128-SHA1(2)_160-MODP1024(2)
000 "awstunnel1": IKE algorithm newest: AES_CBC_128-SHA1-MODP1024
000 "awstunnel1": ESP algorithms wanted: AES(12)_128-SHA1(2)_000; pfsgroup=MODP1024(2); flags=-strict
000 "awstunnel1": ESP algorithms loaded: AES(12)_128-SHA1(2)_160
000 "awstunnel1": ESP algorithm newest: AES_128-HMAC_SHA1; pfsgroup=MODP1024
000 "awstunnel2": 169.254.12.220/30===169.254.255.3<169.254.255.3>[+S=C]...35.163.220.45<35.163.220.45>[+S=C]===0.0.0.0/0; erouted; eroute owner: #7
000 "awstunnel2": myip=unset; hisip=unset;
000 "awstunnel2": ike_life: 28800s; ipsec_life: 3600s; rekey_margin: 540s; rekey_fuzz: 100%; keyingtries: 0
000 "awstunnel2": policy: PSK+ENCRYPT+TUNNEL+PFS+UP+SAREFTRACK+lKOD+rKOD; prio: 30,0; interface: eth0;
000 "awstunnel2": dpd: action:restart_by_peer; delay:10; timeout:30;
000 "awstunnel2": newest ISAKMP SA: #2; newest IPsec SA: #7;
000 "awstunnel2": IKE algorithms wanted: AES_CBC(7)_128-SHA1(2)_000-MODP1536(5), AES_CBC(7)_128-SHA1(2)_000-MODP1024(2); flags=-strict
000 "awstunnel2": IKE algorithms found: AES_CBC(7)_128-SHA1(2)_160-MODP1536(5), AES_CBC(7)_128-SHA1(2)_160-MODP1024(2)
000 "awstunnel2": IKE algorithm newest: AES_CBC_128-SHA1-MODP1024
000 "awstunnel2": ESP algorithms wanted: AES(12)_128-SHA1(2)_000; pfsgroup=MODP1024(2); flags=-strict
000 "awstunnel2": ESP algorithms loaded: AES(12)_128-SHA1(2)_160
000 "awstunnel2": ESP algorithm newest: AES_128-HMAC_SHA1; pfsgroup=MODP1024
000 "awstunnel3": 169.254.47.12/30===169.254.255.4<169.254.255.4>[+S=C]...52.45.134.147<52.45.134.147>[+S=C]===0.0.0.0/0; erouted; eroute owner: #5
000 "awstunnel3": myip=unset; hisip=unset;
000 "awstunnel3": ike_life: 28800s; ipsec_life: 3600s; rekey_margin: 540s; rekey_fuzz: 100%; keyingtries: 0
000 "awstunnel3": policy: PSK+ENCRYPT+TUNNEL+PFS+UP+SAREFTRACK+lKOD+rKOD; prio: 30,0; interface: eth0;
000 "awstunnel3": dpd: action:restart_by_peer; delay:10; timeout:30;
000 "awstunnel3": newest ISAKMP SA: #3; newest IPsec SA: #5;
000 "awstunnel3": IKE algorithms wanted: AES_CBC(7)_128-SHA1(2)_000-MODP1536(5), AES_CBC(7)_128-SHA1(2)_000-MODP1024(2); flags=-strict
000 "awstunnel3": IKE algorithms found: AES_CBC(7)_128-SHA1(2)_160-MODP1536(5), AES_CBC(7)_128-SHA1(2)_160-MODP1024(2)
000 "awstunnel3": IKE algorithm newest: AES_CBC_128-SHA1-MODP1024
000 "awstunnel3": ESP algorithms wanted: AES(12)_128-SHA1(2)_000; pfsgroup=MODP1024(2); flags=-strict
000 "awstunnel3": ESP algorithms loaded: AES(12)_128-SHA1(2)_160
000 "awstunnel3": ESP algorithm newest: AES_128-HMAC_SHA1; pfsgroup=MODP1024
000 "awstunnel4": 169.254.47.0/30===169.254.255.5<169.254.255.5>[+S=C]...52.45.232.151<52.45.232.151>[+S=C]===0.0.0.0/0; erouted; eroute owner: #6
000 "awstunnel4": myip=unset; hisip=unset;
000 "awstunnel4": ike_life: 28800s; ipsec_life: 3600s; rekey_margin: 540s; rekey_fuzz: 100%; keyingtries: 0
000 "awstunnel4": policy: PSK+ENCRYPT+TUNNEL+PFS+UP+SAREFTRACK+lKOD+rKOD; prio: 30,0; interface: eth0;
000 "awstunnel4": dpd: action:restart_by_peer; delay:10; timeout:30;
000 "awstunnel4": newest ISAKMP SA: #4; newest IPsec SA: #6;
000 "awstunnel4": IKE algorithms wanted: AES_CBC(7)_128-SHA1(2)_000-MODP1536(5), AES_CBC(7)_128-SHA1(2)_000-MODP1024(2); flags=-strict
000 "awstunnel4": IKE algorithms found: AES_CBC(7)_128-SHA1(2)_160-MODP1536(5), AES_CBC(7)_128-SHA1(2)_160-MODP1024(2)
000 "awstunnel4": IKE algorithm newest: AES_CBC_128-SHA1-MODP1024
000 "awstunnel4": ESP algorithms wanted: AES(12)_128-SHA1(2)_000; pfsgroup=MODP1024(2); flags=-strict
000 "awstunnel4": ESP algorithms loaded: AES(12)_128-SHA1(2)_160
000 "awstunnel4": ESP algorithm newest: AES_128-HMAC_SHA1; pfsgroup=MODP1024
000
000 #8: "awstunnel1":500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 881s; newest IPSEC; eroute owner; isakmp#1; idle; import:admin initiate
000 #8: "awstunnel1" [email protected] [email protected] [email protected] [email protected] ref=0 refhim=4294901761
000 #1: "awstunnel1":500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 26389s; newest ISAKMP; lastdpd=-1s(seq in:0 out:0); idle; import:admin initiate
000 #7: "awstunnel2":500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 1114s; newest IPSEC; eroute owner; isakmp#2; idle; import:admin initiate
000 #7: "awstunnel2" [email protected] [email protected] [email protected] [email protected] ref=0 refhim=4294901761
000 #2: "awstunnel2":500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 26003s; newest ISAKMP; lastdpd=-1s(seq in:0 out:0); idle; import:admin initiate
000 #5: "awstunnel3":500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 1083s; newest IPSEC; eroute owner; isakmp#3; idle; import:admin initiate
000 #5: "awstunnel3" [email protected] [email protected] [email protected] [email protected] ref=0 refhim=4294901761
000 #3: "awstunnel3":500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 26042s; newest ISAKMP; lastdpd=-1s(seq in:0 out:0); idle; import:admin initiate
000 #6: "awstunnel4":500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 864s; newest IPSEC; eroute owner; isakmp#4; idle; import:admin initiate
000 #6: "awstunnel4" [email protected] [email protected] [email protected] [email protected] ref=0 refhim=4294901761
000 #4: "awstunnel4":500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 26073s; newest ISAKMP; lastdpd=-1s(seq in:0 out:0); idle; import:admin initiate
000
BGP 状态：
# vtysh -c 'show ip bgp summary'
BGP router identifier 52.55.78.109, local AS number 65001
RIB entries 11, using 1056 bytes of memory
Peers 4, using 18 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
169.254.12.53 4 7224 185 188 0 0 0 00:30:21 1
169.254.12.221 4 7224 185 187 0 0 0 00:30:23 1
169.254.47.1 4 7224 185 188 0 0 0 00:30:22 1
169.254.47.13 4 7224 185 187 0 0 0 00:30:22 1
# vtysh -c 'show ip bgp'
BGP table version is 0, local router ID is 52.55.78.109
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 169.254.12.52/30 0.0.0.0 0 32768 i
*> 169.254.12.220/30
0.0.0.0 0 32768 i
*> 169.254.47.0/30 0.0.0.0 0 32768 i
*> 169.254.47.12/30 0.0.0.0 0 32768 i
*> 172.18.0.0 169.254.47.13 100 0 7224 i
* 169.254.47.1 200 0 7224 i
* 172.19.0.0 169.254.12.53 200 0 7224 i
*> 169.254.12.221 100 0 7224 i
命名空间内的 ip xfrm 状态（密钥已剪切）：
# ip xfrm state
src 35.163.197.247 dst 169.254.255.2
proto esp spi 0x7db002d9 reqid 16385 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) x 96
enc cbc(aes) x
anti-replay context: seq 0xfc, oseq 0x0, bitmap 0xffffffff
src 169.254.255.2 dst 35.163.197.247
proto esp spi 0x5759bbc6 reqid 16385 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) x 96
enc cbc(aes) x
anti-replay context: seq 0x0, oseq 0x180, bitmap 0x00000000
src 35.163.220.45 dst 169.254.255.3
proto esp spi 0x0a790d98 reqid 16389 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) x 96
enc cbc(aes) x
anti-replay context: seq 0x8c0, oseq 0x0, bitmap 0xffffffff
src 169.254.255.3 dst 35.163.220.45
proto esp spi 0xc817fa78 reqid 16389 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) x 96
enc cbc(aes) x
anti-replay context: seq 0x0, oseq 0x14b, bitmap 0x00000000
src 52.45.232.151 dst 169.254.255.5
proto esp spi 0x80005db1 reqid 16397 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) x 96
enc cbc(aes) x
anti-replay context: seq 0xe9, oseq 0x0, bitmap 0xffffffff
src 169.254.255.5 dst 52.45.232.151
proto esp spi 0x7f07c4fa reqid 16397 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) x 96
enc cbc(aes) x
anti-replay context: seq 0x0, oseq 0x180, bitmap 0x00000000
src 52.45.134.147 dst 169.254.255.4
proto esp spi 0x70f458c4 reqid 16393 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) x 96
enc cbc(aes) x
anti-replay context: seq 0xfc, oseq 0x0, bitmap 0xffffffff
src 169.254.255.4 dst 52.45.134.147
proto esp spi 0x98c8c16a reqid 16393 mode tunnel
replay-window 32 flag af-unspec
auth-trunc hmac(sha1) x 96
enc cbc(aes) x
anti-replay context: seq 0x0, oseq 0x17f, bitmap 0x00000000
命名空间内的 ip xfrm 策略：
# ip xfrm policy
src 169.254.12.52/30 dst 0.0.0.0/0
dir out priority 2176 ptype main
tmpl src 169.254.255.2 dst 35.163.197.247
proto esp reqid 16385 mode tunnel
src 0.0.0.0/0 dst 169.254.12.52/30
dir fwd priority 2176 ptype main
tmpl src 35.163.197.247 dst 169.254.255.2
proto esp reqid 16385 mode tunnel
src 0.0.0.0/0 dst 169.254.12.52/30
dir in priority 2176 ptype main
tmpl src 35.163.197.247 dst 169.254.255.2
proto esp reqid 16385 mode tunnel
src 169.254.12.220/30 dst 0.0.0.0/0
dir out priority 2176 ptype main
tmpl src 169.254.255.3 dst 35.163.220.45
proto esp reqid 16389 mode tunnel
src 0.0.0.0/0 dst 169.254.12.220/30
dir fwd priority 2176 ptype main
tmpl src 35.163.220.45 dst 169.254.255.3
proto esp reqid 16389 mode tunnel
src 0.0.0.0/0 dst 169.254.12.220/30
dir in priority 2176 ptype main
tmpl src 35.163.220.45 dst 169.254.255.3
proto esp reqid 16389 mode tunnel
src 169.254.47.0/30 dst 0.0.0.0/0
dir out priority 2176 ptype main
tmpl src 169.254.255.5 dst 52.45.232.151
proto esp reqid 16397 mode tunnel
src 0.0.0.0/0 dst 169.254.47.0/30
dir fwd priority 2176 ptype main
tmpl src 52.45.232.151 dst 169.254.255.5
proto esp reqid 16397 mode tunnel
src 0.0.0.0/0 dst 169.254.47.0/30
dir in priority 2176 ptype main
tmpl src 52.45.232.151 dst 169.254.255.5
proto esp reqid 16397 mode tunnel
src 169.254.47.12/30 dst 0.0.0.0/0
dir out priority 2176 ptype main
tmpl src 169.254.255.4 dst 52.45.134.147
proto esp reqid 16393 mode tunnel
src 0.0.0.0/0 dst 169.254.47.12/30
dir fwd priority 2176 ptype main
tmpl src 52.45.134.147 dst 169.254.255.4
proto esp reqid 16393 mode tunnel
src 0.0.0.0/0 dst 169.254.47.12/30
dir in priority 2176 ptype main
tmpl src 52.45.134.147 dst 169.254.255.4
proto esp reqid 16393 mode tunnel
src ::/0 dst ::/0
socket out priority 0 ptype main
src ::/0 dst ::/0
socket in priority 0 ptype main
src 0.0.0.0/0 dst 0.0.0.0/0
socket out priority 0 ptype main
[repeats snipped]
命名空间内的 ip 规则列表：
# ip rule list
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
命名空间内的 ip 地址列表：
# ip addr list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
6: eth0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether d6:fd:61:4b:73:42 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 169.254.255.2/28 scope global eth0
valid_lft forever preferred_lft forever
inet 169.254.12.54/30 scope global eth0
valid_lft forever preferred_lft forever
inet 169.254.12.222/30 scope global eth0
valid_lft forever preferred_lft forever
inet 169.254.47.14/30 scope global eth0
valid_lft forever preferred_lft forever
inet 169.254.47.2/30 scope global eth0
valid_lft forever preferred_lft forever
inet 169.254.255.3/28 scope global secondary eth0
valid_lft forever preferred_lft forever
inet 169.254.255.4/28 scope global secondary eth0
valid_lft forever preferred_lft forever
inet 169.254.255.5/28 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::d4fd:61ff:fe4b:7342/64 scope link
valid_lft forever preferred_lft forever
8: veth1@if7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 8e:9a:f6:27:83:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0
命名空间内的 ifconfig：
# ifconfig -a
eth0 Link encap:Ethernet HWaddr D6:FD:61:4B:73:42
inet addr:169.254.255.2 Bcast:0.0.0.0 Mask:255.255.255.240
inet6 addr: fe80::d4fd:61ff:fe4b:7342/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3803 errors:0 dropped:0 overruns:0 frame:0
TX packets:2076 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:570566 (557.1 KiB) TX bytes:270108 (263.7 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
veth1 Link encap:Ethernet HWaddr 8E:9A:F6:27:83:FE
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Pfkey：
# cat /proc/net/pfkey
sk RefCnt Rmem Wmem User Inode
内核配置：
CONFIG_XFRM=y
CONFIG_XFRM_ALGO=m
CONFIG_XFRM_USER=m
CONFIG_XFRM_SUB_POLICY=y
CONFIG_XFRM_MIGRATE=y
CONFIG_XFRM_STATISTICS=y
CONFIG_XFRM_IPCOMP=m
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_XFRM_MODE_TRANSPORT=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_INET6_XFRM_TUNNEL=m
CONFIG_INET6_XFRM_MODE_TRANSPORT=m
CONFIG_INET6_XFRM_MODE_TUNNEL=m
CONFIG_INET6_XFRM_MODE_BEET=m
CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION=m
CONFIG_SECURITY_NETWORK_XFRM=y
答案1
从我所看到的情况来看，您的路由似乎没有问题。而且您可以从一个方向 ping 到另一个方向，并且 ipsec 隧道已启动，这说明路由器之间显然建立了网络通信。因此，我们可以摆脱问题的第 3 层部分
在这种情况下，关键是先确定问题，然后隔离它。找到问题的最佳方法是嗅探从最后一个响应跳转的源到目的地的网络流量。什么类型的流量通过隧道？您是否有可能拥有巨型帧（MTU 高于 1500）？是否有任何存储流量，如 ISCSI 或 FcOE？这是基于 unix 的 VPN 上流量丢失的最常见原因。一些驱动程序不支持高于 1500 的 MTU。
如果是这种情况，并且驱动程序支持它，请增加隧道在所有终端（客户端和服务器）上将接口连接到 9000，查看您嗅探到的跟踪，您正在查看 2 件事。如果巨型帧可以正常使用，但您遇到很多超时，则可能是基于隧道的问题或基于软件的问题。此时，您必须确定数据包被丢弃的位置。要消除隧道问题，可以在任何设备的出口或入口、流的任何点丢弃数据包。跟踪路由（从隧道内）非常有用，否则您必须对等嗅探源和目标，直到确定丢弃的位置。如果您收到很多 TCP 重置，则它是基于软件的，因此它是第 1 层的问题，我无法帮助您解决这个问题，因为我不是程序员 ;)