LVS-DR 和 OpenVPN:返回未被路由到 VPN 的数据包

LVS-DR 和 OpenVPN:返回未被路由到 VPN 的数据包

我已经在另外两台真实服务器上使用 ldirectord 设置了负载平衡器,以及由它管理的主动-被动 mysql 服务。

LVS 以直接路由模式运行,因此客户端请求被直接路由到当前活动的真实服务器。

真实服务器ip:192.168.3.41
虚拟ip:192.168.3.100
客户端ip(我的工作站):192.168.100.117

我还在我们的本地办公室网关和负载均衡器之间建立了一个 OpenVPN PtP 隧道,以便远程管理主机。

负载均衡器 VPN IP:172.16.3.1
客户端 VPN 端点 IP:172.16.3.2

这就是问题所在:如果我尝试使用真实 ip 连接到 mysql,一切正常,但我需要能够使用虚拟 ip 进行连接,但这样做不行。

进行了一些网络故障排除,似乎在调用虚拟 IP 时,返回的数据包没有被路由到 VPN 隧道中,真实服务器正确回复,但当返回负载均衡器时数据包流停止。

在客户端上启动 NetCat 监听任意端口,并尝试使用其真实 IP 作为源 IP 从真实服务器进行连接,但不能使用虚拟 IP 作为源。

此外,如果我尝试使用相同的虚拟 IP 连接到在负载均衡器本身上运行的服务(端口 80 上的 lighttpd),它就可以工作了!

出于这个原因,我怀疑与 ldirectord 有关的事情,但我无法弄清楚是什么!

总结一下:

客户端在端口 3306 上调用 192.168.3.41:工作正常。
客户端在端口 80 上调用 192.168.3.100(负载均衡器上的本地 lighttpd):工作正常。
客户端在端口 3306 上调用 192.168.3.100:不工作,数据包从真实服务器返回到负载均衡器,未被路由到 vpn。

这些是我的负载均衡器配置:

前向链:

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  95M   68G ACCEPT     all  --  *      *       192.168.3.0/24       0.0.0.0/0           
  58M 8719M ACCEPT     all  --  tun+   *       0.0.0.0/0            0.0.0.0/0           
  17M   16G ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0               state RELATED,ESTABLISHED 
    0     0 DROP  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

后路由链:

Chain POSTROUTING (policy ACCEPT 1377K packets, 84M bytes)
 pkts bytes target     prot opt in     out     source               destination          
 862K   51M MASQUERADE  all  --  *      *       192.168.3.0/24       0.0.0.0/0           
 101K 6159K MASQUERADE  all  --  *      bond1   172.16.3.0/24        0.0.0.0/0           

BOND 1 接口:

# ifconfig bond1
bond1     Link encap:Ethernet  HWaddr 00:22:19:d5:6d:da  
          inet addr:192.168.3.11  Bcast:192.168.3.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:28854406358 errors:0 dropped:356 overruns:0 frame:0
          TX packets:27339806726 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:11071114840327 (10.0 TiB)  TX bytes:6071397046829 (5.5 TiB)

TUN 0 接口:

# ifconfig tun0
tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:172.16.3.1  P-t-P:172.16.3.2  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:6047779884 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7219390246 errors:0 dropped:3234101 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:1060763811007 (987.9 GiB)  TX bytes:5651690621029 (5.1 TiB)

路由表:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.16.3.2      0.0.0.0         255.255.255.255 UH    0      0        0 tun0
192.168.100.0   172.16.3.2      255.255.255.0   UG    0      0        0 tun0
192.168.3.0     0.0.0.0         255.255.255.0   U     0      0        0 bond1
172.16.3.0      172.16.3.2      255.255.255.0   UG    0      0        0 tun0
0.0.0.0         xxx.xxx.xxx.xxx 0.0.0.0         UG    0      0        0 bond0

内核参数:

# sysctl -a|grep forw
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.lo.forwarding = 1
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.eth0.forwarding = 1
net.ipv4.conf.eth0.mc_forwarding = 0
net.ipv4.conf.eth1.forwarding = 1
net.ipv4.conf.eth1.mc_forwarding = 0
net.ipv4.conf.eth2.forwarding = 1
net.ipv4.conf.eth2.mc_forwarding = 0
net.ipv4.conf.eth3.forwarding = 1
net.ipv4.conf.eth3.mc_forwarding = 0
net.ipv4.conf.bond0.forwarding = 1
net.ipv4.conf.bond0.mc_forwarding = 0
net.ipv4.conf.bond1.forwarding = 1
net.ipv4.conf.bond1.mc_forwarding = 0
net.ipv4.conf.bond2.forwarding = 1
net.ipv4.conf.bond2.mc_forwarding = 0
net.ipv4.conf.tun0.forwarding = 1
net.ipv4.conf.tun0.mc_forwarding = 0
net.ipv4.ip_forward = 1

# sysctl -a|grep arp
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.arp_announce = 0
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_accept = 0
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.arp_announce = 0
net.ipv4.conf.default.arp_ignore = 0
net.ipv4.conf.default.arp_accept = 0
net.ipv4.conf.lo.proxy_arp = 0
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.arp_announce = 0
net.ipv4.conf.lo.arp_ignore = 0
net.ipv4.conf.lo.arp_accept = 0
net.ipv4.conf.eth0.proxy_arp = 0
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.arp_announce = 0
net.ipv4.conf.eth0.arp_ignore = 0
net.ipv4.conf.eth0.arp_accept = 0
net.ipv4.conf.eth1.proxy_arp = 0
net.ipv4.conf.eth1.arp_filter = 0
net.ipv4.conf.eth1.arp_announce = 0
net.ipv4.conf.eth1.arp_ignore = 0
net.ipv4.conf.eth1.arp_accept = 0
net.ipv4.conf.eth2.proxy_arp = 0
net.ipv4.conf.eth2.arp_filter = 0
net.ipv4.conf.eth2.arp_announce = 0
net.ipv4.conf.eth2.arp_ignore = 0
net.ipv4.conf.eth2.arp_accept = 0
net.ipv4.conf.eth3.proxy_arp = 0
net.ipv4.conf.eth3.arp_filter = 0
net.ipv4.conf.eth3.arp_announce = 0
net.ipv4.conf.eth3.arp_ignore = 0
net.ipv4.conf.eth3.arp_accept = 0
net.ipv4.conf.bond0.proxy_arp = 0
net.ipv4.conf.bond0.arp_filter = 0
net.ipv4.conf.bond0.arp_announce = 0
net.ipv4.conf.bond0.arp_ignore = 0
net.ipv4.conf.bond0.arp_accept = 0
net.ipv4.conf.bond1.proxy_arp = 0
net.ipv4.conf.bond1.arp_filter = 0
net.ipv4.conf.bond1.arp_announce = 0
net.ipv4.conf.bond1.arp_ignore = 0
net.ipv4.conf.bond1.arp_accept = 0
net.ipv4.conf.bond2.proxy_arp = 0
net.ipv4.conf.bond2.arp_filter = 0
net.ipv4.conf.bond2.arp_announce = 0
net.ipv4.conf.bond2.arp_ignore = 0
net.ipv4.conf.bond2.arp_accept = 0
net.ipv4.conf.tun0.proxy_arp = 0
net.ipv4.conf.tun0.arp_filter = 0
net.ipv4.conf.tun0.arp_announce = 0
net.ipv4.conf.tun0.arp_ignore = 0
net.ipv4.conf.tun0.arp_accept = 0

提前感谢你的帮助。

相关内容