我已经在另外两台真实服务器上使用 ldirectord 设置了负载平衡器,以及由它管理的主动-被动 mysql 服务。
LVS 以直接路由模式运行,因此客户端请求被直接路由到当前活动的真实服务器。
真实服务器ip:192.168.3.41
虚拟ip:192.168.3.100
客户端ip(我的工作站):192.168.100.117
我还在我们的本地办公室网关和负载均衡器之间建立了一个 OpenVPN PtP 隧道,以便远程管理主机。
负载均衡器 VPN IP:172.16.3.1
客户端 VPN 端点 IP:172.16.3.2
这就是问题所在:如果我尝试使用真实 ip 连接到 mysql,一切正常,但我需要能够使用虚拟 ip 进行连接,但这样做不行。
进行了一些网络故障排除,似乎在调用虚拟 IP 时,返回的数据包没有被路由到 VPN 隧道中,真实服务器正确回复,但当返回负载均衡器时数据包流停止。
在客户端上启动 NetCat 监听任意端口,并尝试使用其真实 IP 作为源 IP 从真实服务器进行连接,但不能使用虚拟 IP 作为源。
此外,如果我尝试使用相同的虚拟 IP 连接到在负载均衡器本身上运行的服务(端口 80 上的 lighttpd),它就可以工作了!
出于这个原因,我怀疑与 ldirectord 有关的事情,但我无法弄清楚是什么!
总结一下:
客户端在端口 3306 上调用 192.168.3.41:工作正常。
客户端在端口 80 上调用 192.168.3.100(负载均衡器上的本地 lighttpd):工作正常。
客户端在端口 3306 上调用 192.168.3.100:不工作,数据包从真实服务器返回到负载均衡器,未被路由到 vpn。
这些是我的负载均衡器配置:
前向链:
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
95M 68G ACCEPT all -- * * 192.168.3.0/24 0.0.0.0/0
58M 8719M ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
17M 16G ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
后路由链:
Chain POSTROUTING (policy ACCEPT 1377K packets, 84M bytes)
pkts bytes target prot opt in out source destination
862K 51M MASQUERADE all -- * * 192.168.3.0/24 0.0.0.0/0
101K 6159K MASQUERADE all -- * bond1 172.16.3.0/24 0.0.0.0/0
BOND 1 接口:
# ifconfig bond1
bond1 Link encap:Ethernet HWaddr 00:22:19:d5:6d:da
inet addr:192.168.3.11 Bcast:192.168.3.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:28854406358 errors:0 dropped:356 overruns:0 frame:0
TX packets:27339806726 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:11071114840327 (10.0 TiB) TX bytes:6071397046829 (5.5 TiB)
TUN 0 接口:
# ifconfig tun0
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:172.16.3.1 P-t-P:172.16.3.2 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:6047779884 errors:0 dropped:0 overruns:0 frame:0
TX packets:7219390246 errors:0 dropped:3234101 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:1060763811007 (987.9 GiB) TX bytes:5651690621029 (5.1 TiB)
路由表:
# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.3.2 0.0.0.0 255.255.255.255 UH 0 0 0 tun0
192.168.100.0 172.16.3.2 255.255.255.0 UG 0 0 0 tun0
192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 bond1
172.16.3.0 172.16.3.2 255.255.255.0 UG 0 0 0 tun0
0.0.0.0 xxx.xxx.xxx.xxx 0.0.0.0 UG 0 0 0 bond0
内核参数:
# sysctl -a|grep forw
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.lo.forwarding = 1
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.eth0.forwarding = 1
net.ipv4.conf.eth0.mc_forwarding = 0
net.ipv4.conf.eth1.forwarding = 1
net.ipv4.conf.eth1.mc_forwarding = 0
net.ipv4.conf.eth2.forwarding = 1
net.ipv4.conf.eth2.mc_forwarding = 0
net.ipv4.conf.eth3.forwarding = 1
net.ipv4.conf.eth3.mc_forwarding = 0
net.ipv4.conf.bond0.forwarding = 1
net.ipv4.conf.bond0.mc_forwarding = 0
net.ipv4.conf.bond1.forwarding = 1
net.ipv4.conf.bond1.mc_forwarding = 0
net.ipv4.conf.bond2.forwarding = 1
net.ipv4.conf.bond2.mc_forwarding = 0
net.ipv4.conf.tun0.forwarding = 1
net.ipv4.conf.tun0.mc_forwarding = 0
net.ipv4.ip_forward = 1
# sysctl -a|grep arp
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.arp_announce = 0
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_accept = 0
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.arp_announce = 0
net.ipv4.conf.default.arp_ignore = 0
net.ipv4.conf.default.arp_accept = 0
net.ipv4.conf.lo.proxy_arp = 0
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.arp_announce = 0
net.ipv4.conf.lo.arp_ignore = 0
net.ipv4.conf.lo.arp_accept = 0
net.ipv4.conf.eth0.proxy_arp = 0
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.arp_announce = 0
net.ipv4.conf.eth0.arp_ignore = 0
net.ipv4.conf.eth0.arp_accept = 0
net.ipv4.conf.eth1.proxy_arp = 0
net.ipv4.conf.eth1.arp_filter = 0
net.ipv4.conf.eth1.arp_announce = 0
net.ipv4.conf.eth1.arp_ignore = 0
net.ipv4.conf.eth1.arp_accept = 0
net.ipv4.conf.eth2.proxy_arp = 0
net.ipv4.conf.eth2.arp_filter = 0
net.ipv4.conf.eth2.arp_announce = 0
net.ipv4.conf.eth2.arp_ignore = 0
net.ipv4.conf.eth2.arp_accept = 0
net.ipv4.conf.eth3.proxy_arp = 0
net.ipv4.conf.eth3.arp_filter = 0
net.ipv4.conf.eth3.arp_announce = 0
net.ipv4.conf.eth3.arp_ignore = 0
net.ipv4.conf.eth3.arp_accept = 0
net.ipv4.conf.bond0.proxy_arp = 0
net.ipv4.conf.bond0.arp_filter = 0
net.ipv4.conf.bond0.arp_announce = 0
net.ipv4.conf.bond0.arp_ignore = 0
net.ipv4.conf.bond0.arp_accept = 0
net.ipv4.conf.bond1.proxy_arp = 0
net.ipv4.conf.bond1.arp_filter = 0
net.ipv4.conf.bond1.arp_announce = 0
net.ipv4.conf.bond1.arp_ignore = 0
net.ipv4.conf.bond1.arp_accept = 0
net.ipv4.conf.bond2.proxy_arp = 0
net.ipv4.conf.bond2.arp_filter = 0
net.ipv4.conf.bond2.arp_announce = 0
net.ipv4.conf.bond2.arp_ignore = 0
net.ipv4.conf.bond2.arp_accept = 0
net.ipv4.conf.tun0.proxy_arp = 0
net.ipv4.conf.tun0.arp_filter = 0
net.ipv4.conf.tun0.arp_announce = 0
net.ipv4.conf.tun0.arp_ignore = 0
net.ipv4.conf.tun0.arp_accept = 0
提前感谢你的帮助。