LVS 集群帮助

2024-5-26 • tag-icon

我正在尝试为一个项目设置 LVS 集群，但我对这些网络层不太了解。我遇到了一个问题，真实服务器拒绝来自控制器的任何数据包，但允许来自我的客户端的数据包到达同一端口。我在 2.6.16.60-0.21-default 内核上使用 suse linux。

导演剧本：

  
#!/bin/bash
#----------mini-rc.lvs_dr-director------------------------
#set ip_forward OFF for lvs-dr director (1 on, 0 off)
#(there is no forwarding in the conventional sense for LVS-DR)
cat       /proc/sys/net/ipv4/ip_forward
echo "0" >/proc/sys/net/ipv4/ip_forward

#director is not gw for realservers: leave icmp redirects on
echo 'setting icmp redirects (1 on, 0 off) '
echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects
cat       /proc/sys/net/ipv4/conf/all/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects
cat       /proc/sys/net/ipv4/conf/default/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/eth1/send_redirects
cat       /proc/sys/net/ipv4/conf/eth1/send_redirects

#add ethernet device and routing for VIP 192.168.1.110
/sbin/ifconfig eth1:110 172.18.9.192 broadcast 172.18.9.192 netmask 255.255.255.255
/sbin/route add -host 172.18.9.192 dev eth1:110
#listing ifconfig info for VIP 192.168.1.110
/sbin/ifconfig eth1:110

#check VIP 192.168.1.110 is reachable from self (director)
/bin/ping -c 1 172.18.9.192
#listing routing info for VIP 192.168.1.110
/bin/netstat -rn

#setup_ipvsadm_table
#clear ipvsadm table
/sbin/ipvsadm -C
#installing LVS services with ipvsadm
#add telnet to VIP with round robin scheduling
/sbin/ipvsadm -A -t 172.18.9.192:8080 -s rr

#forward telnet to realserver using direct routing with weight 1
#/sbin/ipvsadm -a -t 172.18.9.192:telnet -r 172.18.9.191 -g -w 1
/sbin/ipvsadm -a -t 172.18.9.192:8080 -r 172.18.9.191:8080 -g -w 1
#check realserver reachable from director
ping -c 1 172.18.9.191

#forward telnet to realserver using direct routing with weight 1
/sbin/ipvsadm -a -t 172.18.9.192:8080 -r 172.18.1.136:8080 -g -w 1
#check realserver reachable from director
ping -c 1 172.18.1.136

#displaying ipvsadm settings
/sbin/ipvsadm

#not installing a default gw for LVS_TYPE vs-dr
#---------------mini-rc.lvs_dr-director------------------------

真实服务器脚本

  
#!/bin/bash
#----------mini-rc.lvs_dr-realserver------------------
#installing default gw 192.168.1.254 for vs-dr
#/sbin/route add default gw 192.168.1.254
/sbin/route add default gw 172.18.9.254
#showing routing table
/bin/netstat -rn
#checking if DEFAULT_GW 192.168.1.254 is reachable
ping -c 1 172.18.9.254

#set_realserver_ip_forwarding to OFF (1 on, 0 off).
echo "0" >/proc/sys/net/ipv4/ip_forward
cat       /proc/sys/net/ipv4/ip_forward

#looking for DIP 192.168.1.9
ping -c 1 172.18.9.110

#looking for VIP (will be on director)
ping -c 1 172.18.9.192

#install_realserver_vip

/sbin/ifconfig lo 172.18.9.192 broadcast 172.18.9.192 netmask 0xffffffff up
#ifconfig output
/sbin/ifconfig lo:110
#installing route for VIP 192.168.1.110 on device lo:110
/sbin/route add -host 172.18.9.192 dev lo:110
#listing routing info for VIP 192.168.1.110
/bin/netstat -rn

#hiding interface lo:0, will not arp
echo "1" >/proc/sys/net/ipv4/conf/all/hidden
cat       /proc/sys/net/ipv4/conf/all/hidden
echo "1" >/proc/sys/net/ipv4/conf/lo/hidden
cat       /proc/sys/net/ipv4/conf/lo/hidden

#----------mini-rc.lvs_dr-realserver------------------

上述 arp 修复无法运行，因为 /hidden 不存在。我修改了 /etc/sysctl1.conf 以添加：


net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

并运行 sysctl -p

当我运行 VIP 时，在真实服务器上使用 tcpdump 的数据包如下所示：


sudo /usr/sbin/tcpdump -n -i any port 8080
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
14:37:53.617753 IP 172.16.10.74.4592 > 172.18.9.192.8080: S 3169956976:316995697                                                                              6(0) win 65535 
14:37:53.617851 IP 172.18.9.192.8080 > 172.16.10.74.4592: R 0:0(0) ack 316995697                                                                              7 win 0

当我直接运行到真实服务器时，数据包被推送：


14:58:23.595729 IP 172.16.10.74.4887 > 172.18.9.191.8080: S 2691152412:2691152412(0) win 65535 
14:58:23.595794 IP 172.18.9.191.8080 > 172.16.10.74.4887: S 3988256836:3988256836(0) ack 2691152413 win 5840 
14:58:23.602995 IP 172.16.10.74.4887 > 172.18.9.191.8080: . ack 1 win 65535
14:58:23.600264 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 1:483(482) ack 1 win 65535
14:58:23.600275 IP 172.18.9.191.8080 > 172.16.10.74.4887: . ack 483 win 6432
14:58:23.605755 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 1:175(174) ack 483 win 6432
14:58:23.622787 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 483:969(486) ack 175 win 65361
14:58:23.619738 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 175:349(174) ack 969 win 7504
14:58:23.674299 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 969:1473(504) ack 349 win 65187
14:58:23.671410 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 349:524(175) ack 1473 win 8576
14:58:23.852775 IP 172.16.10.74.4887 > 172.18.9.191.8080: . ack 524 win 65012

如果需要更多信息来帮助调试此问题，请告诉我。

答案1

在 LVS 邮件列表中发布了我的问题，第一个回复是：

在 realserver 上，httpd 是否正在监听 172.18.9.192:8080？

事实证明我已经正确设置了 LVS 集群，但是 JBoss 尚未设置来处理 VIP。

答案1

相关内容