![LVS 集群帮助](https://linux22.com/image/488070/LVS%20%E9%9B%86%E7%BE%A4%E5%B8%AE%E5%8A%A9.png)
我正在尝试为一个项目设置 LVS 集群,但我对这些网络层不太了解。我遇到了一个问题,真实服务器拒绝来自控制器的任何数据包,但允许来自我的客户端的数据包到达同一端口。我在 2.6.16.60-0.21-default 内核上使用 suse linux。
导演剧本:
#!/bin/bash
#----------mini-rc.lvs_dr-director------------------------
#set ip_forward OFF for lvs-dr director (1 on, 0 off)
#(there is no forwarding in the conventional sense for LVS-DR)
cat /proc/sys/net/ipv4/ip_forward
echo "0" >/proc/sys/net/ipv4/ip_forward
#director is not gw for realservers: leave icmp redirects on
echo 'setting icmp redirects (1 on, 0 off) '
echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects
cat /proc/sys/net/ipv4/conf/all/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects
cat /proc/sys/net/ipv4/conf/default/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/eth1/send_redirects
cat /proc/sys/net/ipv4/conf/eth1/send_redirects
#add ethernet device and routing for VIP 192.168.1.110
/sbin/ifconfig eth1:110 172.18.9.192 broadcast 172.18.9.192 netmask 255.255.255.255
/sbin/route add -host 172.18.9.192 dev eth1:110
#listing ifconfig info for VIP 192.168.1.110
/sbin/ifconfig eth1:110
#check VIP 192.168.1.110 is reachable from self (director)
/bin/ping -c 1 172.18.9.192
#listing routing info for VIP 192.168.1.110
/bin/netstat -rn
#setup_ipvsadm_table
#clear ipvsadm table
/sbin/ipvsadm -C
#installing LVS services with ipvsadm
#add telnet to VIP with round robin scheduling
/sbin/ipvsadm -A -t 172.18.9.192:8080 -s rr
#forward telnet to realserver using direct routing with weight 1
#/sbin/ipvsadm -a -t 172.18.9.192:telnet -r 172.18.9.191 -g -w 1
/sbin/ipvsadm -a -t 172.18.9.192:8080 -r 172.18.9.191:8080 -g -w 1
#check realserver reachable from director
ping -c 1 172.18.9.191
#forward telnet to realserver using direct routing with weight 1
/sbin/ipvsadm -a -t 172.18.9.192:8080 -r 172.18.1.136:8080 -g -w 1
#check realserver reachable from director
ping -c 1 172.18.1.136
#displaying ipvsadm settings
/sbin/ipvsadm
#not installing a default gw for LVS_TYPE vs-dr
#---------------mini-rc.lvs_dr-director------------------------
真实服务器脚本
#!/bin/bash
#----------mini-rc.lvs_dr-realserver------------------
#installing default gw 192.168.1.254 for vs-dr
#/sbin/route add default gw 192.168.1.254
/sbin/route add default gw 172.18.9.254
#showing routing table
/bin/netstat -rn
#checking if DEFAULT_GW 192.168.1.254 is reachable
ping -c 1 172.18.9.254
#set_realserver_ip_forwarding to OFF (1 on, 0 off).
echo "0" >/proc/sys/net/ipv4/ip_forward
cat /proc/sys/net/ipv4/ip_forward
#looking for DIP 192.168.1.9
ping -c 1 172.18.9.110
#looking for VIP (will be on director)
ping -c 1 172.18.9.192
#install_realserver_vip
/sbin/ifconfig lo 172.18.9.192 broadcast 172.18.9.192 netmask 0xffffffff up
#ifconfig output
/sbin/ifconfig lo:110
#installing route for VIP 192.168.1.110 on device lo:110
/sbin/route add -host 172.18.9.192 dev lo:110
#listing routing info for VIP 192.168.1.110
/bin/netstat -rn
#hiding interface lo:0, will not arp
echo "1" >/proc/sys/net/ipv4/conf/all/hidden
cat /proc/sys/net/ipv4/conf/all/hidden
echo "1" >/proc/sys/net/ipv4/conf/lo/hidden
cat /proc/sys/net/ipv4/conf/lo/hidden
#----------mini-rc.lvs_dr-realserver------------------
上述 arp 修复无法运行,因为 /hidden 不存在。我修改了 /etc/sysctl1.conf 以添加:
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
并运行 sysctl -p
当我运行 VIP 时,在真实服务器上使用 tcpdump 的数据包如下所示:
sudo /usr/sbin/tcpdump -n -i any port 8080
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
14:37:53.617753 IP 172.16.10.74.4592 > 172.18.9.192.8080: S 3169956976:316995697 6(0) win 65535
14:37:53.617851 IP 172.18.9.192.8080 > 172.16.10.74.4592: R 0:0(0) ack 316995697 7 win 0
当我直接运行到真实服务器时,数据包被推送:
14:58:23.595729 IP 172.16.10.74.4887 > 172.18.9.191.8080: S 2691152412:2691152412(0) win 65535
14:58:23.595794 IP 172.18.9.191.8080 > 172.16.10.74.4887: S 3988256836:3988256836(0) ack 2691152413 win 5840
14:58:23.602995 IP 172.16.10.74.4887 > 172.18.9.191.8080: . ack 1 win 65535
14:58:23.600264 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 1:483(482) ack 1 win 65535
14:58:23.600275 IP 172.18.9.191.8080 > 172.16.10.74.4887: . ack 483 win 6432
14:58:23.605755 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 1:175(174) ack 483 win 6432
14:58:23.622787 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 483:969(486) ack 175 win 65361
14:58:23.619738 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 175:349(174) ack 969 win 7504
14:58:23.674299 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 969:1473(504) ack 349 win 65187
14:58:23.671410 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 349:524(175) ack 1473 win 8576
14:58:23.852775 IP 172.16.10.74.4887 > 172.18.9.191.8080: . ack 524 win 65012
如果需要更多信息来帮助调试此问题,请告诉我。
答案1
在 LVS 邮件列表中发布了我的问题,第一个回复是:
在 realserver 上,httpd 是否正在监听 172.18.9.192:8080?
事实证明我已经正确设置了 LVS 集群,但是 JBoss 尚未设置来处理 VIP。