☰

描述

tag-icon • tag-icon keepalived tinc
描述

描述

配置

我有 3 个节点，使用 Tinc VPN 连接在一起，我想在其中安装 HAproxy 并拥有一个 VIP，以便 HAproxy 本身处于高可用性模式。

以下是节点的详细信息：

  • 节点 1 有 IP 地址10.0.0.222/32在界面上VPN 安全
  • 节点 2 有 IP 地址10.0.0.13/32在界面上VPN 安全
  • 节点 3 有 IP 地址10.0.0.103/32在界面上VPN 安全

为此，我keepalived在每台机器上都进行了安装。

我还启用了以下 sysctl：

net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1

节点 1 具有以下内容/etc/keepalived/keepalived.conf文件：

global_defs {
  enable_script_security
  router_id node-1
}

vrrp_script haproxy-check {
    script "/usr/bin/killall -0 haproxy"
    interval 2
    weight 2
}

vrrp_instance haproxy-vip {
    state MASTER
    priority 150
    interface vpn
    virtual_router_id 1
    advert_int 1

    virtual_ipaddress {
        10.0.0.1/32
    }

    track_script {
        haproxy-check
    }
}

节点 2 和 3 具有以下内容/etc/keepalived/keepalived.conf文件 ：

global_defs {
  enable_script_security
  router_id node-2 # Node 3 has "node-3" here.
}

vrrp_script haproxy-check {
    script "/usr/bin/killall -0 haproxy"
    interval 2
    weight 2
}

vrrp_instance haproxy-vip {
    state BACKUP
    priority 100
    interface vpn
    virtual_router_id 1
    advert_int 1

    virtual_ipaddress {
        10.0.0.1/32
    }

    track_script {
        haproxy-check
    }
}

当所有节点都运行时keepalived，节点 1 为主节点，并且 VIP10.0.0.1配置良好，其他 2 个节点对其进行 ping 操作。

节点 1 日志

启动时记录keepalived：

Dec  5 14:07:53 node-1 systemd[1]: Starting Keepalive Daemon (LVS and VRRP)...
Dec  5 14:07:53 node-1 Keepalived[5870]: Starting Keepalived v1.3.2 (12/03,2016)
Dec  5 14:07:53 node-1 systemd[1]: Started Keepalive Daemon (LVS and VRRP).
Dec  5 14:07:53 node-1 Keepalived[5870]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Dec  5 14:07:53 node-1 Keepalived[5870]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:07:53 node-1 Keepalived[5871]: Starting Healthcheck child process, pid=5872
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Initializing ipvs
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Registering Kernel netlink reflector
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Registering Kernel netlink command channel
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:07:53 node-1 Keepalived[5871]: Starting VRRP child process, pid=5873
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Registering Kernel netlink reflector
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Registering Kernel netlink command channel
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Registering gratuitous ARP shared channel
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Using LinkWatch kernel netlink reflector...
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Using LinkWatch kernel netlink reflector...
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: VRRP_Script(haproxy-check) succeeded
Dec  5 14:07:54 node-1 Keepalived_vrrp[5873]: VRRP_Instance(haproxy-vip) Transition to MASTER STATE
Dec  5 14:07:54 node-1 Keepalived_vrrp[5873]: VRRP_Instance(haproxy-vip) Changing effective priority from 150 to 152
Dec  5 14:07:55 node-1 Keepalived_vrrp[5873]: VRRP_Instance(haproxy-vip) Entering MASTER STATE
Dec  5 14:07:57 node-1 ntpd[946]: Listen normally on 45 vpn 10.0.0.1:123

节点 1 ip addr：

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.222/24 scope global vpn
   valid_lft forever preferred_lft forever
inet 10.0.0.1/24 scope global secondary vpn
   valid_lft forever preferred_lft forever

节点 2 和 3 日志

Dec  5 14:14:32 node-2 systemd[1]: Starting Keepalive Daemon (LVS and VRRP)...
Dec  5 14:14:32 node-2 Keepalived[13745]: Starting Keepalived v1.3.2 (12/03,2016)
Dec  5 14:14:32 node-2 Keepalived[13745]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Dec  5 14:14:32 node-2 Keepalived[13745]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:14:32 node-2 Keepalived[13746]: Starting Healthcheck child process, pid=13747
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Initializing ipvs
Dec  5 14:14:32 node-2 systemd[1]: Started Keepalive Daemon (LVS and VRRP).
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Registering Kernel netlink reflector
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Registering Kernel netlink command channel
Dec  5 14:14:32 node-2 Keepalived[13746]: Starting VRRP child process, pid=13748
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Registering Kernel netlink reflector
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Registering Kernel netlink command channel
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Registering gratuitous ARP shared channel
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Using LinkWatch kernel netlink reflector...
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Using LinkWatch kernel netlink reflector...
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Entering BACKUP STATE
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: VRRP_Script(haproxy-check) succeeded
Dec  5 14:14:33 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Changing effective priority from 100 to 102

节点 2 和 3 ip addr：

节点 2

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.13/24 scope global vpn
   valid_lft forever preferred_lft forever

节点 3

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.103/24 scope global vpn
   valid_lft forever preferred_lft forever

问题

但是，当我keepalived在节点 1 上停止时，节点 3 被选为主节点，并注册 VIP，并且只有节点 3 ping 10.0.0.1。

节点 1 日志

停止时：

Dec  5 14:15:26 node-1 systemd[1]: Stopping Keepalive Daemon (LVS and VRRP)...
Dec  5 14:15:26 node-1 Keepalived[5871]: Stopping
Dec  5 14:15:26 node-1 Keepalived_healthcheckers[5872]: Stopped
Dec  5 14:15:26 node-1 Keepalived_vrrp[5873]: VRRP_Instance(haproxy-vip) sent 0 priority
Dec  5 14:15:27 node-1 Keepalived_vrrp[5873]: Stopped
Dec  5 14:15:27 node-1 Keepalived[5871]: Stopped Keepalived v1.3.2 (12/03,2016)
Dec  5 14:15:27 node-1 systemd[1]: Stopped Keepalive Daemon (LVS and VRRP).
Dec  5 14:15:28 node-1 ntpd[946]: Deleting interface #45 vpn, 10.0.0.1#123, interface stats: received=0, sent=0, dropped=0, active_time=451 secs

节点 1 ip addr：

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.222/24 scope global vpn
   valid_lft forever preferred_lft forever

节点 2 日志

Dec  5 14:15:27 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Transition to MASTER STATE
Dec  5 14:15:27 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Received advert with higher priority 102, ours 102
Dec  5 14:15:27 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Entering BACKUP STATE

节点 2 ip addr：

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.13/24 scope global vpn
   valid_lft forever preferred_lft forever

节点 3 日志

Dec  5 14:15:27 node-3 Keepalived_vrrp[31252]: VRRP_Instance(haproxy-vip) Transition to MASTER STATE
Dec  5 14:15:27 node-3 Keepalived_vrrp[31252]: VRRP_Instance(haproxy-vip) Received advert with lower priority 102, ours 102, forcing new election
Dec  5 14:15:28 node-3 Keepalived_vrrp[31252]: VRRP_Instance(haproxy-vip) Entering MASTER STATE
Dec  5 14:15:29 node-3 ntpd[27734]: Listen normally on 36 vpn 10.0.0.1:123

节点 3 ip addr：

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.103/24 scope global vpn
   valid_lft forever preferred_lft forever
inet 10.0.0.1/24 scope global secondary vpn
   valid_lft forever preferred_lft forever

更多细节

路由追踪

我traceroute曾经尝试获取有关该问题的更多信息。

当所有节点都在运行keepalived，并且 ping VIP 到处都有效时，traceroute显示所有节点的情况：

$ traceroute 10.0.0.1
traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
 1  10.0.0.1 (10.0.0.1)  0.094 ms  0.030 ms  0.019 ms

当keepalived在节点 1 上停止时，并且节点 3 被选举，节点 1 无法确定 VIP 在哪里：

$ traceroute 10.0.0.1
traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 ...
 29  * * *
 30  * * *

节点 2 期望节点 1 拥有 VIP：

$ traceroute 10.0.0.1
traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
 1  10.0.0.222 (10.0.0.222)  0.791 ms  0.962 ms  1.080 ms
 2  * * *
 3  * * *
 ...

并且节点 3 具有 VIP，因此它可以工作。

Tinc 设备类型

我读了一些邮件存档，其中建议使用DeviceType = tapTinc 配置来传输 ARP 数据包（据我所知），但这没有帮助。

实际上，我不确定 Tinc 是否是根本原因，因为选举正在进行。

尝试不使用 Tinc

我改变了keepalived配置，以便它使用公共互联网接口，使用单播。

我已经在每个节点上的每个 keepalived 配置中添加了以下块（这里是node-1）：

    unicast_src_ip XXX.XXX.XXX.XXX # node's public IP address
    unicast_peer {
        XXX.XXX.XXX.XXX # other node's public IP address
        XXX.XXX.XXX.XXX # other node's public IP address
    }

但其行为与上面描述的完全相同，因此 Tinc 不应该相关。

要求

有人能帮助我找出问题所在并解决这个问题，以便当进行新的选举时，节点能够在新的位置找到 VIP？

答案1

我刚刚通过添加到我的 tinc.conf 解决了类似的问题Mode = switch。

我遇到的问题与您描述的类似；keepalived 会按预期在我的 3 个节点之间转换我设置的虚拟 IP（指向一个简单的 nginx 服务器）。但是，唯一能够访问服务的节点是选举出来的 MASTER。这是因为路由表是静态构建的，来自主机配置文件，而不是来自 ARP 数据。

我确实觉得你没有使用 tinc 的尝试失败了，这很奇怪。当我更改配置以在具有路由器的本地网络上运行时，keepalived 和 haproxy 按照预期运行，并且 vip 在路由器的 ARP 表中可见。你确定你为本地测试更改了 haproxy 和 keepalived 配置吗？

祝你好运！

参考：

  1. https://www.tinc-vpn.org/documentation/Main-configuration-variables.html
  2. https://www.tinc-vpn.org/pipermail/tinc/2010-February/002191.html

相关内容