我们有一个由两个节点组成的 vsftpd 服务器的 Pacemaker 集群，采用主动/被动方案。我们遇到了以下行为。
- vsftpd.log 停止记录活动而 xferlog 继续记录（传输继续发生）。vsftpd.log 不显示任何错误或警告消息。
- 10 到 12 小时后，服务不可用，并且没有新的连接发生
- 我们尝试移动资源但失败了很多次，我们需要执行多次清理才能在被动节点上正确启动服务。
corsync 配置：
totem {
version: 2
secauth: off
cluster_name: myCluster
transport: udpu
}
nodelist {
node {
ring0_addr: node1
nodeid: 1
}
node {
ring0_addr: node2
nodeid: 2
}
}
quorum {
provider: corosync_votequorum
two_node: 1
}
logging {
to_logfile: yes
logfile: /var/log/cluster/corosync.log
to_syslog: yes
}
vsftpd 配置文件
anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=002
allow_ftpd_full_access
dirmessage_enable=YES
xferlog_enable=YES
connect_from_port_20=YES
xferlog_file=/var/log/xferlog
chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd/chroot_list
listen=NO
listen_ipv6=YES
pam_service_name=vsftpd
userlist_enable=YES
tcp_wrappers=YES
allow_writeable_chroot=YES
dual_log_enable=YES
vsftpd_log_file=/var/log/vsftpd.log
我发现的唯一重要的日志是由 journalctl 生成的，它显示尝试停止服务器时超时以及大量失败，说实话我不知道它们是如何发生的。
Jun 09 17:50:02 node2.company.corp pengine[4720]: notice: Stop FTP_SERVICE (node2.company.corp)
Jun 09 17:50:02 node2.company.corp pengine[4720]: notice: Calculated transition 5460, saving inputs in /var/lib/pacemaker/pengine/pe-input-1567.bz2
Jun 09 17:50:02 node2.company.corp crmd[4721]: notice: Initiating stop operation FTP_SERVICE_stop_0 locally on node2.company.corp | action 8
Jun 09 17:50:02 node2.company.corp systemd[1]: Reloading.
Jun 09 17:50:02 node2.company.corp systemd[1]: Stopping Vsftpd ftp daemon...
-- Subject: Unit vsftpd.service has begun shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit vsftpd.service has begun shutting down.
Jun 09 17:50:22 node2.company.corp lrmd[4718]: notice: Giving up on FTP_SERVICE stop (rc=196): timeout (elapsed=19988ms, remaining=12ms)
Jun 09 17:50:22 node2.company.corp crmd[4721]: error: Result of stop operation for FTP_SERVICE on node2.company.corp: Timed Out | call=255 key=FTP_SERVICE_stop_0 timeout=20000ms
Jun 09 17:50:22 node2.company.corp crmd[4721]: warning: Action 8 (FTP_SERVICE_stop_0) on node2.company.corp failed (target: 0 vs. rc: 198): Error
Jun 09 17:50:22 node2.company.corp crmd[4721]: notice: Transition aborted by operation FTP_SERVICE_stop_0 'modify' on node2.company.corp: Event failed | magic=2:198;8:5460:0:bf959758-83cf-4e4b-bd33-a360ffa08557 cib=0.449.1 source=match_graph_event:310 complete=false
Jun 09 17:50:22 node2.company.corp crmd[4721]: warning: Action 8 (FTP_SERVICE_stop_0) on node2.company.corp failed (target: 0 vs. rc: 198): Error
Jun 09 17:50:22 node2.company.corp crmd[4721]: notice: Transition 5460 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=1, Source=/var/lib/pacemaker/pengine/pe-input-1567.bz2): Complete
Jun 09 17:50:22 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:50:22 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:22 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:22 node2.company.corp pengine[4720]: notice: Calculated transition 5461, saving inputs in /var/lib/pacemaker/pengine/pe-input-1568.bz2
Jun 09 17:50:22 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:50:22 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:22 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:22 node2.company.corp pengine[4720]: warning: Forcing FTP_SERVICE away from node2.company.corp after 1000000 failures (max=1000000)
Jun 09 17:50:22 node2.company.corp pengine[4720]: notice: Calculated transition 5462, saving inputs in /var/lib/pacemaker/pengine/pe-input-1569.bz2
Jun 09 17:50:22 node2.company.corp crmd[4721]: notice: Transition 5462 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1569.bz2): Complete
Jun 09 17:50:22 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:50:26 node2.company.corp crmd[4721]: notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
Jun 09 17:50:26 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:50:26 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:26 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:26 node2.company.corp pengine[4720]: warning: Forcing FTP_SERVICE away from node2.company.corp after 1000000 failures (max=1000000)
Jun 09 17:50:26 node2.company.corp pengine[4720]: notice: Calculated transition 5463, saving inputs in /var/lib/pacemaker/pengine/pe-input-1570.bz2
Jun 09 17:50:26 node2.company.corp crmd[4721]: notice: Transition 5463 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1570.bz2): Complete
Jun 09 17:50:26 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:50:33 node2.company.corp crmd[4721]: warning: No reason to expect node 2 to be down
Jun 09 17:50:33 node2.company.corp crmd[4721]: notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
Jun 09 17:50:33 node2.company.corp crmd[4721]: warning: No reason to expect node 2 to be down
Jun 09 17:50:33 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:50:33 node2.company.corp pengine[4720]: notice: Start FTP_SERVICE (node2.company.corp)
Jun 09 17:50:33 node2.company.corp pengine[4720]: notice: Calculated transition 5464, saving inputs in /var/lib/pacemaker/pengine/pe-input-1571.bz2
Jun 09 17:50:34 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:50:34 node2.company.corp pengine[4720]: notice: Start FTP_SERVICE (node2.company.corp)
Jun 09 17:50:34 node2.company.corp pengine[4720]: notice: Calculated transition 5465, saving inputs in /var/lib/pacemaker/pengine/pe-input-1571.bz2
Jun 09 17:50:34 node2.company.corp crmd[4721]: notice: Initiating monitor operation FTP_SERVICE_monitor_0 locally on node2.company.corp | action 5
Jun 09 17:50:34 node2.company.corp crmd[4721]: notice: Result of probe operation for FTP_SERVICE on node2.company.corp: 196 (OCF_PENDING) | call=260 key=FTP_SERVICE_monitor_0 confirmed=true cib-update=5999
Jun 09 17:50:34 node2.company.corp crmd[4721]: warning: Action 5 (FTP_SERVICE_monitor_0) on node2.company.corp failed (target: 7 vs. rc: 196): Error
Jun 09 17:50:34 node2.company.corp crmd[4721]: notice: Transition aborted by operation FTP_SERVICE_monitor_0 'create' on node2.company.corp: Event failed | magic=0:196;5:5465:7:bf959758-83cf-4e4b-bd33-a360ffa08557 cib=0.451.1 source=match_graph_event:310 complete=false
Jun 09 17:50:34 node2.company.corp crmd[4721]: warning: Action 5 (FTP_SERVICE_monitor_0) on node2.company.corp failed (target: 7 vs. rc: 196): Error
Jun 09 17:50:34 node2.company.corp crmd[4721]: notice: Transition 5465 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-1571.bz2): Complete
Jun 09 17:50:34 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:50:34 node2.company.corp pengine[4720]: warning: Processing failed op monitor for FTP_SERVICE on node2.company.corp: OCF_PENDING (196)
Jun 09 17:50:34 node2.company.corp pengine[4720]: warning: Processing failed op monitor for FTP_SERVICE on node2.company.corp: OCF_PENDING (196)
Jun 09 17:50:34 node2.company.corp pengine[4720]: notice: Recover FTP_SERVICE (Started node2.company.corp)
Jun 09 17:50:34 node2.company.corp pengine[4720]: notice: Calculated transition 5466, saving inputs in /var/lib/pacemaker/pengine/pe-input-1572.bz2
Jun 09 17:50:34 node2.company.corp crmd[4721]: notice: Initiating stop operation FTP_SERVICE_stop_0 locally on node2.company.corp | action 4
Jun 09 17:50:34 node2.company.corp systemd[1]: Reloading.
Jun 09 17:50:54 node2.company.corp lrmd[4718]: notice: Giving up on FTP_SERVICE stop (rc=196): timeout (elapsed=19988ms, remaining=12ms)
Jun 09 17:50:54 node2.company.corp crmd[4721]: error: Result of stop operation for FTP_SERVICE on node2.company.corp: Timed Out | call=261 key=FTP_SERVICE_stop_0 timeout=20000ms
Jun 09 17:50:54 node2.company.corp crmd[4721]: warning: Action 4 (FTP_SERVICE_stop_0) on node2.company.corp failed (target: 0 vs. rc: 198): Error
Jun 09 17:50:54 node2.company.corp crmd[4721]: notice: Transition aborted by operation FTP_SERVICE_stop_0 'modify' on node2.company.corp: Event failed | magic=2:198;4:5466:0:bf959758-83cf-4e4b-bd33-a360ffa08557 cib=0.451.2 source=match_graph_event:310 complete=false
Jun 09 17:50:54 node2.company.corp crmd[4721]: warning: Action 4 (FTP_SERVICE_stop_0) on node2.company.corp failed (target: 0 vs. rc: 198): Error
Jun 09 17:50:54 node2.company.corp crmd[4721]: notice: Transition 5466 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=3, Source=/var/lib/pacemaker/pengine/pe-input-1572.bz2): Complete
Jun 09 17:50:54 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:50:54 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:54 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:54 node2.company.corp pengine[4720]: notice: Calculated transition 5467, saving inputs in /var/lib/pacemaker/pengine/pe-input-1573.bz2
Jun 09 17:50:54 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:50:54 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:54 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:50:54 node2.company.corp pengine[4720]: warning: Forcing FTP_SERVICE away from node2.company.corp after 1000000 failures (max=1000000)
Jun 09 17:50:54 node2.company.corp pengine[4720]: notice: Calculated transition 5468, saving inputs in /var/lib/pacemaker/pengine/pe-input-1574.bz2
Jun 09 17:50:54 node2.company.corp crmd[4721]: notice: Transition 5468 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1574.bz2): Complete
Jun 09 17:50:54 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:51:03 node2.company.corp crmd[4721]: warning: No reason to expect node 2 to be down
Jun 09 17:51:03 node2.company.corp crmd[4721]: notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
Jun 09 17:51:03 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:51:03 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:51:03 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:51:03 node2.company.corp pengine[4720]: notice: Calculated transition 5469, saving inputs in /var/lib/pacemaker/pengine/pe-input-1575.bz2
Jun 09 17:51:03 node2.company.corp crmd[4721]: notice: Transition 5469 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1575.bz2): Complete
Jun 09 17:51:03 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:51:38 node2.company.corp crmd[4721]: notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
Jun 09 17:51:38 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:51:38 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:51:38 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:51:38 node2.company.corp pengine[4720]: notice: Calculated transition 5470, saving inputs in /var/lib/pacemaker/pengine/pe-input-1576.bz2
Jun 09 17:51:39 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:51:39 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:51:39 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:51:39 node2.company.corp pengine[4720]: notice: Calculated transition 5471, saving inputs in /var/lib/pacemaker/pengine/pe-input-1577.bz2
Jun 09 17:51:39 node2.company.corp crmd[4721]: notice: Transition 5471 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1577.bz2): Complete
Jun 09 17:51:39 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:52:00 node2.company.corp crmd[4721]: warning: No reason to expect node 2 to be down
Jun 09 17:52:00 node2.company.corp crmd[4721]: notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
Jun 09 17:52:01 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:52:01 node2.company.corp pengine[4720]: notice: Start FTP_SERVICE (node2.company.corp)
Jun 09 17:52:01 node2.company.corp pengine[4720]: notice: Calculated transition 5472, saving inputs in /var/lib/pacemaker/pengine/pe-input-1578.bz2
Jun 09 17:52:01 node2.company.corp crmd[4721]: notice: Initiating monitor operation FTP_SERVICE_monitor_0 locally on node2.company.corp | action 5
Jun 09 17:52:01 node2.company.corp crmd[4721]: notice: Result of probe operation for FTP_SERVICE on node2.company.corp: 196 (OCF_PENDING) | call=266 key=FTP_SERVICE_monitor_0 confirmed=true cib-update=6016
Jun 09 17:52:01 node2.company.corp crmd[4721]: warning: Action 5 (FTP_SERVICE_monitor_0) on node2.company.corp failed (target: 7 vs. rc: 196): Error
Jun 09 17:52:01 node2.company.corp crmd[4721]: notice: Transition aborted by operation FTP_SERVICE_monitor_0 'create' on node2.company.corp: Event failed | magic=0:196;5:5472:7:bf959758-83cf-4e4b-bd33-a360ffa08557 cib=0.454.1 source=match_graph_event:310 complete=false
Jun 09 17:52:01 node2.company.corp crmd[4721]: warning: Action 5 (FTP_SERVICE_monitor_0) on node2.company.corp failed (target: 7 vs. rc: 196): Error
Jun 09 17:52:01 node2.company.corp crmd[4721]: notice: Transition 5472 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-1578.bz2): Complete
Jun 09 17:52:01 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:52:01 node2.company.corp pengine[4720]: warning: Processing failed op monitor for FTP_SERVICE on node2.company.corp: OCF_PENDING (196)
Jun 09 17:52:01 node2.company.corp pengine[4720]: warning: Processing failed op monitor for FTP_SERVICE on node2.company.corp: OCF_PENDING (196)
Jun 09 17:52:01 node2.company.corp pengine[4720]: notice: Recover FTP_SERVICE (Started node2.company.corp)
Jun 09 17:52:01 node2.company.corp pengine[4720]: notice: Calculated transition 5473, saving inputs in /var/lib/pacemaker/pengine/pe-input-1579.bz2
Jun 09 17:52:01 node2.company.corp crmd[4721]: notice: Initiating stop operation FTP_SERVICE_stop_0 locally on node2.company.corp | action 4
Jun 09 17:52:01 node2.company.corp systemd[1]: Reloading.
Jun 09 17:52:21 node2.company.corp lrmd[4718]: notice: Giving up on FTP_SERVICE stop (rc=196): timeout (elapsed=19988ms, remaining=12ms)
Jun 09 17:52:21 node2.company.corp crmd[4721]: error: Result of stop operation for FTP_SERVICE on node2.company.corp: Timed Out | call=267 key=FTP_SERVICE_stop_0 timeout=20000ms
Jun 09 17:52:21 node2.company.corp crmd[4721]: warning: Action 4 (FTP_SERVICE_stop_0) on node2.company.corp failed (target: 0 vs. rc: 198): Error
Jun 09 17:52:21 node2.company.corp crmd[4721]: notice: Transition aborted by operation FTP_SERVICE_stop_0 'modify' on node2.company.corp: Event failed | magic=2:198;4:5473:0:bf959758-83cf-4e4b-bd33-a360ffa08557 cib=0.454.2 source=match_graph_event:310 complete=false
Jun 09 17:52:21 node2.company.corp crmd[4721]: warning: Action 4 (FTP_SERVICE_stop_0) on node2.company.corp failed (target: 0 vs. rc: 198): Error
Jun 09 17:52:21 node2.company.corp crmd[4721]: notice: Transition 5473 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=3, Source=/var/lib/pacemaker/pengine/pe-input-1579.bz2): Complete
Jun 09 17:52:21 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:52:21 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:52:21 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:52:21 node2.company.corp pengine[4720]: notice: Calculated transition 5474, saving inputs in /var/lib/pacemaker/pengine/pe-input-1580.bz2
Jun 09 17:52:21 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:52:21 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:52:21 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:52:21 node2.company.corp pengine[4720]: warning: Forcing FTP_SERVICE away from node2.company.corp after 1000000 failures (max=1000000)
Jun 09 17:52:21 node2.company.corp pengine[4720]: notice: Calculated transition 5475, saving inputs in /var/lib/pacemaker/pengine/pe-input-1581.bz2
Jun 09 17:52:21 node2.company.corp crmd[4721]: notice: Transition 5475 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1581.bz2): Complete
Jun 09 17:52:21 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:52:58 node2.company.corp crmd[4721]: notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
Jun 09 17:52:58 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:52:58 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:52:58 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:52:58 node2.company.corp pengine[4720]: warning: Forcing FTP_SERVICE away from node2.company.corp after 1000000 failures (max=1000000)
Jun 09 17:52:58 node2.company.corp pengine[4720]: notice: Calculated transition 5476, saving inputs in /var/lib/pacemaker/pengine/pe-input-1582.bz2
Jun 09 17:52:58 node2.company.corp crmd[4721]: notice: Transition 5476 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1582.bz2): Complete
Jun 09 17:52:58 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:53:09 node2.company.corp crmd[4721]: notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
Jun 09 17:53:09 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:53:09 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:53:09 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:53:09 node2.company.corp pengine[4720]: warning: Forcing FTP_SERVICE away from node2.company.corp after 1000000 failures (max=1000000)
Jun 09 17:53:09 node2.company.corp pengine[4720]: notice: Calculated transition 5477, saving inputs in /var/lib/pacemaker/pengine/pe-input-1583.bz2
Jun 09 17:53:09 node2.company.corp crmd[4721]: notice: Transition 5477 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1583.bz2): Complete
Jun 09 17:53:09 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:53:28 node2.company.corp crmd[4721]: notice: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph
Jun 09 17:53:28 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:53:28 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:53:28 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:53:28 node2.company.corp pengine[4720]: warning: Forcing FTP_SERVICE away from node2.company.corp after 1000000 failures (max=1000000)
Jun 09 17:53:28 node2.company.corp pengine[4720]: notice: Calculated transition 5478, saving inputs in /var/lib/pacemaker/pengine/pe-input-1584.bz2
Jun 09 17:53:28 node2.company.corp pengine[4720]: notice: On loss of CCM Quorum: Ignore
Jun 09 17:53:28 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:53:28 node2.company.corp pengine[4720]: warning: Processing failed op stop for FTP_SERVICE on node2.company.corp: OCF_TIMEOUT (198)
Jun 09 17:53:28 node2.company.corp pengine[4720]: warning: Forcing FTP_SERVICE away from node2.company.corp after 1000000 failures (max=1000000)
Jun 09 17:53:28 node2.company.corp pengine[4720]: notice: Calculated transition 5479, saving inputs in /var/lib/pacemaker/pengine/pe-input-1585.bz2
Jun 09 17:53:28 node2.company.corp crmd[4721]: notice: Transition 5479 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1585.bz2): Complete
Jun 09 17:53:28 node2.company.corp crmd[4721]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE | input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd
Jun 09 17:53:31 node2.company.corp systemd[1]: vsftpd.service stop-final-sigterm timed out. Killing.
Jun 09 17:53:31 node2.company.corp systemd[1]: Stopped Vsftpd ftp daemon.
-- Subject: Unit vsftpd.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit vsftpd.service has finished shutting down.
Jun 09 17:53:31 node2.company.corp systemd[1]: Unit vsftpd.service entered failed state.
Jun 09 17:53:31 node2.company.corp systemd[1]: vsftpd.service failed.
Pacemaker 配置
Cluster Name: FTP_CLUSTER
Corosync Nodes:
node1 node2
Pacemaker Nodes:
node1 node2
Resources:
Resource: INTERNAL_FTP_VIRTUAL_IP (class=ocf provider=heartbeat type=IPaddr2)
Attributes: ip=111.55.3.25 cidr_netmask=24
Operations: start interval=0s timeout=20s (INTERNAL_FTP_VIRTUAL_IP-start-interval-0s)
stop interval=0s timeout=20s (INTERNAL_FTP_VIRTUAL_IP-stop-interval-0s)
monitor interval=10s (INTERNAL_FTP_VIRTUAL_IP-monitor-interval-10s)
Resource: INTERNAL_FTP (class=systemd type=vsftpd)
Operations: monitor interval=10s (INTERNAL_FTP-monitor-interval-10s)
Resource: TOMCAT (class=ocf provider=heartbeat type=tomcat)
Attributes: java_home=/usr/java/jdk1.8.0_181 catalina_home=/usr/local/tomcat
Operations: start interval=0s timeout=60s (TOMCAT-start-interval-0s)
stop interval=0s timeout=120s (TOMCAT-stop-interval-0s)
monitor interval=15s (TOMCAT-monitor-interval-15s)
Resource: INTERNAL_FTP_VIRTUAL_IP_APACHE (class=ocf provider=heartbeat type=IPaddr2)
Attributes: ip=111.99.3.25 cidr_netmask=32 nic=bond0.1104
Operations: start interval=0s timeout=20s (INTERNAL_FTP_VIRTUAL_IP_APACHE-start-interval-0s)
stop interval=0s timeout=20s (INTERNAL_FTP_VIRTUAL_IP_APACHE-stop-interval-0s)
monitor interval=30s (INTERNAL_FTP_VIRTUAL_IP_APACHE-monitor-interval-30s)
Stonith Devices:
Fencing Levels:
Location Constraints:
Resource: INTERNAL_FTP
Enabled on: node2 (score:INFINITY) (role: Started) (id:cli-prefer-INTERNAL_FTP)
Resource: TOMCAT
Enabled on: node2 (score:INFINITY) (role: Started) (id:cli-prefer-TOMCAT)
Resource: INTERNAL_FTP_VIRTUAL_IP
Enabled on: node1 (score:INFINITY) (id:location-INTERNAL_FTP_VIRTUAL_IP-node1-INFINITY)
Enabled on: node2 (score:INFINITY) (role: Started) (id:cli-prefer-INTERNAL_FTP_VIRTUAL_IP)
Resource: INTERNAL_FTP_VIRTUAL_IP_APACHE
Enabled on: node2 (score:INFINITY) (role: Started) (id:cli-prefer-INTERNAL_FTP_VIRTUAL_IP_APACHE)
Ordering Constraints:
start INTERNAL_FTP_VIRTUAL_IP then start INTERNAL_FTP (kind:Mandatory) (id:order-INTERNAL_FTP_VIRTUAL_IP-INTERNAL_FTP-mandatory)
Colocation Constraints:
INTERNAL_FTP with INTERNAL_FTP_VIRTUAL_IP (score:INFINITY) (id:colocation-INTERNAL_FTP-INTERNAL_FTP_VIRTUAL_IP-INFINITY)
Ticket Constraints:
Alerts:
No alerts defined
Resources Defaults:
No defaults set
Operations Defaults:
No defaults set
Cluster Properties:
cluster-infrastructure: corosync
cluster-name: FTP_CLUSTER
dc-version: 1.1.15-11.el7-e174ec8
have-watchdog: false
last-lrm-refresh: 1624555372
no-quorum-policy: ignore
stonith-enabled: false
Quorum:
Options:
版本：Oracle Linux Server 版本 7.2 vsftpd：版本 3.0.2 pacemaker-1.1.15-11.el7.x86_64