我有一个 Django/Nginx/Gunicorn Web 服务器(“web02”)和一个 Nginx 文件服务器(“fs02”),用于存储用户图像。当用户通过网站上传图像时,它们会通过一个目录保存到文件服务器,该目录通过 NFS 从文件服务器交叉挂载。我使用 Ansible playbook 构建服务器,该剧本先配置每个服务器,然后首先配置文件服务器,然后再配置文件服务器。当我最初构建服务器时,NFS 运行良好。但是,如果我仅重建和重新配置文件服务器(例如,如果它崩溃并且我需要重建和恢复它),NFS 不起作用。在这种情况下,我的 Web 服务器无法看到文件服务器上导出的目录。我已经通过两种方式确认了这一点:
# From web02
$ sudo rpcinfo -u fs02 mountd
rpcinfo: RPC: Unable to receive; errno = Connection refused
program 100005 version 0 is not available
$ sudo showmount -e fs02
rpc mount export: RPC: Unable to receive; errno = Connection refused
如果我关闭文件服务器上的防火墙并重新运行上述两个命令,它们将成功运行,我可以挂载文件系统。但是,如果我重新启用防火墙,这两个命令将再次失败。令人困惑的是,我在重建文件服务器时启用的防火墙规则与最初构建文件服务器时启用的规则相同,因为规则文件是由我的 Ansible 剧本构建的。这些规则如下:
*filter
# Allow all loopback (lo0) traffic and reject traffic
# to localhost that does not originate from lo0.
-A INPUT -i lo -j ACCEPT
-A INPUT ! -i lo -s 127.0.0.0/8 -j REJECT
# Allow ping.
-A INPUT -p icmp -m state --state NEW --icmp-type 8 -j ACCEPT
# Allow SSH connections.
-A INPUT -p tcp --dport 22 -m state --state NEW -j ACCEPT
# Allow HTTP and HTTPS connections from anywhere
# (the normal ports for web servers).
-A INPUT -p tcp --dport 80 -m state --state NEW -j ACCEPT
-A INPUT -p tcp --dport 443 -m state --state NEW -j ACCEPT
# Allow rsync from the web server
-A INPUT -p tcp -s <web-server-ip-addr> --dport 873 -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --sport 873 -m state --state ESTABLISHED -j ACCEPT
# Allow NFS from the web server
-A INPUT -s <web-server-ip-addr> -p tcp -m multiport --dport 111,2049 -j ACCEPT
-A INPUT -s <web-server-ip-addr> -p udp -m multiport --dport 111,2049 -j ACCEPT
# Allow inbound traffic from established connections.
# This includes ICMP error returns.
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# Log what was incoming but denied (optional but useful).
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables_INPUT_denied: " --log-level 7
# Reject all other inbound.
-A INPUT -j REJECT
# Log any traffic which was sent to you
# for forwarding (optional but useful).
-A FORWARD -m limit --limit 5/min -j LOG --log-prefix "iptables_FORWARD_denied: " --log-level 7
# Reject all traffic forwarding.
-A FORWARD -j REJECT
COMMIT
以下是应用上述规则后在文件服务器上执行“iptables -L”的结果:
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
REJECT all -- loopback/8 anywhere reject-with icmp-port-unreachable
ACCEPT icmp -- anywhere anywhere state NEW icmp echo-request
ACCEPT tcp -- anywhere anywhere tcp dpt:ssh state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:http state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:https state NEW
ACCEPT tcp -- li470-156.members.linode.com anywhere tcp dpt:rsync state NEW,ESTABLISHED
ACCEPT tcp -- li470-156.members.linode.com anywhere multiport dports sunrpc,nfs
ACCEPT udp -- li470-156.members.linode.com anywhere multiport dports sunrpc,nfs
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
LOG all -- anywhere anywhere limit: avg 5/min burst 5 LOG level debug prefix "iptables_INPUT_denied: "
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain FORWARD (policy ACCEPT)
target prot opt source destination
LOG all -- anywhere anywhere limit: avg 5/min burst 5 LOG level debug prefix "iptables_FORWARD_denied: "
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp spt:rsync state ESTABLISHED
文件服务器知道将共享目录导出到 Web 服务器:
# /etc/exports on fs02
/var/www/mysite.com <web-server-ip-addr>/32(rw,no_root_squash,subtree_check)
Web 服务器知道如何从文件服务器挂载共享目录:
# /etc/fstab on web02
/dev/sda / ext4 errors=remount-ro 0 1
/dev/sdb none swap sw 0 0
<file-server-ip-addr>:/var/www/mysite.com /var/www/mysite.com nfs rw 0 0
还有其他我不知道的步骤吗?
更新
如果我运行命令“sudo iptables -L -n -v”,则防火墙规则如下:
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- !lo * 127.0.0.0/8 0.0.0.0/0 reject-with icmp-port-unreachable
1 84 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW icmptype 8
10 592 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 state NEW
3 144 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 state NEW
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 state NEW
0 0 ACCEPT tcp -- * * 45.79.66.59 0.0.0.0/0 tcp dpt:873 state NEW,ESTABLISHED
6 364 ACCEPT tcp -- * * 45.79.66.59 0.0.0.0/0 multiport dports 111,2049
15 1196 ACCEPT udp -- * * 45.79.66.59 0.0.0.0/0 multiport dports 111,2049
2508 1101K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
41 1941 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 5/min burst 5 LOG flags 0 level 7 prefix "iptables_FORWARD_denied: "
0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT 1983 packets, 259K bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp spt:873 state ESTABLISHED
它们显示了 web02 的正确 IP 地址。
我还禁用了日志限制指令,并检查了 fs02 上的 /var/log/syslog 和 /var/log/messages,但没有看到任何相关信息。但是,我检查了 web02 上的 /var/log/messages,看到了以下消息:
<timestamp> debian kernel: nfs: server 45.79.65.48 not responding, timed out
此消息每五分钟重复一次。我怀疑这是文件服务器早期版本的 IP 地址,因为如果我在其上运行“whois”,我可以看到它是一个 linode.com 地址。但更有趣的是,如果我输入“grep -Rn 45.79.65.48 /etc”,我会在 /etc/mtab 文件中看到这个地址。我现在看到这是以前的文件服务器的 IP 地址,因为我在销毁并重建文件服务器之前忘记卸载文件服务器的目录。我执行了“sudo umount -l /var/www/mysite.com”来卸载它。然后我在 Web 服务器上执行了“sudo mount -a”,现在我可以看到文件服务器目录已安装到 Web 服务器上。但是,如果我在 Web 服务器上重新运行“sudo rpcinfo -u fs02 mountd”命令,我仍然会收到“连接被拒绝”消息。如果我现在看到的是跨挂载目录,我不知道如何获得该消息。我已经整晚没睡了,正在做这件事,所以也许我很累,错过了一些东西。