在繁忙的服务器上，Nginx 消耗了 100% 的 RAM。我应该从哪里开始查找问题，也许有办法查看详细的内存使用情况？
nginx 版本：nginx/1.17.3
缓存繁忙：
proxy_cache_path /mnt/disk2/cfs levels=1:2 keys_zone=cfs:50m max_size=270g inactive=3h use_temp_path=off;
缓存也位于其他服务器上，对内存使用没有影响。
free -h
total used free shared buff/cache available
Mem: 7.8G 7.0G 262M 4.8M 553M 546M
Swap: 2.0G 41M 2.0G
swapon -s
Filename Type Size Used Priority
/dev/xvdb1 partition 2096444 42736 -2
24580: nginx: worker process
Address Kbytes RSS Dirty Mode Mapping
000055f7200af000 1160 520 0 r-x-- nginx
000055f7200af000 0 0 0 r-x-- nginx
000055f7203d0000 8 8 8 r---- nginx
000055f7203d0000 0 0 0 r---- nginx
000055f7203d2000 132 84 72 rw--- nginx
000055f7203d2000 0 0 0 rw--- nginx
000055f7203f3000 124 20 20 rw--- [ anon ]
000055f7203f3000 0 0 0 rw--- [ anon ]
000055f7213f3000 4412 3932 3932 rw--- [ anon ]
000055f7213f3000 0 0 0 rw--- [ anon ]
000055f721842000 3129980 3128776 3128776 rw--- [ anon ]
000055f721842000 0 0 0 rw--- [ anon ]
00007f1b212d5000 21488 16256 16256 rw--- [ anon ]
00007f1b212d5000 0 0 0 rw--- [ anon ]
00007f1b227d1000 28 28 0 r-x-- librt-2.27.so
00007f1b227d1000 0 0 0 r-x-- librt-2.27.so
00007f1b227d8000 2044 0 0 ----- librt-2.27.so
00007f1b227d8000 0 0 0 ----- librt-2.27.so
00007f1b229d7000 4 0 0 r---- librt-2.27.so
00007f1b229d7000 0 0 0 r---- librt-2.27.so
00007f1b229d8000 4 0 0 rw--- librt-2.27.so
00007f1b229d8000 0 0 0 rw--- librt-2.27.so
00007f1b229d9000 240 0 0 r-x-- libnss_systemd.so.2
00007f1b229d9000 0 0 0 r-x-- libnss_systemd.so.2
00007f1b22a15000 2044 0 0 ----- libnss_systemd.so.2
00007f1b22a15000 0 0 0 ----- libnss_systemd.so.2
00007f1b22c14000 12 0 0 r---- libnss_systemd.so.2
00007f1b22c14000 0 0 0 r---- libnss_systemd.so.2
00007f1b22c17000 4 0 0 rw--- libnss_systemd.so.2
00007f1b22c17000 0 0 0 rw--- libnss_systemd.so.2
00007f1b22c18000 10240 72 72 rw-s- zero (deleted)
00007f1b22c18000 0 0 0 rw-s- zero (deleted)
00007f1b23618000 51200 5480 5480 rw-s- zero (deleted)
00007f1b23618000 0 0 0 rw-s- zero (deleted)
00007f1b26818000 10240 636 636 rw-s- zero (deleted)
00007f1b26818000 0 0 0 rw-s- zero (deleted)
00007f1b27218000 51200 220 220 rw-s- zero (deleted)
00007f1b27218000 0 0 0 rw-s- zero (deleted)
00007f1b2a418000 92 4 0 r-x-- libresolv-2.27.so
00007f1b2a418000 0 0 0 r-x-- libresolv-2.27.so
00007f1b2a42f000 2048 0 0 ----- libresolv-2.27.so
00007f1b2a42f000 0 0 0 ----- libresolv-2.27.so
00007f1b2a62f000 4 0 0 r---- libresolv-2.27.so
00007f1b2a62f000 0 0 0 r---- libresolv-2.27.so
00007f1b2a630000 4 0 0 rw--- libresolv-2.27.so
00007f1b2a630000 0 0 0 rw--- libresolv-2.27.so
00007f1b2a631000 8 0 0 rw--- [ anon ]
00007f1b2a631000 0 0 0 rw--- [ anon ]
00007f1b2a633000 20 8 0 r-x-- libnss_dns-2.27.so
00007f1b2a633000 0 0 0 r-x-- libnss_dns-2.27.so
00007f1b2a638000 2048 0 0 ----- libnss_dns-2.27.so
00007f1b2a638000 0 0 0 ----- libnss_dns-2.27.so
00007f1b2a838000 4 0 0 r---- libnss_dns-2.27.so
00007f1b2a838000 0 0 0 r---- libnss_dns-2.27.so
00007f1b2a839000 4 0 0 rw--- libnss_dns-2.27.so
00007f1b2a839000 0 0 0 rw--- libnss_dns-2.27.so
00007f1b2a83a000 44 44 0 r-x-- libnss_files-2.27.so
00007f1b2a83a000 0 0 0 r-x-- libnss_files-2.27.so
00007f1b2a845000 2044 0 0 ----- libnss_files-2.27.so
00007f1b2a845000 0 0 0 ----- libnss_files-2.27.so
00007f1b2aa44000 4 0 0 r---- libnss_files-2.27.so
00007f1b2aa44000 0 0 0 r---- libnss_files-2.27.so
00007f1b2aa45000 4 0 0 rw--- libnss_files-2.27.so
00007f1b2aa45000 0 0 0 rw--- libnss_files-2.27.so
00007f1b2aa46000 24 0 0 rw--- [ anon ]
00007f1b2aa46000 0 0 0 rw--- [ anon ]
00007f1b2aa4c000 92 48 0 r-x-- libnsl-2.27.so
00007f1b2aa4c000 0 0 0 r-x-- libnsl-2.27.so
00007f1b2aa63000 2044 0 0 ----- libnsl-2.27.so
00007f1b2aa63000 0 0 0 ----- libnsl-2.27.so
00007f1b2ac62000 4 0 0 r---- libnsl-2.27.so
00007f1b2ac62000 0 0 0 r---- libnsl-2.27.so
00007f1b2ac63000 4 0 0 rw--- libnsl-2.27.so
00007f1b2ac63000 0 0 0 rw--- libnsl-2.27.so
00007f1b2ac64000 8 0 0 rw--- [ anon ]
00007f1b2ac64000 0 0 0 rw--- [ anon ]
00007f1b2ac66000 44 32 0 r-x-- libnss_nis-2.27.so
00007f1b2ac66000 0 0 0 r-x-- libnss_nis-2.27.so
00007f1b2ac71000 2044 0 0 ----- libnss_nis-2.27.so
00007f1b2ac71000 0 0 0 ----- libnss_nis-2.27.so
00007f1b2ae70000 4 0 0 r---- libnss_nis-2.27.so
00007f1b2ae70000 0 0 0 r---- libnss_nis-2.27.so
00007f1b2ae71000 4 0 0 rw--- libnss_nis-2.27.so
00007f1b2ae71000 0 0 0 rw--- libnss_nis-2.27.so
00007f1b2ae72000 32 24 0 r-x-- libnss_compat-2.27.so
00007f1b2ae72000 0 0 0 r-x-- libnss_compat-2.27.so
00007f1b2ae7a000 2048 0 0 ----- libnss_compat-2.27.so
00007f1b2ae7a000 0 0 0 ----- libnss_compat-2.27.so
00007f1b2b07a000 4 0 0 r---- libnss_compat-2.27.so
00007f1b2b07a000 0 0 0 r---- libnss_compat-2.27.so
00007f1b2b07b000 4 0 0 rw--- libnss_compat-2.27.so
00007f1b2b07b000 0 0 0 rw--- libnss_compat-2.27.so
00007f1b2b07c000 1948 1488 0 r-x-- libc-2.27.so
00007f1b2b07c000 0 0 0 r-x-- libc-2.27.so
00007f1b2b263000 2048 0 0 ----- libc-2.27.so
00007f1b2b263000 0 0 0 ----- libc-2.27.so
00007f1b2b463000 16 8 8 r---- libc-2.27.so
00007f1b2b463000 0 0 0 r---- libc-2.27.so
00007f1b2b467000 8 8 8 rw--- libc-2.27.so
00007f1b2b467000 0 0 0 rw--- libc-2.27.so
00007f1b2b469000 16 12 12 rw--- [ anon ]
00007f1b2b469000 0 0 0 rw--- [ anon ]
00007f1b2b46d000 112 112 0 r-x-- libz.so.1.2.11
00007f1b2b46d000 0 0 0 r-x-- libz.so.1.2.11
00007f1b2b489000 2044 0 0 ----- libz.so.1.2.11
00007f1b2b489000 0 0 0 ----- libz.so.1.2.11
00007f1b2b688000 4 4 4 r---- libz.so.1.2.11
00007f1b2b688000 0 0 0 r---- libz.so.1.2.11
00007f1b2b689000 4 4 4 rw--- libz.so.1.2.11
00007f1b2b689000 0 0 0 rw--- libz.so.1.2.11
00007f1b2b68a000 2148 724 0 r-x-- libcrypto.so.1.0.0
00007f1b2b68a000 0 0 0 r-x-- libcrypto.so.1.0.0
00007f1b2b8a3000 2044 0 0 ----- libcrypto.so.1.0.0
00007f1b2b8a3000 0 0 0 ----- libcrypto.so.1.0.0
00007f1b2baa2000 112 52 52 r---- libcrypto.so.1.0.0
00007f1b2baa2000 0 0 0 r---- libcrypto.so.1.0.0
00007f1b2babe000 48 28 28 rw--- libcrypto.so.1.0.0
00007f1b2babe000 0 0 0 rw--- libcrypto.so.1.0.0
00007f1b2baca000 12 8 8 rw--- [ anon ]
00007f1b2baca000 0 0 0 rw--- [ anon ]
00007f1b2bacd000 376 192 0 r-x-- libssl.so.1.0.0
00007f1b2bacd000 0 0 0 r-x-- libssl.so.1.0.0
00007f1b2bb2b000 2048 0 0 ----- libssl.so.1.0.0
00007f1b2bb2b000 0 0 0 ----- libssl.so.1.0.0
00007f1b2bd2b000 16 8 8 r---- libssl.so.1.0.0
00007f1b2bd2b000 0 0 0 r---- libssl.so.1.0.0
00007f1b2bd2f000 24 24 24 rw--- libssl.so.1.0.0
00007f1b2bd2f000 0 0 0 rw--- libssl.so.1.0.0
00007f1b2bd35000 448 136 0 r-x-- libpcre.so.3.13.3
00007f1b2bd35000 0 0 0 r-x-- libpcre.so.3.13.3
00007f1b2bda5000 2048 0 0 ----- libpcre.so.3.13.3
00007f1b2bda5000 0 0 0 ----- libpcre.so.3.13.3
00007f1b2bfa5000 4 4 4 r---- libpcre.so.3.13.3
00007f1b2bfa5000 0 0 0 r---- libpcre.so.3.13.3
00007f1b2bfa6000 4 0 0 rw--- libpcre.so.3.13.3
00007f1b2bfa6000 0 0 0 rw--- libpcre.so.3.13.3
00007f1b2bfa7000 36 36 0 r-x-- libcrypt-2.27.so
00007f1b2bfa7000 0 0 0 r-x-- libcrypt-2.27.so
00007f1b2bfb0000 2044 0 0 ----- libcrypt-2.27.so
00007f1b2bfb0000 0 0 0 ----- libcrypt-2.27.so
00007f1b2c1af000 4 4 4 r---- libcrypt-2.27.so
00007f1b2c1af000 0 0 0 r---- libcrypt-2.27.so
00007f1b2c1b0000 4 0 0 rw--- libcrypt-2.27.so
00007f1b2c1b0000 0 0 0 rw--- libcrypt-2.27.so
00007f1b2c1b1000 184 0 0 rw--- [ anon ]
00007f1b2c1b1000 0 0 0 rw--- [ anon ]
00007f1b2c1df000 104 104 0 r-x-- libpthread-2.27.so
00007f1b2c1df000 0 0 0 r-x-- libpthread-2.27.so
00007f1b2c1f9000 2044 0 0 ----- libpthread-2.27.so
00007f1b2c1f9000 0 0 0 ----- libpthread-2.27.so
00007f1b2c3f8000 4 4 4 r---- libpthread-2.27.so
00007f1b2c3f8000 0 0 0 r---- libpthread-2.27.so
00007f1b2c3f9000 4 0 0 rw--- libpthread-2.27.so
00007f1b2c3f9000 0 0 0 rw--- libpthread-2.27.so
00007f1b2c3fa000 16 4 4 rw--- [ anon ]
00007f1b2c3fa000 0 0 0 rw--- [ anon ]
00007f1b2c3fe000 12 8 0 r-x-- libdl-2.27.so
00007f1b2c3fe000 0 0 0 r-x-- libdl-2.27.so
00007f1b2c401000 2044 0 0 ----- libdl-2.27.so
00007f1b2c401000 0 0 0 ----- libdl-2.27.so
00007f1b2c600000 4 4 4 r---- libdl-2.27.so
00007f1b2c600000 0 0 0 r---- libdl-2.27.so
00007f1b2c601000 4 0 0 rw--- libdl-2.27.so
00007f1b2c601000 0 0 0 rw--- libdl-2.27.so
00007f1b2c602000 156 156 0 r-x-- ld-2.27.so
00007f1b2c602000 0 0 0 r-x-- ld-2.27.so
00007f1b2c7f8000 128 60 60 rw-s- zero (deleted)
00007f1b2c7f8000 0 0 0 rw-s- zero (deleted)
00007f1b2c818000 28 20 20 rw--- [ anon ]
00007f1b2c818000 0 0 0 rw--- [ anon ]
00007f1b2c827000 4 0 0 rw-s- [aio] (deleted)
00007f1b2c827000 0 0 0 rw-s- [aio] (deleted)
00007f1b2c828000 4 4 4 rw-s- zero (deleted)
00007f1b2c828000 0 0 0 rw-s- zero (deleted)
00007f1b2c829000 4 4 4 r---- ld-2.27.so
00007f1b2c829000 0 0 0 r---- ld-2.27.so
00007f1b2c82a000 4 4 4 rw--- ld-2.27.so
00007f1b2c82a000 0 0 0 rw--- ld-2.27.so
00007f1b2c82b000 4 4 4 rw--- [ anon ]
00007f1b2c82b000 0 0 0 rw--- [ anon ]
00007fff6e5c1000 132 16 16 rw--- [ stack ]
00007fff6e5c1000 0 0 0 rw--- [ stack ]
00007fff6e5f1000 12 0 0 r---- [ anon ]
00007fff6e5f1000 0 0 0 r---- [ anon ]
00007fff6e5f4000 8 4 0 r-x-- [ anon ]
00007fff6e5f4000 0 0 0 r-x-- [ anon ]
ffffffffff600000 4 0 0 r-x-- [ anon ]
ffffffffff600000 0 0 0 r-x-- [ anon ]
---------------- ------- ------- -------
total kB 3319780 3159440 3155760
答案1
事实证明，问题出在上游服务器进程上，该进程正在监听 unix 套接字并且不关闭连接（以及泄漏文件描述符）。我猜 nginx 保留了与其相关的连接和资源，这就是内存泄漏的根源。我通过修复上游服务器解决了这个问题，因为我是造成这个错误的人 :)