我有一个运行 Ubuntu 16.04 的 EC2 实例，它不断耗尽内存，从而终止我的 mongod 进程。
做df -h给予
udev 2.0G 0 2.0G 0% /dev
tmpfs 396M 41M 355M 11% /run
/dev/xvda1 7.8G 7.4G 0 100% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
tmpfs 396M 0 396M 0% /run/user/1000
因为我的目录/dev/xvda1满了。我进入我的“/”或根目录并执行
ls -l --block-size=M
total 1M
drwxr-xr-x 2 root root 1M Nov 22 04:49 bin
drwxr-xr-x 3 root root 1M Dec 21 13:14 boot
drwxrw-rwx 3 root root 1M Oct 18 21:01 data
drwxr-xr-x 16 root root 1M Oct 18 20:30 dev
drwxr-xr-x 91 root root 1M Dec 14 01:29 etc
drwxr-xr-x 3 root root 1M Oct 18 20:30 home
lrwxrwxrwx 1 root root 1M Dec 21 13:14 initrd.img -> boot/initrd.img-4.4.0-57-generic
lrwxrwxrwx 1 root root 1M Dec 6 05:03 initrd.img.old -> boot/initrd.img-4.4.0-53-generic
drwxr-xr-x 21 root root 1M Sep 7 19:24 lib
drwxr-xr-x 2 root root 1M Sep 7 19:22 lib64
drwx------ 2 root root 1M Sep 7 19:26 lost+found
drwxr-xr-x 2 root root 1M Sep 7 19:22 media
drwxr-xr-x 2 root root 1M Sep 7 19:22 mnt
drwxr-xr-x 2 root root 1M Sep 7 19:22 opt
dr-xr-xr-x 139 root root 0M Oct 18 20:29 proc
drwx------ 4 root root 1M Oct 18 21:00 root
drwxr-xr-x 23 root root 1M Dec 25 13:55 run
drwxr-xr-x 2 root root 1M Oct 19 06:11 sbin
drwxr-xr-x 2 root root 1M Sep 1 17:37 snap
drwxr-xr-x 2 root root 1M Sep 7 19:22 srv
dr-xr-xr-x 13 root root 0M Dec 25 13:59 sys
drwxrwxrwt 11 root root 1M Dec 25 14:17 tmp
drwxr-xr-x 10 root root 1M Sep 7 19:22 usr
drwxr-xr-x 14 root root 1M Oct 18 20:52 var
lrwxrwxrwx 1 root root 1M Dec 21 13:14 vmlinuz -> boot/vmlinuz-4.4.0-57-generic
lrwxrwxrwx 1 root root 1M Dec 6 05:03 vmlinuz.old -> boot/vmlinuz-4.4.0-53-generic
如果我把所有文件大小加起来，结果加起来不是 7.4 GB。那么这是什么？我该如何解决这个问题？这样它就不会溢出并杀死我的 mongod 进程。
在一些答案中。它说重新启动系统。重新启动后，这是输出。
udev 2.0G 0 2.0G 0% /dev
tmpfs 396M 5.6M 390M 2% /run
/dev/xvda1 7.8G 5.3G 2.2G 72% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
tmpfs 396M 0 396M 0% /run/user/1000
我仍使用 5.3G 空间。是什么原因导致 2 GB 被填满？我该如何深入了解这个问题？
最新编辑 启动 2 个 NodeJS 进程和一个 mongod 进程后重启。不到 14 小时，它又一次关闭了内存。
不是重复的
他们正在运行 Ubuntu 的本地实例，而我没有。有助于深入研究原因的答案不会在这里运行。它因错误而停止(gksudo:3647): Gtk-WARNING **: cannot open display:。这里的大多数答案都需要某种基于 GUI 的工具。这是一个限制。因此这不是重复的，我要求保持开放。
编辑1
ubuntu@ip-172-31-29-235:/$ sudo du -hcsx .[!.]* * | sort -rh | head
du: cannot access '.[!.]*': No such file or directory
du: cannot access 'proc/4433/task/4433/fd/4': No such file or directory
du: cannot access 'proc/4433/task/4433/fdinfo/4': No such file or directory
du: cannot access 'proc/4433/fd/4': No such file or directory
du: cannot access 'proc/4433/fdinfo/4': No such file or directory
3.9G total
1.8G usr
983M home
363M lib
323M data
249M var
163M boot
16M bin
13M sbin
5.8M etc
如果完全利用了3.9 GB。
ubuntu@ip-172-31-29-235:/$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 2.0G 0 2.0G 0% /dev
tmpfs 396M 5.6M 390M 2% /run
/dev/xvda1 7.8G 6.5G 936M 88% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
tmpfs 396M 0 396M 0% /run/user/1000
已使用容量为 6.5 GB，那么 2.6 GB 去哪儿了？
编辑2
添加 sudo lsof | grep removed 的输出
ubuntu@ip-172-31-29-235:/$ sudo lsof | grep deleted
mongod 1272 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1273 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1275 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1276 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1277 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1278 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1279 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1280 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1281 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1282 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1283 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1284 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1285 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1286 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1287 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1288 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1289 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1291 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1292 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1293 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1294 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1297 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1298 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1299 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1300 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1301 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1302 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1303 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1304 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1305 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1306 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1307 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1308 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1309 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1310 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1311 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1312 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1313 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1314 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1315 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1316 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1317 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1318 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1319 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1320 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1321 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1322 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1323 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1324 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1325 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1326 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1327 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1328 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1329 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1330 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1331 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1332 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1333 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1334 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1335 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1336 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1337 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1338 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1342 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 1753 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 2499 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
mongod 1272 2500 ubuntu 4w REG 202,1 2727852655 160748 /var/log/mongodb/mongod.log (deleted)
答案1
根据输出，lsof有一个很大的（2.7 GB）未链接（“ deleted”）文件曾经通过路径打开/var/log/mongodb/mongod.log。
触发文件删除的唯一方法是删除对该文件的所有引用。其中包括
取消链接所有目录条目（您已这样做）和
关闭所有引用它的文件描述符。
只有持有文件描述符的进程才能关闭它；或者，当进程终止或被终止时，内核会关闭所有剩余的文件描述符。由于相关文件是的日志文件，并且其描述符由服务持有，因此mongod实现此目的的最佳方法是终止服务（例如，使用sudo service mongod stop或，如果守护进程没有响应，则使用sudo killall -s KILL mongod）。
如果日志文件不断被填满，这将无法阻止问题再次发生。您需要查看日志内容并调查mongod试图通知的问题。请提出新问题如果你需要帮助的话。