我们在 mongodb gridfs 平台上遇到了一些奇怪的事情。该平台实际上是一个双 Xeon E5（双四核），具有 128GB 内存，在 freebsd 9 上运行，带有一个专用于 mongodb 的 zfs 池。
[root@mongofile1 ~]# uname -sr
FreeBSD 9.1-RELEASE
我们的/boot/loader.conf
vfs.zfs.arc_min="2048M"
vfs.zfs.arc_max="7680M"
vm.kmem_size_max="16G"
vm.kmem_size="12G"
vfs.zfs.prefetch_disable="1"
kern.ipc.nmbclusters="32768"
/etc/sysctl.conf
net.inet.tcp.msl=15000
net.inet.tcp.keepidle=300000
kern.ipc.nmbclusters=32768
kern.ipc.maxsockbuf=2097152
kern.ipc.somaxconn=8192
kern.maxfiles=65536
kern.maxfilesperproc=32768
net.inet.tcp.delayed_ack=0
net.inet.tcp.sendspace=65535
net.inet.udp.recvspace=65535
net.inet.udp.maxdgram=57344
net.local.stream.recvspace=65535
net.local.stream.sendspace=65535
我们遵循 ulimit 的建议：
[root@mongofile1 ~]# su - mongodb
$ ulimit -a
cpu time (seconds, -t) unlimited
file size (512-blocks, -f) unlimited
data seg size (kbytes, -d) 33554432
stack size (kbytes, -s) 524288
core file size (512-blocks, -c) unlimited
max memory size (kbytes, -m) unlimited
locked memory (kbytes, -l) unlimited
max user processes (-u) 5547
open files (-n) 32768
virtual mem size (kbytes, -v) unlimited
swap limit (kbytes, -w) unlimited
sbsize (bytes, -b) unlimited
pseudo-terminals (-p) unlimited
该服务器与其他数据中心的 ReplSet 有一个孪生兄弟（完全相同的配置），并且我们有一个虚拟化仲裁器。
一段时间，差不多3天了，mongodb进程退出了。问题开始于：
Fri Nov 8 11:27:31.741 [conn774697] end connection 192.168.10.162:47963 (23 connections now open)
Fri Nov 8 11:27:31.770 [initandlisten] can't create new thread, closing connection
Fri Nov 8 11:27:31.771 [rsHealthPoll] replSet member mongofile2:27017 is now in state DOWN
Fri Nov 8 11:27:31.774 [initandlisten] connection accepted from 192.168.10.162:47968 #774702 (20 connections now open)
Fri Nov 8 11:27:31.774 [initandlisten] connection accepted from 192.168.10.161:28522 #774703 (21 connections now open)
Fri Nov 8 11:27:31.774 [initandlisten] connection accepted from 192.168.10.164:15406 #774704 (22 connections now open)
Fri Nov 8 11:27:31.774 [initandlisten] connection accepted from 192.168.10.163:25750 #774705 (23 connections now open)
Fri Nov 8 11:27:31.810 [initandlisten] connection accepted from 192.168.10.182:20779 #774706 (24 connections now open)
Fri Nov 8 11:27:31.855 [initandlisten] connection accepted from 192.168.10.161:28524 #774707 (25 connections now open)
Fri Nov 8 11:27:31.869 [initandlisten] connection accepted from 192.168.10.182:20786 #774708 (26 connections now open)
经过多次“可以创建新线程”
[root@mongofile1 /usr/mongodb]# tail -n 15000 mongod.log.old |grep "create new thread"|wc
5020 55220 421680
并以一场盛大的
Fri Nov 8 11:30:22.333 [rsMgr] replSet warning caught unexpected exception in electSelf()
pure virtual method called
Fri Nov 8 11:30:22.333 Got signal: 6 (Abort trap: 6).
Fri Nov 8 11:30:22.337 Backtrace:
0x599efc 0x8035cb516
0x599efc <_ZN5mongo10abruptQuitEi+988> at /usr/local/bin/mongod
0x8035cb516 <_pthread_sigmask+918> at /lib/libthr.so.3
从顶部提取 mongodb
78126 mongodb 77 20 0 1253G 1449M sbwait 0 0:20 0.00% mongod
如果我在崩溃时重新启动该进程，问题将近 3 天后就解决了。
有人曾经见过这个问题或者知道解决办法吗？
答案1
Mongo DB 项目建议设置
kern.threads.max_threads_per_proc=32000
kern.maxfilesperproc=64000
在 中/etc/sysctl.conf。您可以运行/etc/rc.d/sysctl restart以使新设置立即生效，或者重新启动（无论您喜欢哪种方式）