我们遇到一个问题,我们的 OpenNMS 实例在某些时间发送错误警报,日志显示以下内容:
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method) ~[?:1.8.0_45]
at java.lang.Thread.start(Thread.java:714) ~[?:1.8.0_45]
at org.postgresql.Driver.connect(Driver.java:283) ~[postgresql-9.3-1100-jdbc4.jar:?]
at org.opennms.netmgt.collectd.jdbc.JdbcAgentState.getJdbcConnection(JdbcAgentState.java:120) ~[org.opennms.features.jdbc-collector-17.1.1.jar:?]
at org.opennms.netmgt.collectd.JdbcCollector.collect(JdbcCollector.java:220) ~[org.opennms.features.jdbc-collector-17.1.1.jar:?]
at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:274) ~[opennms-services-17.1.1.jar:?]
at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:395) ~[opennms-services-17.1.1.jar:?]
我在网上查了一下,可能是我们达到了 ulimit 设置的最大限制。但是,通过发出 ulimit -a,我们可以看到应该没有问题:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 257405
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 257405
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
系统分配了 64 GB RAM,但仍然需要交换。只需为机器和 JVM 分配更多 RAM 即可解决这个问题吗?
编辑:ps -ef | grep java
root 10055 1 99 Jul04 ? 27-06:47:17 /usr/bin/java -Djava.endorsed.dirs=/usr/share/opennms/lib/endorsed -Dopennms.home=/usr/share/opennms -Xmx50248m -XX:+HeapDumpOnOutOfMemoryError -d64 -Xms50248m -Duser.language=sv -XX:MaxPermSize=6144m -XX:+UseStringDeduplication -XX:+UseG1GC -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.login.config=opennms -Dcom.sun.management.jmxremote.access.file=/usr/share/opennms/etc/jmxremote.access -DisThreadContextMapInheritable=true -XX:MaxMetaspaceSize=256m -Djava.io.tmpdir=/usr/share/opennms/data/tmp -jar /usr/share/opennms/lib/opennms_bootstrap.jar start
在 /etc/opennms/opennms.conf 中我得到以下内容:
JAVA_HEAP_SIZE=50248
ADDITIONAL_MANAGER_OPTIONS="-d64 -Xms"$JAVA_HEAP_SIZE"m -Duser.language=sv -XX:MaxPermSize=6144m -XX:+UseStringDeduplication -XX:+UseG1GC"
#MAXIMUM_FILE_DESCRIPTORS=204800 #2017-03-23
MAXIMUM_FILE_DESCRIPTORS=1024000
MAXIMUM_SIZE_STACK_SEGMENT=16384