我的 Apache2.4(mpm_worker)实例退出并显示以下日志:
[Mon May 27 11:27:33.196177 2019] [core:warn] [pid 567365:tid 139765793668032] AH00045: child process 567368 still did not exit, sending a SIGTERM
[Mon May 27 11:27:35.198179 2019] [core:warn] [pid 567365:tid 139765793668032] AH00045: child process 567368 still did not exit, sending a SIGTERM
[Mon May 27 11:27:37.200177 2019] [core:error] [pid 567365:tid 139765793668032] AH00046: child process 567368 still did not exit, sending a SIGKILL
[Mon May 27 11:27:38.297736 2019] [core:warn] [pid 569972:tid 140126180117440] AH00098: pid file /var/run/apache2/apache2.pid overwritten -- Unclean shutdown of previous Apache run?
[Mon May 27 11:27:38.300264 2019] [mpm_worker:notice] [pid 569972:tid 140126180117440] AH00292: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.0g configured -- resuming normal operations
[Mon May 27 11:27:38.300296 2019] [core:notice] [pid 569972:tid 140126180117440] AH00094: Command line: '/usr/sbin/apache2'
[Mon May 27 11:27:38.303713 2019] [mpm_worker:alert] [pid 569975:tid 140126052628224] (11)Resource temporarily unavailable: AH03142: apr_thread_create: unable to create worker thread
[Mon May 27 11:27:38.359830 2019] [mpm_worker:alert] [pid 569973:tid 140126180117440] (11)Resource temporarily unavailable: AH00282: apr_thread_create: unable to create worker thread
[Mon May 27 11:27:38.371350 2019] [mpm_worker:alert] [pid 569974:tid 140126060156672] (11)Resource temporarily unavailable: AH03142: apr_thread_create: unable to create worker thread
[Mon May 27 11:27:40.302219 2019] [mpm_worker:alert] [pid 569972:tid 140126180117440] AH02325: A resource shortage or other unrecoverable failure was encountered before any child process initialized successfully... httpd is exiting!
读到日志的最后一行,我以为 Apache 现在应该已经死了。但事实却top
显示 Apache 的 CPU 使用率为 320%。这种情况会一直持续下去(至少很多天)。读到日志的前几行,很明显,在 Apache 假装退出后,它无法出于任何原因关闭或终止某个子进程。
谷歌搜索“ unable to create worker thread
”引导我找到服务器资源限制,并调整正确的 apache2 设置。因此,这是 www-data 的 Ulimit:
Apache 的 Ulimit
root@xxx:~# sudo -u www-data bash -c 'ulimit -a'
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 1546431
max locked memory (kbytes, -l) 16384
max memory size (kbytes, -m) unlimited
open files (-n) 105000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 10000
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
mpm_worker.conf
<IfModule mpm_worker_module>
StartServers 2
MinSpareThreads 5
MaxSpareThreads 10
ThreadLimit 64
ThreadsPerChild 5
MaxRequestWorkers 100
MaxConnectionsPerChild 50
</IfModule>
服务器信息:
VPS: 6core 2Ghz
RAM: 8GB
Usage: Only one domain/apache instance for Nextcloud.
#1
在 Apache 显示退出但导致 CPU 使用率过高后,我该如何调试它发生了什么?(修复会更好)
#2
我如何根据用例调整工人设置?(我猜这就是根本原因)
任何帮助都非常感谢!谢谢
答案1
就我而言,Ubuntu 18.04(和 20.04)和 SystemD 出现了同样的错误:
Resource temporarily unavailable: AH03142: apr_thread_create: unable to create worker thread
没有内存问题,上面描述的问题都无法解决,只有这个对我有用:
systemctl set-property apache2.service TasksMax=infinity
答案2
检查您的 mpm_worker_module 配置以确定您是否有足够的内存来创建配置的线程数量。
( ThreadLimit X ServerLimit ) X ThreadStackSize = Memory_Max (in bytes) <= System_Memory
注意:ThreadStackSize 的默认值在其他操作系统上有所不同。您可以在 RHEL 中使用 ulimit -s 进行检查。
审查您的/etc/security/limits.conf和/或/etc/security/limits.d/90-nproc.conf查看是否需要增加进程限制。
下面将显示 httpd worker 正在运行多少个进程。
ps -elf | grep httpd | grep -v grep | wc -l
如果它超出了 etc/security/limits.conf 中的 nproc 设置,那么您将无法创建足够的进程(资源不足)来继续。请参阅 ulimit 说明,了解如何增加设置。
造成此类问题的最常见原因是:
Not enough memory to create a new thread.
Not enough resources to create a new thread (files, process limits)
查看 Apache 最大内存(用于线程)的算法 ThreadStackSize 以字节为单位。
( ThreadLimit X ServerLimit ) X ThreadStackSize = Memory_Max in bytes
降低 ServerLimit 和/或 ThreadsPerChild 将减少所需的线程总数,因此应该避免错误(基于可用资源)