我的 Apache2.4(mpm_worker)实例退出并显示以下日志:

[Mon May 27 11:27:33.196177 2019] [core:warn] [pid 567365:tid 139765793668032] AH00045: child process 567368 still did not exit, sending a SIGTERM
[Mon May 27 11:27:35.198179 2019] [core:warn] [pid 567365:tid 139765793668032] AH00045: child process 567368 still did not exit, sending a SIGTERM
[Mon May 27 11:27:37.200177 2019] [core:error] [pid 567365:tid 139765793668032] AH00046: child process 567368 still did not exit, sending a SIGKILL
[Mon May 27 11:27:38.297736 2019] [core:warn] [pid 569972:tid 140126180117440] AH00098: pid file /var/run/apache2/apache2.pid overwritten -- Unclean shutdown of previous Apache run?
[Mon May 27 11:27:38.300264 2019] [mpm_worker:notice] [pid 569972:tid 140126180117440] AH00292: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.0g configured -- resuming normal operations
[Mon May 27 11:27:38.300296 2019] [core:notice] [pid 569972:tid 140126180117440] AH00094: Command line: '/usr/sbin/apache2'
[Mon May 27 11:27:38.303713 2019] [mpm_worker:alert] [pid 569975:tid 140126052628224] (11)Resource temporarily unavailable: AH03142: apr_thread_create: unable to create worker thread
[Mon May 27 11:27:38.359830 2019] [mpm_worker:alert] [pid 569973:tid 140126180117440] (11)Resource temporarily unavailable: AH00282: apr_thread_create: unable to create worker thread
[Mon May 27 11:27:38.371350 2019] [mpm_worker:alert] [pid 569974:tid 140126060156672] (11)Resource temporarily unavailable: AH03142: apr_thread_create: unable to create worker thread
[Mon May 27 11:27:40.302219 2019] [mpm_worker:alert] [pid 569972:tid 140126180117440] AH02325: A resource shortage or other unrecoverable failure was encountered before any child process initialized successfully... httpd is exiting!

读到日志的最后一行,我以为 Apache 现在应该已经死了。但事实却top显示 Apache 的 CPU 使用率为 320%。这种情况会一直持续下去(至少很多天)。读到日志的前几行,很明显,在 Apache 假装退出后,它无法出于任何原因关闭或终止某个子进程。

谷歌搜索“ unable to create worker thread”引导我找到服务器资源限制,并调整正确的 apache2 设置。因此,这是 www-data 的 Ulimit:

Apache 的 Ulimit

root@xxx:~# sudo -u www-data bash -c 'ulimit -a'
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 1546431
max locked memory       (kbytes, -l) 16384
max memory size         (kbytes, -m) unlimited
open files                      (-n) 105000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 10000
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

mpm_worker.conf

<IfModule mpm_worker_module>
    StartServers             2
    MinSpareThreads      5
    MaxSpareThreads      10
    ThreadLimit          64
    ThreadsPerChild      5
    MaxRequestWorkers     100
    MaxConnectionsPerChild   50
</IfModule>

服务器信息:

VPS: 6core 2Ghz
RAM: 8GB
Usage: Only one domain/apache instance for Nextcloud.

#1

在 Apache 显示退出但导致 CPU 使用率过高后,我该如何调试它发生了什么?(修复会更好)

#2

我如何根据用例调整工人设置?(我猜这就是根本原因)

任何帮助都非常感谢!谢谢

答案1

就我而言,Ubuntu 18.04(和 20.04)和 SystemD 出现了同样的错误:

Resource temporarily unavailable: AH03142: apr_thread_create: unable to create worker thread

没有内存问题,上面描述的问题都无法解决,只有这个对我有用:

systemctl set-property apache2.service TasksMax=infinity

答案2

检查您的 mpm_worker_module 配置以确定您是否有足够的内存来创建配置的线程数量。

( ThreadLimit X ServerLimit ) X ThreadStackSize = Memory_Max (in bytes) <= System_Memory

注意:ThreadStackSize 的默认值在其他操作系统上有所不同。您可以在 RHEL 中使用 ulimit -s 进行检查。

审查您的/etc/security/limits.conf和/或/etc/security/limits.d/90-nproc.conf查看是否需要增加进程限制。

下面将显示 httpd worker 正在运行多少个进程。

ps -elf | grep httpd | grep -v grep | wc -l 

如果它超出了 etc/security/limits.conf 中的 nproc 设置,那么您将无法创建足够的进程(资源不足)来继续。请参阅 ulimit 说明,了解如何增加设置。

造成此类问题的最常见原因是:

Not enough memory to create a new thread.
Not enough resources to create a new thread (files, process limits)

查看 Apache 最大内存(用于线程)的算法 ThreadStackSize 以字节为单位。

( ThreadLimit X ServerLimit ) X ThreadStackSize = Memory_Max in bytes

降低 ServerLimit 和/或 ThreadsPerChild 将减少所需的线程总数,因此应该避免错误(基于可用资源)

相关内容