httpd 占用所有内存直到服务器崩溃

httpd 占用所有内存直到服务器崩溃

有问题的服务器用于将数据处理成报告,它有三个 cron 作业,每分钟执行一次以检查进程队列,如果发现任何内容,它们将继续运行,直到队列为空。在特别繁忙的日子(过去 3 个月的每周二和其他奇数天)服务器会崩溃。

通过运行,ps aux | grep 'httpd' | awk '{print $6/1024 " MB";}'我可以看到三个 httpd 工作人员持续以线性方式获取内存。由于有大量报告,进程不会结束,并且会继续吸收更多内存。

这是一个 PHP 运行时,检查内部的内存使用情况,我们可以看到它根本不是一个问题。 PHP 内存有 256MB 的硬限制,并且在 PHP 层中,至少在报告处理完成后会释放内存。当前在重复服务器上使用 PHP 内存分析器来查看 PHP 层中是否存在导致此问题的任何问题。然而,上面提到的 ps aux 命令显示,httpd 工作线程的内存从大约 270MB 开始,并且不断上升,直到整个服务器(8GB RAM)耗尽。因此,在我看来,httpd 可能会占用 PHP 层正在使用的所有内存,而不是将其回收到 PHP 进程或系统中。下面出现这种情况时的错误日志片段。

[Tue Dec 04 09:20:34.805175 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:20:45.142735 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process

mmap() failed: [
mmap() failed: [
mmap() failed: [1122]]  CCannot allocate memoryannot allocate memory12] Cannot allocate memory


mmap() failed: [
mmap() failed: [1122]]  CCannot allocate memoryannot allocate memory

mmap() failed: [12] Cannot allocate memory[Tue Dec 04 09:20:45.764357 2018] [php7:error] [pid 12198] [client 127.0.0.1:37694] PHP Fatal error:
  Out of memory (allocated 2097152) (tried to allocate 65536 bytes) in /var/www/html/vendor/composer/ClassLoader.php on line 440
[Tue Dec 04 09:20:45.764411 2018] [php7:error] [pid 12197] [client 127.0.0.1:37692] PHP Fatal error:  Out of memory (allocated 2097152) (tried
 to allocate 65536 bytes) in /var/www/html/vendor/composer/ClassLoader.php on line 440
[Tue Dec 04 09:20:45.764358 2018] [php7:error] [pid 12191] [client 127.0.0.1:37690] PHP Fatal error:  Out of memory (allocated 2097152) (tried
 to allocate 65536 bytes) in /var/www/html/vendor/composer/ClassLoader.php on line 440



mmap() failed: [
mmap() failed: [1122]]  CCannot allocate memoryannot allocate memory

mmap() failed: [12] Cannot allocate memory


mmap() failed: [
mmap() failed: [1122]]  CCannot allocate memoryannot allocate memory

mmap() failed: [12] Cannot allocate memory[Tue Dec 04 09:20:46.257438 2018] [php7:error] [pid 12191] [client 127.0.0.1:37690] PHP Fatal error:
  Out of memory (allocated 2097152) (tried to allocate 73728 bytes) in /var/www/html/vendor/composer/ClassLoader.php on line 440
[Tue Dec 04 09:20:46.257439 2018] [php7:error] [pid 12197] [client 127.0.0.1:37692] PHP Fatal error:  Out of memory (allocated 2097152) (tried
 to allocate 73728 bytes) in /var/www/html/vendor/composer/ClassLoader.php on line 440
[Tue Dec 04 09:20:46.257481 2018] [php7:error] [pid 12198] [client 127.0.0.1:37694] PHP Fatal error:  Out of memory (allocated 2097152) (tried
 to allocate 73728 bytes) in /var/www/html/vendor/composer/ClassLoader.php on line 440
[Tue Dec 04 09:20:57.249945 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:23:08.171314 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:24:33.415351 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:26:22.308600 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:27:55.057324 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:29:17.174173 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:30:39.193341 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:33:16.023329 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:34:53.208958 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:35:30.902310 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:36:05.215192 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:37:36.511811 2018] [mpm_prefork:error] [pid 2751] (12)Cannot allocate memory: AH00159: fork: Unable to fork new process
[Tue Dec 04 09:42:18.453045 2018] [suexec:notice] [pid 2699] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)

使用中的httpd mods:

$ sudo httpd -M
Loaded Modules:
 core_module (static)
 so_module (static)
 http_module (static)
 access_compat_module (shared)
 actions_module (shared)
 alias_module (shared)
 allowmethods_module (shared)
 auth_basic_module (shared)
 auth_digest_module (shared)
 authn_anon_module (shared)
 authn_core_module (shared)
 authn_dbd_module (shared)
 authn_dbm_module (shared)
 authn_file_module (shared)
 authn_socache_module (shared)
 authz_core_module (shared)
 authz_dbd_module (shared)
 authz_dbm_module (shared)
 authz_groupfile_module (shared)
 authz_host_module (shared)
 authz_owner_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 cache_module (shared)
 cache_disk_module (shared)
 cache_socache_module (shared)
 data_module (shared)
 dbd_module (shared)
 deflate_module (shared)
 dir_module (shared)
 dumpio_module (shared)
 echo_module (shared)
 env_module (shared)
 expires_module (shared)
 ext_filter_module (shared)
 filter_module (shared)
 headers_module (shared)
 http2_module (shared)
 include_module (shared)
 info_module (shared)
 log_config_module (shared)
 logio_module (shared)
 macro_module (shared)
 mime_magic_module (shared)
 mime_module (shared)
 negotiation_module (shared)
 remoteip_module (shared)
 reqtimeout_module (shared)
 request_module (shared)
 rewrite_module (shared)
 setenvif_module (shared)
 slotmem_plain_module (shared)
 slotmem_shm_module (shared)
 socache_dbm_module (shared)
 socache_memcache_module (shared)
 socache_shmcb_module (shared)
 status_module (shared)
 substitute_module (shared)
 suexec_module (shared)
 unixd_module (shared)
 userdir_module (shared)
 version_module (shared)
 vhost_alias_module (shared)
 watchdog_module (shared)
 dav_module (shared)
 dav_fs_module (shared)
 dav_lock_module (shared)
 lua_module (shared)
 mpm_prefork_module (shared)
 proxy_module (shared)
 lbmethod_bybusyness_module (shared)
 lbmethod_byrequests_module (shared)
 lbmethod_bytraffic_module (shared)
 lbmethod_heartbeat_module (shared)
 proxy_ajp_module (shared)
 proxy_balancer_module (shared)
 proxy_connect_module (shared)
 proxy_express_module (shared)
 proxy_fcgi_module (shared)
 proxy_fdpass_module (shared)
 proxy_ftp_module (shared)
 proxy_http_module (shared)
 proxy_hcheck_module (shared)
 proxy_scgi_module (shared)
 proxy_uwsgi_module (shared)
 proxy_wstunnel_module (shared)
 ssl_module (shared)
 php7_module (shared)
$ httpd -l
Compiled in modules:
  core.c
  mod_so.c
  http_core.c

httpd/http.d/http.conf

<IfModule mpm_prefork_module>
    StartServers             2
    MinSpareServers          2
    MaxSpareServers          4
    MaxRequestWorkers        10
    MaxConnectionsPerChild 1
</IfModule>

答案1

...三个 cron 作业每分钟执行一次以检查进程队列,如果发现任何内容,它们将继续运行,直到队列为空

这很容易导致内存泄漏,因此请使用 bash 恶魔为队列中的每个项目依次生成一个新实例;

while true ; do
    while read -r F ; do
        php /var/www/html/my.site.com/process1.php "$F"
    done < <(find /path/to/queue -type f)
    sleep 1
done

呼叫

wget -q -O /dev/null http://localhost/core/processing

不必要地涉及 apache 并允许其保留内存的时间超过一项的持续时间。

相关内容