Apache 2.4 - 进程间歇性挂起

Question

我遇到了 httpd 进程间歇性挂起的问题。在进行故障排除时，我发现一些信号量计数大于零。我可以通过清除这些 semID 来使其正常工作。但我想了解是什么原因造成的。

以下是一些信息。

版本：服务器版本：Apache/2.4.6 (CentOS)

操作系统——Centos 7

ipcs -s -i 45187195
Semaphore Array semid=45187195
uid=48 gid=48 cuid=0 cgid=0
mode=0600, access_perms=0600
nsems = 1
otime = Tue Feb 23 12:14:35 2021
ctime = Mon Feb 8 09:45:38 2021
semnum value ncount zcount pid
0 0 1 0 19944
 
ps -ef | grep 19944
root 2409 19130 0 17:06 pts/0 00:00:00 grep --color=auto 19944
apache 19944 24691 0 12:14 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
 
ipcs -us
------ Semaphore Status --------
used arrays = 128
allocated semaphores = 128
 
ipcs -ls
------ Semaphore Limits --------
max number of arrays = 256
max semaphores per array = 250
max semaphores system wide = 64000
max ops per semop call = 32
semaphore max value = 32767
 
ps -efL|grep httpd | awk '{print $2}' | uniq -c
60 2471
1 2509
80 12617
202 19944
1 20149
76 21108
1 24691
1 24692
75 28558
76 32344

wget localhost
--2021-02-23 17:10:24-- http://localhost/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80...

tcp6 512 0 :::80 :::* LISTEN 19944/httpd
 
nc -zv localhost 80
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:80.
Ncat: 0 bytes sent, 0 bytes received in 1.01 seconds

**Modules loaded:**
Loaded Modules:
 core_module (static)
 so_module (static)
 http_module (static)
 access_compat_module (shared)
 actions_module (shared)
 alias_module (shared)
 allowmethods_module (shared)
 auth_basic_module (shared)
 auth_digest_module (shared)
 authn_anon_module (shared)
 authn_core_module (shared)
 authn_dbd_module (shared)
 authn_dbm_module (shared)
 authn_file_module (shared)
 authn_socache_module (shared)
 authz_core_module (shared)
 authz_dbd_module (shared)
 authz_dbm_module (shared)
 authz_groupfile_module (shared)
 authz_host_module (shared)
 authz_owner_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 cache_module (shared)
 cache_disk_module (shared)
 data_module (shared)
 dbd_module (shared)
 deflate_module (shared)
 dir_module (shared)
 dumpio_module (shared)
 echo_module (shared)
 env_module (shared)
 expires_module (shared)
 ext_filter_module (shared)
 filter_module (shared)
 headers_module (shared)
 include_module (shared)
 info_module (shared)
 log_config_module (shared)
 logio_module (shared)
 mime_magic_module (shared)
 mime_module (shared)
 negotiation_module (shared)
 remoteip_module (shared)
 reqtimeout_module (shared)
 rewrite_module (shared)
 setenvif_module (shared)
 slotmem_plain_module (shared)
 slotmem_shm_module (shared)
 socache_dbm_module (shared)
 socache_memcache_module (shared)
 socache_shmcb_module (shared)
 status_module (shared)
 substitute_module (shared)
 suexec_module (shared)
 unique_id_module (shared)
 unixd_module (shared)
 userdir_module (shared)
 version_module (shared)
 vhost_alias_module (shared)
 dav_module (shared)
 dav_fs_module (shared)
 dav_lock_module (shared)
 lua_module (shared)
 mpm_worker_module (shared)
 proxy_module (shared)
 lbmethod_bybusyness_module (shared)
 lbmethod_byrequests_module (shared)
 lbmethod_bytraffic_module (shared)
 lbmethod_heartbeat_module (shared)
 proxy_ajp_module (shared)
 proxy_balancer_module (shared)
 proxy_connect_module (shared)
 proxy_express_module (shared)
 proxy_fcgi_module (shared)
 proxy_fdpass_module (shared)
 proxy_ftp_module (shared)
 proxy_http_module (shared)
 proxy_scgi_module (shared)
 proxy_wstunnel_module (shared)
 systemd_module (shared)
 cgid_module (shared)
 wsgi_module (shared)

1.在此期间，连接数激增，直到进程挂起。2.日志中没有任何错误或任何内容。它只是挂起并停止接受流量。日志中没有 maxclient 错误。3.我确实尝试过 gdb 和 strace，但没有任何帮助。

感谢您指出导致这种情况的原因。如果您还有其他问题，请告诉我。

Answer 1

不确定这是否真的有帮助，但当 Apache 用完信号量时，我也遇到了类似的问题。为了解决这个问题，我必须这样做：

for i in `ipcs -s | awk '/nobody/ {print $2}'`; do { ipcrm -s $i ; } done;
echo "kernel.msgmni = 1024" >> /etc/sysctl.conf
echo "kernel.sem = 250 256000 32 1024" >> /etc/sysctl.conf
sysctl -p
systemctl restart httpd

在你的情况下数组的最大数量是256。通过运行上述命令将其增加到 1024！

您实际上可以通过运行以下命令来查看当前信号量配置：

ipcs -l

注意：如果你的 Apache Web 服务器使用其他用户运行，则只需替换没有人和阿帕奇或者www-数据或您正在使用的任何用户。希望它能有所帮助！

Answer 2

我确实尝试过更新配置。即使问题仍然存在。

ipcs -l

------ Messages Limits --------
max queues system wide = 1024
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 67108864
max total shared memory (kbytes) = 17179869184
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 1024
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767

我在日志中没有看到任何内容。

**ipcs -s -i 111968451**

Semaphore Array semid=111968451
uid=48   gid=48  cuid=0  cgid=0
mode=0600, access_perms=0600
nsems = 1
otime = Fri Mar  5 16:13:49 2021
ctime = Fri Mar  5 07:53:50 2021
semnum     value      ncount     zcount     pid
0          0          1          0          3305

**ps -ef | grep 3305**

root      1598 28631  0 18:18 pts/0    00:00:00 grep --color=auto 3305
apache    3305  1362  0 16:13 ?        00:00:01 /usr/sbin/httpd -DFOREGROUND

]#**netstat -tnlp | grep 80**
tcp        0      0 0.0.0.0:8089            0.0.0.0:*               LISTEN      3391/splunkd
tcp6     512      0 :::80                   :::*                    LISTEN      1362/httpd

如果有任何其他方法可以解决此问题，我将不胜感激。谢谢