反复面临 mysqld 内存不足的问题。大量 PHP 进程被创建，占用了所有交换空间。
[562074.326710] mysqld 调用 oom-killer：gfp_mask=0x201da，order=0，oom_adj=0，oom_score_adj=0 [562074.329646]mysqld cpuset=/mems_allowed=0 [562074.330882] Pid：21351，通信：mysqld 未受污染 2.6.32-642.1.1.el6.x86_64 #1 [562074.333171] 呼叫追踪: [562074.334870][]?dump_header+0x90/0x1b0 [562074.336812][]?安全_real_capable_noaudit+0x3c/0x70 [562074.338692][]?oom_kill_process+0x82/0x2a0 [562074.340270][]?select_bad_process+0xe1/0x120 [562074.341582][]?内存不足+0x220/0x3c0 [562074.343229][]?__alloc_pages_nodemask+0x93c/0x950 [562074.344930][] ?ext4_get_block+0x0/0x120 [ext4] [562074.346436][]?alloc_pages_current+0xaa/0x110 [562074.347604][]?__page_cache_alloc+0x87/0x90 [562074.348889][]?find_get_page+0x1e/0xa0 [562074.350660][]?文件映射故障+0x1a7/0x500 [562074.352207][]?__do_fault+0x54/0x530 [562074.353872][]?handle_pte_fault+0xf7/0xb20 [562074.355966][]?rwsem_down_failed_common+0x95/0x1d0 [562074.357531][]?rwsem_down_read_failed+0x26/0x30 [562074.359503][]?处理mm故障+0x299/0x3d0 [562074.361105][]?call_rwsem_down_read_failed+0x14/0x30 [562074.362522][]?__do_page_fault+0x146/0x500 [562074.364590][]?lock_sock_nested+0xac/0xc0 [562074.387996][]?_spin_unlock_bh+0x1b/0x20 [562074.389187][]?pvclock_clocksource_read+0x58/0xd0 [562074.390457][]?kvm_clock_read+0x1c/0x20 [562074.391586][]?kvm_clock_get_cycles+0x9/0x10 [562074.392673][]?获取nstimeofday+0x58/0xf0 [562074.394697][]?do_page_fault+0x3e/0xa0 [562074.395974] [] ?页面错误+0x25/0x30 [562074.397218] 内存信息： [562074.399103] 节点 0 每 CPU DMA： [562074.400605]CPU 0：hi：0，btch：1 美元：0 [562074.402129] CPU 1：hi：0，btch：1 美元：0 [562074.403506] 节点 0 DMA32 每个 CPU： [562074.405629] CPU 0：hi：186，btch：31 美元：0 [562074.406749] CPU 1：hi：186，btch：31 美元：30 [562074.407914] 节点 0 每 CPU 正常： [562074.409325] CPU 0：hi：186，btch：31 美元：0 [562074.410787] CPU 1：hi：186，btch：31 美元：30 [562074.411941] 活跃匿名：717855 非活跃匿名：182000 孤立匿名：5083 [562074.411943]活动文件：32非活动文件：192孤立文件：64 [562074.411944] 不可驱逐：0 脏：0 写回：259 不稳定：0 [562074.411944] 释放：21353 slab_reclaimable：5762 slab_unreclaimable：10906 [562074.411945] 映射：63 共享内存：43 页表：21560 跳出率：0 [562074.418218] 节点 0 DMA 可用：15724kB 最小：248kB 低：308kB 高：372kB active_anon：0kB inactive_anon：0kB active_file：0kB inactive_file：0kB unevictable：0kB 隔离（匿名）：0kB 隔离（文件）：0kB 存在：15312kB mlocked：0kB dirty：0kB writeback：0kB 映射：0kB shmem：0kB slab_reclaimable：0kB slab_unreclaimable：0kB kernel_stack：0kB pagetables：0kB 不稳定：0kB bounce：0kB writeback_tmp：0kB pages_scanned：0 all_unreclaimable？是 [562074.424382]lowmem_reserve[]: 0 3512 4017 4017 [562074.426602] 节点 0 DMA32 可用：61060kB 最小：58868kB 低：73584kB 高：88300kB active_anon：2667432kB inactive_anon：524160kB active_file：4kB inactive_file：396kB 不可驱逐：0kB 隔离（匿名）：15980kB 隔离（文件）：384kB 存在：3596500kB mlocked：0kB 脏：0kB 回写：560kB 映射：232kB shmem：172kB slab_reclaimable：15184kB slab_unreclaimable：17264kB kernel_stack：6736kB 页表：66680kB 不稳定：0kB 反弹：0kB writeback_tmp：0kB pages_scanned:64 all_unreclaimable？否 [562074.432920] lowmem_reserve[]: 0 0 505 505 [562074.434773] 节点 0 正常 空闲：8392kB 最小：8464kB 低：10580kB 高：12696kB active_anon：203604kB inactive_anon：203616kB active_file：124kB inactive_file：320kB 不可驱逐：0kB 隔离（匿名）：4992kB 隔离（文件）：128kB 存在：517120kB mlocked：0kB 脏：0kB 回写：476kB 映射：20kB shmem：0kB slab_reclaimable：7864kB slab_unreclaimable：26360kB kernel_stack：6832kB 页表：19560kB 不稳定：0kB 反弹：0kB writeback_tmp：0kB pages_scanned：32 all_unreclaimable？否 [562074.441562]lowmem_reserve[]: 0 0 0 0 [562074.443251] 节点 0 DMA：3*4kB 0*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15724kB [562074.449972] 节点 0 DMA32：492*4kB 4530*8kB 511*16kB 140*32kB 89*64kB 3*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 60784kB [562074.455851] 节点 0 正常：1455*4kB 68*8kB 3*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 8396kB [562074.461844] 共 22462 个页面缓存页面 [562074.463320] 交换缓存中有 21989 个页面 [562074.464571] 交换缓存统计信息：添加 34726956，删除 34704967，查找 25405257/28809084 [562074.466666] 可用交换 = 0kB [562074.467793] 总交换 = 4194252kB [562074.508867] 1048575 页 RAM [562074.510066] 保留 67960 页 [562074.511918] 7658 页共享 [562074.513082] 947230 页未共享 [562074.514178] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj 名称 [562074.516236] [ 490] 0 490 2680 0 1 -17 -1000 udevd [562074.518561] [ 790] 0 790 2662 0 1 -17 -1000 udevd [562074.520752] [ 1342] 0 1342 62368 50 1 0 0 rsyslogd [562074.522842] [ 1357] 0 1357 4563 23 0 0 0 中断平衡 [562074.525130] [ 1376] 25 1376 59678 129 0 0 0 命名 [562074.527174] [ 1394] 81 1394 5359 1 0 0 0 dbus 守护进程 [562074.529698] [ 1432] 0 1432 1020 0 0 0 0 acpid [562074.532021] [ 1451] 28 1451 336791 177 0 0 0 nscd [562074.534053] [ 1569] 0 1569 16086 24 1 -17 -1000 sshd [562074.536171] [ 1580] 38 1580 6262 27 1 0 0 ntpd [562074.539282] [ 2053] 0 2053 4662 1 0 0 0 鸽舍 [562074.542072] [ 2065] 401 2065 10639 1 0 0 0 pop3 登录 [562074.545036] [ 2066] 401 2066 10637 1 0 0 0 imap 登录 [562074.547647] [ 2067] 97 2067 3368 1 0 0 0 铁砧 [562074.550029] [ 2068] 0 2068 3401 1 1 0 0 对数 [562074.552190] [ 2070] 401 2070 10655 1 0 0 0 pop3 登录 [562074.554224] [ 2071] 401 2071 10670 1 0 0 0 imap 登录 [562074.556886] [ 2072] 0 2072 3659 1 0 0 0 配置 [562074.560112] [ 2082] 47 2082 18238 1 1 0 0 进出口 [562074.562160] [ 2100] 0 2100 49742 335 1 0 0 /usr/local/cpan [562074.564197] [ 2102] 0 2102 53119 223 0 0 0 spamd 子项 [562074.566209] [ 2129] 0 2129 45352 1 0 0 0 abrtd [562074.568273] [ 2139] 0 2139 45197 1 0 0 0 abrt-dump-oops [562074.570489] [ 2150] 0 2150 19189 124 0 0 0 httpd [562074.572607] [ 2165] 0 2165 34191 0 0 0 0 纯 ftpd [562074.574719] [ 2167] 0 2167 34036 1 0 0 0 纯认证 [562074.577609] [ 2260] 0 2260 29271 27 1 0 0 克朗德 [562074.580212] [ 2275] 0 2275 4857 5 1 0 0 atd [562074.582301] [ 2375] 0 2375 26952 125 0 0 0 cpsrvd (SSL) - [562074.584483] [ 2381] 0 2381 8172 69 0 0 0 队列进程-wa [562074.586500] [ 2391] 0 2391 27506 53 0 0 0 dnsadmin - 服务 [562074.589509] [ 2419] 0 2419 20822 258 0 0 0 tailwatchd [562074.592045] [ 2433] 0 2433 19586 1 0 0 0 cPhulkd - 进程 [562074.594295] [2476] 0 2476 33028 60 0 0 0 cpdavd-接受 [562074.597210] [ 2512] 0 2512 5915 102 1 0 0 cpanellogd - sl [562074.600503] [ 2540] 0 2540 1016 1 1 0 0 明盖蒂 [562074.603689] [ 2542] 0 2542 1016 1 0 0 0 明盖蒂 [562074.610302] [ 2545] 0 2545 1016 1 1 0 0 明盖蒂 [562074.615154] [ 2547] 0 2547 2679 0 0 -17 -1000 udevd [562074.617989] [ 2548] 0 2548 1016 1 1 0 0 明盖蒂 [562074.622920] [ 2550] 0 2550 1016 1 1 0 0 明盖蒂 [562074.625853] [ 2552] 0 2552 1016 1 1 0 0 明盖蒂 [562074.628530] [30499] 0 30499 147473 23 0 0 0 clamd [562074.631934] [ 8712] 0 8712 2394 1 1 0 0 mysqld_safe [562074.634865] [ 9408] 402 9408 564397 42943 0 0 0 mysqld [562074.637620] [24553] 0 24553 27283 1 1 0 0 sshd [562074.641062] [24562] 0 24562 13893 1 0 0 0 sftp 服务器 [562074.643785] [25296] 0 25296 27283 1 1 0 0 sshd [562074.646513] [25312] 0 25312 26649 1 0 0 0 重击 [562074.649571] [25954] 99 25954 19222 60 0 0 0 httpd [562074.652579] [25955] 0 25955 19061 1 0 0 0 leechprotect [562074.655407] [25958] 99 25958 334134 795 0 0 0 httpd [562074.657912] [25959] 99 25959 334193 806 1 0 0 httpd [562074.660551] [25960] 99 25960 334438 802 0 0 0 httpd [562074.663110] [26044] 99 26044 334439 803 0 0 0 httpd [562074.665618] [26206] 99 26206 334191 844 1 0 0 httpd [562074.668698] [26361] 99 26361 334134 886 0 0 0 httpd [562074.671241] [28324] 0 28324 27283 1 1 0 0 sshd [562074.673759] [28333] 0 28333 26649 1 0 0 0 重击 [562074.676257] [28546] 99 28546 334167 821 0 0 0 httpd [562074.702428] [ 1091] 0 1091 27283 1 1 0 0 sshd [562074.704538] [ 1130] 0 1130 26649 1 0 0 0 重击 [562074.706652] [ 5276] 500 5276 77474 7801 1 0 0 php [562074.708866] [ 5278] 500 5278 77582 7649 0 0 0 php [562074.711662] [ 5281] 500 5281 77580 7259 1 0 0 php [562074.714235] [ 5290] 500 5290 77601 7474 1 0 0 php [562074.716490] [ 5295] 500 5295 77538 7540 1 0 0 php [562074.718824] [ 5299] 500 5299 77414 7551 1 0 0 php [562074.721384] [ 5302] 500 5302 77518 8104 1 0 0 php [562074.724114] [ 5304] 500 5304 77414 7522 1 0 0 php [562074.726653] [ 5308] 500 5308 77582 7241 0 0 0 php [562074.729482] [ 5309] 500 5309 77542 7294 0 0 0 php [562074.731758] [ 5367] 500 5367 79215 8223 0 0 0 php [562074.734948] [ 5369] 500 5369 79230 7073 1 0 0 php [562074.738000] [ 5389] 500 5389 74670 6863 1 0 0 php [562074.740609] [ 5407] 500 5407 74996 5832 0 0 0 php [562074.742809] [ 5415] 500 5415 75850 6233 1 0 0 php [562074.745813] [ 5416] 500 5416 75061 5948 1 0 0 php [562074.748265] [ 5418] 500 5418 74992 5796 1 0 0 php [562074.750694] [ 5420] 500 5420 76190 7332 0 0 0 php [562074.753004] [ 5430] 500 5430 74932 6113 1 0 0 php [562074.755384] [ 5434] 500 5434 75460 6480 1 0 0 php [562074.757639] [ 5439] 500 5439 74928 5879 1 0 0 php [562074.761562] [ 5440] 500 5440 75125 5701 1 0 0 php [562074.764306] [ 5442] 500 5442 75914 6693 1 0 0 php [562074.766600] [ 5446] 500 5446 75214 5475 1 0 0 php [562074.768955] [ 5448] 500 5448 74705 6309 1 0 0 php [562074.771372] [ 5450] 500 5450 73976 6564 0 0 0 php [562074.773778] [ 5451] 500 5451 73831 6459 1 0 0 php [562074.776178] [ 5458] 500 5458 74611 7603 0 0 0 php [562074.778666] [ 5459] 500 5459 75781 8240 1 0 0 php [562074.781649] [ 5465] 500 5465 73772 7310 1 0 0 php [562074.784412] [ 5470] 500 5470 74414 7797 1 0 0 php [562074.788349] [ 5476] 500 5476 74864 8214 1 0 0 php [562074.790625] [ 5480] 500 5480 73510 8366 0 0 0 php [562074.793613] [ 5482] 500 5482 73708 8698 0 0 0 php [562074.796689] [ 5492] 500 5492 72444 11027 1 0 0 php [562074.799184] [ 5496] 500 5496 72784 12415 1 0 0 php [562074.802301] [ 5499] 500 5499 73768 11388 0 0 0 php [562074.804993] [ 5500] 500 5500 72582 11302 0 0 0 php [562074.808597] [ 5503] 500 5503 70752 9687 0 0 0 php [562074.810941] [ 5513] 500 5513 73447 10543 0 0 0 php [562074.813474] [ 5519] 500 5519 72728 12005 1 0 0 php [562074.815596] [ 5520] 500 5520 72710 11134 1 0 0 php [562074.818435] [ 5523] 500 5523 72246 11313 0 0 0 php [562074.821278] [ 5533] 500 5533 73576 12705 0 0 0 php [562074.824227] [ 5536] 500 5536 70305 11674 1 0 0 php [562074.826292] [ 5541] 500 5541 73378 13119 1 0 0 php [562074.828638] [ 5545] 500 5545 69769 10932 1 0 0 php [562074.831461] [ 5548] 500 5548 69382 10296 0 0 0 php [562074.833803] [ 5549] 500 5549 73458 13978 1 0 0 php [562074.836589] [ 5564] 500 5564 69708 11383 1 0 0 php [562074.839962] [ 5568] 500 5568 69380 12267 0 0 0 php [562074.842304] [ 5570] 500 5570 69580 13324 1 0 0 php [562074.844533] [ 5574] 500 5574 69057 13993 0 0 0 php [562074.847082] [ 5577] 500 5577 69450 13875 1 0 0 php [562074.849309] [ 5578] 500 5578 70048 14829 1 0 0 php [562074.851653] [ 5587] 500 5587 69514 15274 1 0 0 php [562074.854140] [ 5589] 500 5589 69382 16196 1 0 0 php [562074.856537] [ 5593] 500 5593 68012 14763 0 0 0 php [562074.858820] [ 5597] 500 5597 71343 18344 0 0 0 php [562074.861408] [ 5598] 500 5598 66115 12838 1 0 0 php [562074.864285] [ 5603] 500 5603 69319 16732 1 0 0 php [562074.866510] [ 5612] 500 5612 63573 11969 0 0 0 php [562074.868753] [ 5613] 500 5613 61574 9964 1 0 0 php [562074.871009] [ 5616] 500 5616 63637 12150 0 0 0 php [562074.874032] [ 5624] 500 5624 63726 12707 1 0 0 php [562074.876563] [ 5625] 500 5625 55276 4013 1 0 0 php [562074.879156] [ 5630] 500 5630 52951 2211 0 0 0 php [562074.881476] [ 5631] 500 5631 53736 3070 1 0 0 php [562074.883968] [ 5633] 500 5633 53245 2474 0 0 0 php [562074.886305] [ 5634] 500 5634 54533 3868 1 0 0 php [562074.888585] [ 5635] 500 5635 54650 3839 0 0 0 php [562074.890872] [ 5636] 500 5636 56802 6104 0 0 0 php [562074.893228] [ 5637] 500 5637 54189 3375 1 0 0 php [562074.895530] [ 5638] 500 5638 54563 3901 0 0 0 php [562074.897784] [ 5639] 500 5639 52667 1958 1 0 0 php [562074.900122] [ 5640] 500 5640 54533 3711 0 0 0 php [562074.902475] [ 5641] 500 5641 52922 2197 0 0 0 php [562074.904697] [ 5642] 500 5642 52667 1954 1 0 0 php [562074.907013] [ 5643] 500 5643 52958 2226 0 0 0 php [562074.909381] [ 5644] 500 5644 52667 1927 0 0 0 php [562074.911628] [ 5645] 500 5645 54015 3402 1 0 0 php [562074.913914] [ 5646] 500 5646 52951 2165 0 0 0 php [562074.916614] [ 5647] 500 5647 54635 3863 1 0 0 php [562074.919109] [ 5648] 99 5648 219408 400 1 0 0 httpd [562074.921510] [ 5649] 500 5649 53928 3331 1 0 0 php [562074.924011] [ 5650] 500 5650 56653 5958 0 0 0 php [562074.926651] [ 5651] 500 5651 52951 2223 0 0 0 php [562074.929157] [ 5652] 500 5652 52735 1997 1 0 0 php [562074.954567] [ 5653] 500 5653 52457 1707 1 0 0 php [562074.957251] [ 5654] 99 5654 235792 481 1 0 0 httpd [562074.959480] [ 5660] 500 5660 53928 3321 0 0 0 php [562074.961682] [ 5686] 500 5686 54417 3563 0 0 0 php [562074.964573] [ 5712] 500 5712 52958 2216 0 0 0 php [562074.966804] [ 5713] 500 5713 53887 3247 0 0 0 php [562074.969044] [ 5714] 500 5714 52951 2206 0 0 0 php [562074.971840] [ 5715] 500 5715 53736 3103 1 0 0 php [562074.974827] [ 5716] 500 5716 52663 1938 1 0 0 php [562074.977527] [ 5717] 500 5717 56118 5385 0 0 0 php [562074.980867] [ 5726] 500 5726 57172 6396 1 0 0 php [562074.984658] [ 5727] 500 5727 52523 1760 0 0 0 php [562074.987267] [ 5728] 500 5728 52587 1869 1 0 0 php [562074.989641] [ 5730] 500 5730 52430 1711 1 0 0 php [562074.992423] [ 5731] 500 5731 53672 3061 0 0 0 php [562074.995140] [ 5732] 500 5732 52587 1860 1 0 0 php [562074.998136] [ 5733] 500 5733 52770 2016 0 0 0 php [562075.002104] [ 5734] 500 5734 53216 2537 1 0 0 php [562075.004987] [ 5735] 500 5735 54351 3615 1 0 0 php [562075.007912] [ 5737] 500 5737 54329 3665 0 0 0 php [562075.010646] [ 5738] 500 5738 53864 3231 1 0 0 php [562075.013388] [ 5750] 500 5750 53992 3332 1 0 0 php [562075.016259] [ 5751] 500 5751 53992 3356 1 0 0 php [562075.018239] [ 5754] 500 5754 52457 1707 0 0 0 php [562075.020361] [ 5755] 500 5755 53672 3008 0 0 0 php [562075.022769] [ 5758] 500 5758 53928 3334 1 0 0 php [562075.024717] [ 5759] 500 5759 53800 3153 0 0 0 php [562075.026820] [ 5760] 500 5760 52457 1715 1 0 0 php [562075.029743] [ 5761] 500 5761 53672 2999 1 0 0 php [562075.031868] [ 5767] 500 5767 52430 1709 0 0 0 php [562075.033922] [ 5768] 500 5768 53672 3015 1 0 0 php [562075.035960] [ 5772] 500 5772 52457 1708 1 0 0 php [562075.038017] [ 5774] 500 5774 53928 3307 0 0 0 php [562075.040342] [ 5777] 500 5777 52430 1709 0 0 0 php [562075.042585] [ 5780] 500 5780 52922 2200 1 0 0 php [562075.044900] [ 5781] 500 5781 29874 419 0 0 0 php [562075.046949] [ 5782] 500 5782 52430 1710 0 0 0 php [562075.049163] [ 5787] 500 5787 53736 3111 0 0 0 php [562075.051348] [ 5788] 500 5788 52428 1681 0 0 0 php [562075.053411] [ 5789] 500 5789 16226 45 1 0 0 php [562075.055455] [ 5790] 500 5790 53992 3361 0 0 0 php [562075.057385] [ 5796] 500 5796 53800 3145 0 0 0 php [562075.059354] [ 5802] 500 5802 51933 1164 0 0 0 php [562075.061403] [ 5803] 500 5803 53736 3088 1 0 0 php [562075.063319] [ 5828] 500 5828 52457 1712 1 0 0 php [562075.065328] [ 5830] 500 5830 53992 3362 1 0 0 php [562075.067304] [ 5833] 500 5833 52457 1711 0 0 0 php [562075.069272] [ 5834] 500 5834 52457 1710 0 0 0 php [562075.071342] [ 5838] 500 5838 25576 65 0 0 0 php [562075.073811] [ 5839] 99 5839 121104 248 1 0 0 httpd [562075.075876] [ 5848] 99 5848 121104 279 1 0 0 httpd [562075.078224] [ 5849] 500 5849 29230 282 1 0 0 php [562075.080532] [ 5850] 500 5850 29118 217 1 0 0 php [562075.082861] [ 5851] 500 5851 53352 2643 0 0 0 php [562075.084933] [ 5852] 99 5852 121104 281 1 0 0 httpd [562075.087091] [ 5854] 500 5854 52430 1708 0 0 0 php [562075.089180] [ 5856] 500 5856 51703 882 0 0 0 php [562075.091154] [ 5863] 99 5863 121104 286 1 0 0 httpd [562075.093112] [ 5970] 500 5970 28797 186 1 0 0 php [562075.095140] [ 5986] 0 5986 5651 60 0 0 0 upcp 运行 [562075.097396] [ 5988] 500 5988 29118 223 0 0 0 php [562075.099621] [ 5991] 99 5991 104720 241 0 0 0 httpd [562075.101929] [ 5993] 500 5993 3116 43 0 0 0 suphp [562075.104055] [ 5994] 500 5994 29329 408 0 0 0 php [562075.106361] [ 6026] 500 6026 29118 223 0 0 0 php [562075.108660] [6027] 500 6027 29118 231 1 0 0 php [562075.111327] [6028] 500 6028 7365 23 1 0 0 php [562075.113563] [ 6032] 99 6032 121104 279 1 0 0 httpd [562075.115682] 内存不足：终止进程 9408 (mysqld) 得分 44 或牺牲子进程 [562075.117786] 已终止进程 9408，UID 402，(mysqld) total-vm：2257588kB，anon-rss：171860kB，file-rss：544kB
这种情况一天只发生一两次。否则，负载完全在限制范围内（<2.00）。当发生此错误时，负载达到>100。根据访问日志，当发生这种情况时，我没有获得更多流量。我应该怎么做才能解决这个问题？