寻求一些专家建议。我是第一次在自己的服务器上担任系统管理员，我无法找出服务器的瓶颈。
我从 CSF 收到大量 5 分钟高负载平均警报，范围在 8 - 80 之间
所以我继续在我的服务器上安装了 sqltuner，并让它运行了 3 天
结果没有显示任何异常，但每天的平均负载仍然很高
我正在尝试找到瓶颈（CPU、内存不足问题导致的负载或 I/O 绑定负载）
如果有人能分享任何见解，将会非常兴奋...
（我在下面附上了 sqltuner 的报告和高负载电子邮件输出）
-------- Security Recommendations -------------------------------------------
[OK] There are no anonymous accounts for any database users
[OK] All database users have passwords assigned
[!!] There is no basic password file list!
-------- Performance Metrics -------------------------------------------------
[--] Up for: 120d 18h 27m 20s (227M q [21.795 qps], 51M conn, TX: 907B, RX: 26B)
[--] Reads / Writes: 38% / 62%
[--] Binary logging is disabled
[--] Total buffers: 15.4G global + 4.1M per thread (600 max threads)
[OK] Maximum reached memory usage: 16.2G (51.75% of installed RAM)
[OK] Maximum possible memory usage: 17.8G (56.91% of installed RAM)
[OK] Slow queries: 0% (11/227M)
[OK] Highest usage of available connections: 33% (199/600)
[OK] Aborted connections: 0.56% (284327/51183230)
[OK] Query cache efficiency: 83.0% (78M cached / 94M selects)
[!!] Query cache prunes per day: 10288
[OK] Sorts requiring temporary tables: 0% (392 temp sorts / 1M sorts)
[OK] Temporary tables created on disk: 4% (65K on disk / 1M total)
[OK] Thread cache hit rate: 99% (199 created / 51M connections)
[OK] Table cache hit rate: 22% (425 open / 1K opened)
[OK] Open file limit used: 0% (433/50K)
[OK] Table locks acquired immediately: 99% (41M immediate / 41M locks)
-------- MyISAM Metrics -----------------------------------------------------
[!!] Key buffer used: 20.2% (108M used / 536M cache)
[OK] Key buffer size / total MyISAM indexes: 512.0M/14.7M
[OK] Read Key buffer hit rate: 99.8% (51M cached / 121K reads)
[!!] Write Key buffer hit rate: 40.8% (4M cached / 2M writes)
-------- InnoDB Metrics -----------------------------------------------------
[--] InnoDB is enabled.
[OK] InnoDB buffer pool / data size: 14.6G/140.9M
[!!] InnoDB buffer pool instances: 1
[!!] InnoDB Used buffer: 3.39% (32546 used/ 959999 total)
[OK] InnoDB Read buffer efficiency: 100.00% (5437258684 hits/ 5437259670 total)
[!!] InnoDB Write buffer efficiency: 0.00% (0 hits/ 1 total)
[OK] InnoDB log waits: 0.00% (0 waits / 24069213 writes)
-------- AriaDB Metrics -----------------------------------------------------
[--] AriaDB is disabled.
-------- Replication Metrics -------------------------------------------------
[--] No replication slave(s) for this server.
[--] This is a standalone server..
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Increasing the query_cache size over 128M may reduce performance
Variables to adjust:
query_cache_size (> 128M) [see warning above]
innodb_buffer_pool_instances(=14)
----------------------
(The only change I've made is to reduce InnoDB size and add multiple pool instances)
每日负载较高的电子邮件：
Time: Sun Dec 6 05:43:53 2015 -0500
1 Min Load Avg: 80.26
5 Min Load Avg: 21.19
15 Min Load Avg: 7.46
Running/Total Processes: 221/875
ps.txt
O
Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process
Srv PID Acc M CPU SS Req Conn Child Slot Client VHost Request
0-1424 16243 0/149/2713129 W 1.28 14 0 0.0 2.34 65107.59 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
1-1424 17057 0/18/2701770 W 2.15 4 0 0.0 0.30 62402.50 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
2-1424 17064 0/24/2685073 W 2.11 13 0 0.0 0.32 62668.14 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
3-1424 15319 0/215/2657841 W 3.50 4 0 0.0 3.88 61950.21 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
4-1424 11567 0/204/2651294 W 7.10 7 0 0.0 3.00 63562.61 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
5-1424 16512 0/37/2640191 W 2.19 5 0 0.0 0.60 63637.48 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
6-1424 17735 0/8/2630311 W 0.62 19 0 0.0 0.06 65036.68 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
7-1424 16521 0/31/2613938 W 2.20 19 0 0.0 0.36 62385.07 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
8-1424 16081 0/33/2611913 W 2.46 5 0 0.0 0.42 60535.12 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
9-1424 14711 0/120/2603042 W 1.89 18 0 0.0 2.11 59868.26 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
10-1424 16838 0/21/2592501 W 1.77 15 0 0.0 0.24 62195.33 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
11-1424 16531 0/42/2584776 W 2.45 11 0 0.0 0.39 62253.11 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
12-1424 17065 0/20/2570161 W 1.29 12 0 0.0 0.18 60474.65 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
13-1424 17770 0/13/2564128 W 1.27 2 0 0.0 0.63 59748.24 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
14-1424 17771 0/14/2542936 W 1.30 2 0 0.0 0.17 60513.73 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
15-1424 15736 0/64/2536855 W 2.91 7 0 0.0 1.16 61453.61 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
16-1424 17077 0/19/2522131 W 2.76 15 0 0.0 0.35 59307.60 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
17-1424 14723 0/93/2521068 W 3.38 6 0 0.0 1.77 60437.40 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
18-1424 16279 0/62/2509938 W 1.81 15 0 0.0 1.07 61401.24 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
19-1424 15333 0/116/2498356 W 3.24 19 0 0.0 1.69 57911.45 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
20-1424 16297 1/35/2494463 W 0.98 53 62 16.1 0.47 59474.66 58.174.24.65 suspensionrevolution.com:80 GET /new/wp-content/themes/optimizePressTheme/lib/assets/defaul
21-1424 16298 0/40/2473943 W 3.83 3 0 0.0 0.54 57987.71 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
22-1424 18054 0/6/2469193 W 1.23 1 0 0.0 0.05 59122.65 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
23-1424 12894 0/162/2458774 W 5.90 17 0 0.0 2.42 56404.92 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
24-1424 18088 0/4/2452422 W 0.90 11 0 0.0 0.00 58405.08 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
25-1424 18089 0/6/2446570 W 1.22 1 0 0.0 0.03 57036.34 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
26-1424 17079 0/30/2439491 W 1.88 0 0 0.0 0.43 54697.67 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
27-1424 16101 0/64/2416961 W 1.53 18 0 0.0 1.69 57160.43 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
28-1424 18140 0/9/2403931 W 0.62 18 0 0.0 0.02 55901.03 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
29-982 1505 1/18/1548355 G 0.14 2914733 450294 2.8 0.29 34947.04 96.47.70.4 suspensionrevolution.com:80 POST /dap/dap-clickbank-6.0.php HTTP/1.1
30-1424 15338 0/100/2384316 W 2.20 7 0 0.0 1.24 53919.77 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
31-1424 16300 2/56/2379897 K 3.47 3 1365 2.4 1.01 55195.75 89.166.18.35 appcoiner.com:80 GET /favicon.ico HTTP/1.1
32-1424 15749 0/108/2369131 W 3.26 17 0 0.0 2.09 55452.02 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
33-1424 17100 0/17/2359616 W 1.70 11 0 0.0 0.16 52564.23 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
34-1424 16310 0/162/2356424 W 3.95 15 0 0.0 2.68 55800.32 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
35-1424 16543 0/63/2326471 W 1.29 4 0 0.0 0.75 55028.80 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
36-1424 17101 0/18/2331624 W 2.05 14 0 0.0 0.21 53656.66 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
37-1424 17102 0/20/2314444 W 1.51 19 0 0.0 0.29 55684.29 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
38-1424 19665 0/1/2295814 W 0.00 3 0 0.0 0.00 52187.61 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
39-984 8727 1/69/1464486 G 0.68 2906097 450284 2.8 0.83 33844.50 74.63.153.4 suspensionrevolution.com:80 POST /dap/dap-clickbank-6.0.php HTTP/1.1
40-1424 19720 0/1/2277467 W 0.00 2 0 0.0 0.00 55864.93 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
41-1424 18141 0/6/2270838 W 0.62 14 0 0.0 0.02 54059.95 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
42-983 18177 1/49/1440183 G 0.63 2910665 450302 2.8 0.57 31224.74 74.63.153.4 suspensionrevolution.com:80 POST /dap/dap-clickbank-6.0.php HTTP/1.1
43-1424 16104 2/57/2242969 W 2.62 5 0 8.8 0.83 56170.39 54.202.7.147 appcoiner.com:80 GET /start-2/?utm_expid=111102625-1.-ThtNpCTSByWcbkMGdBOow.1&ho
44-1424 16547 0/28/2247277 W 3.84 5 0 0.0 0.31 53028.08 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
45-1424 15797 0/80/2225028 W 3.24 5 0 0.0 1.63 51333.94 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
46-1424 19721 0/1/2205346 W 0.00 2 0 0.0 0.00 52025.79 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
47-1424 18142 0/11/2207016 W 0.94 1 0 0.0 0.07 51355.07 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
48-1424 17104 0/137/2172322 W 1.28 7 0 0.0 2.32 49665.11 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
49-1424 16314 0/63/2168481 W 4.14 5 0 0.0 1.16 49191.04 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
50-1424 19763 0/0/2141243 W 2.41 12 0 0.0 0.00 49538.97 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
51-1424 17106 0/20/2137681 W 1.24 7 0 0.0 0.29 49973.70 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
52-1424 16549 0/34/2125106 W 1.99 7 0 0.0 0.50 50442.63 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
53-1424 17107 0/20/2109740 W 2.18 2 0 0.0 0.31 48074.92 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
54-1424 18143 0/4/2087977 W 1.21 8 0 0.0 0.00 49243.64 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
55-1424 17114 0/30/2062106 W 0.34 17 0 0.0 0.46 48605.36 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
56-1424 17115 0/19/2064562 W 2.07 0 0 0.0 0.30 47600.46 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
57-1424 16569 0/41/2051051 W 3.50 8 0 0.0 0.54 47547.57 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
58-1424 17116 0/28/2023150 W 1.28 4 0 0.0 0.42 49170.14 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
59-1424 17117 0/30/2010767 W 1.41 1 0 0.0 0.51 47681.03 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
60-1424 17118 0/17/1999913 R 0.05 53 5 0.0 0.26 46914.84 65.30.135.196
61-1424 16572 0/35/1978028 W 2.73 16 0 0.0 0.45 45848.82 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
62-1424 16573 0/27/1957566 W 1.60 0 0 0.0 0.45 46768.01 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
63-1424 16574 0/43/1936669 W 2.37 3 0 0.0 0.54 43520.07 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
64-1424 16575 0/28/1922381 W 1.54 1 0 0.0 0.33 45007.49 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
65-1424 16576 0/32/1903916 W 2.15 13 0 0.0 0.73 45117.83 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
66-1424 17119 0/28/1878566 W 1.18 6 0 0.0 0.45 44448.25 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
67-1424 16578 1/39/1869043 K 2.01 0 0 1.2 0.53 44966.25 114.79.47.51 suspensionrevolution.com:80 GET /favicon.ico HTTP/1.1
68-1424 16579 0/37/1841958 W 2.59 5 0 0.0 0.48 44262.26 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
这是我的 TOP 输出：
root@ns513521 [~]# top
top - 08:21:31 up 153 days, 3:51, 1 user, load average: 0.15, 0.27, 0.51
Tasks: 230 total, 2 running, 227 sleeping, 0 stopped, 1 zombie
Cpu(s): 0.4%us, 0.3%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 32855908k total, 25102984k used, 7752924k free, 886004k buffers
Swap: 1569780k total, 63984k used, 1505796k free, 21254784k cached
这是我的 ioStat 输出：
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 21.99 1476.57 549.50 19540299168 7271856568
sdb 19.53 982.85 549.49 13006647390 7271718616
sdc 19.46 978.26 549.49 12945853934 7271718616
md2 9.78 492.22 399.94 6513868322 5292584264
md1 20.05 27.80 140.15 367920858 1854711136
答案1
如果这总是发生在凌晨 4 点到 6 点之间，那么罪魁祸首可能是一些日常的 cron 作业。updatedb特别是会减慢磁盘访问速度，这会导致使用率很高的数据库超过阈值，从而增加负载。
在一般情况下，top将显示足以诊断任何高负载问题的信息，而 sqltuner 和您的电子邮件（似乎提供 http 连接）仅在问题出在您的数据库或您的 http 连接中时才有用。但是，您的顶部输出是在几个小时后获取的，问题已经消失（并且您将需要页面的其余部分）。
答案2
尝试atsar在后台运行一两天。这会将性能数据捕获到文件中以供日后检查。检查平均负载较高的时间段的数据。如果您不确定这种情况何时发生，可以从中获取平均负载值sar。
如果您还启用了进程记帐，您可能能够识别导致问题的程序。有几个程序可以最大限度地减少后台作业的影响。 nice可以限制程序使用其他空闲的 CPU 周期。 ionice可以重新安排 IO，以减少对高优先级进程的影响。但是，这两种方法都会显著增加问题程序的运行时间。
答案3
CSF 还会对文件进行 md5sum 以将其与前一天的 md5sum 进行比较。我曾见过它实际上在生产环境中导致问题（托管 300 名客户的托管公司的共享服务器）。如果您的服务器有很多文件，这可能是罪魁祸首之一。如果它发生在一天中的特定时间，Cronjobs 肯定是罪魁祸首。以下命令的输出是什么？
for u in `cat /etc/passwd | cut -d":" -f1`;do crontab -l -u $u;done