这是运行 Apache2 和 Postfix 作为服务的邮件服务器。它已经运行了 135 天,此后服务没有任何变化。通常的平均负载约为 0.xx,但一周前它持续飙升至 1.00 左右。我在网上查看了所有关于这个问题的解决方案,并彻底调查了常见的罪魁祸首,例如 IO 队列、CPU 利用率等。在我看来一切正常。
该服务器在某些 ESXI VMWare 服务器上作为 VM 运行。其他工程师可能已在虚拟机管理程序级别执行了迁移。迁移的目的是更新主机服务器固件版本。之后,此 VM 被迁移回升级后的主机。迁移是无缝的,这意味着此 VM 的运行在此过程中不会中断。
uname -a
Linux mail.net 4.9.0-040900rc2-generic #201610232131 SMP Mon Oct 24 01:32:49 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
处理器
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 1
On-line CPU(s) list: 0
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Stepping: 2
CPU MHz: 2399.569
BogoMIPS: 4799.99
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 20480K
NUMA node0 CPU(s): 0
顶部
top - 17:12:23 up 135 days, 6:37, 1 user, load average: 1.00, 1.00, 1.00
Tasks: 154 total, 1 running, 153 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 12303728 total, 10122732 used, 2180996 free, 215672 buffers
KiB Swap: 1048572 total, 0 used, 1048572 free. 1321424 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 35604 4024 2660 S 0.0 0.0 0:13.76 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.92 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 2:18.67 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
7 root 20 0 0 0 0 S 0.0 0.0 4:44.58 rcu_sched
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
10 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 lru-add-drain
11 root rt 0 0 0 0 S 0.0 0.0 0:18.56 watchdog/0
12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/0
13 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kdevtmpfs
14 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 netns
15 root 20 0 0 0 0 S 0.0 0.0 0:05.61 khungtaskd
16 root 20 0 0 0 0 S 0.0 0.0 0:00.00 oom_reaper
17 root 0 -20 0 0 0 S 0.0 0.0 0:00.24 writeback
18 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kcompactd0
19 root 25 5 0 0 0 S 0.0 0.0 0:00.00 ksmd
20 root 39 19 0 0 0 S 0.0 0.0 0:43.27 khugepaged
21 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 crypto
22 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kintegrityd
网络状态-natp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN 1546/rsync
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 15493/mysqld
tcp 0 0 127.0.0.1:10031 0.0.0.0:* LISTEN 1524/perl
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1058/sshd
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 1409/master
tcp 0 0 0.0.0.0:10080 0.0.0.0:* LISTEN 1066/xinetd
tcp 0 0 203.92.11.241:25 43.245.16.27:1304 TIME_WAIT -
tcp 0 0 203.92.11.241:25 43.245.16.27:1307 ESTABLISHED 29082/smtpd
tcp 0 0 127.0.0.1:10031 127.0.0.1:52966 TIME_WAIT -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52956 TIME_WAIT -
tcp 0 0 127.0.0.1:52984 127.0.0.1:10031 CLOSE_WAIT 29082/smtpd
tcp 0 384 203.92.11.241:22 203.92.90.66:56408 ESTABLISHED 28753/sshd: xxx [
tcp 0 0 203.92.11.241:25 43.245.16.27:1305 TIME_WAIT -
tcp 0 0 203.92.11.241:25 203.92.81.184:8509 TIME_WAIT -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52962 TIME_WAIT -
tcp 1 0 127.0.0.1:52982 127.0.0.1:10031 CLOSE_WAIT 29107/smtpd
tcp 0 0 203.92.11.241:25 43.245.17.28:52703 TIME_WAIT -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52958 TIME_WAIT -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52978 TIME_WAIT -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52988 TIME_WAIT -
tcp 0 0 203.92.11.241:47072 203.92.86.79:389 ESTABLISHED 1079/sssd_be
tcp 0 0 127.0.0.1:10031 127.0.0.1:52982 FIN_WAIT2 -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52952 TIME_WAIT -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52984 FIN_WAIT2 -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52974 TIME_WAIT -
tcp 0 0 203.92.11.241:25 43.245.17.28:61916 TIME_WAIT -
tcp 0 0 127.0.0.1:10031 127.0.0.1:52968 TIME_WAIT -
tcp6 0 0 :::873 :::* LISTEN 1546/rsync
tcp6 0 0 :::80 :::* LISTEN 1788/apache2
tcp6 0 0 :::22 :::* LISTEN 1058/sshd
tcp6 0 0 :::25 :::* LISTEN 1409/master
iostat
Linux 4.9.0-040900rc2-generic (mail1.lga.net.sg) 03/10/19 _x86_64_ (1 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.14 0.02 0.12 0.19 0.00 99.53
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 1.26 0.07 24.53 816686 286303390
dm-0 2.04 0.07 24.53 796673 286303376
dm-1 0.00 0.00 0.00 896 0
虚拟机状态 5
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 0 0 2180892 215672 1321628 0 0 0 25 0 3 0 0 100 0 0
0 0 0 2180768 215672 1321428 0 0 0 0 64 140 0 0 100 0 0
0 0 0 2180768 215672 1321432 0 0 0 115 71 157 0 0 95 5 0
0 0 0 2180768 215672 1321432 0 0 0 12 59 125 0 0 100 0 0
0 0 0 2180768 215672 1321432 0 0 0 2 62 132 0 0 100 0 0
0 0 0 2180768 215672 1321432 0 0 0 6 62 133 0 0 100 0 0
0 0 0 2185960 215672 1321432 0 0 0 0 60 131 0 0 100 0 0
0 0 0 2179896 215672 1321432 0 0 0 46 123 267 1 0 99 0 0
0 0 0 2180392 215672 1321432 0 0 0 10 66 142 0 0 100 0 0
0 0 0 2180392 215672 1321432 0 0 0 132 71 151 0 0 93 7 0
0 0 0 2180392 215672 1321432 0 0 0 4 64 134 0 0 99 0 0
0 0 0 2180392 215672 1321432 0 0 0 13 65 133 0 0 100 0 0
0 0 0 2181548 215672 1321432 0 0 0 0 64 137 0 0 100 0 0