CPU 负载高、核心使用率低、内核中存在(ECC)内存错误

CPU 负载高、核心使用率低、内核中存在(ECC)内存错误

我遇到了一种非常奇怪的现象...我的计算机的 CPU 负载急剧上升(8 核机器上为 >4),但没有任何进程占用太多 CPU(见附图)尽管机器的 8 个核心负载很高(htop 显示它们的负载都在 30-70% 之间波动。

CpuLoad 顶部输出

这种现象会在使用电脑X分钟后出现(随机,从几分钟到几小时不等)。而且,这种情况发生后,电脑最终会死机。

我很困惑,我在 15.04 上遇到了这个问题,更新到 15.10 后还是一样。

该机器有以下部件:主板:华硕 Z10PE-D8WS CPU:英特尔(R)至强(R)CPU E5-1620 v3 @ 3.50GHz 内存:2x 金士顿 16Go PC4-2133 CL15 - ECC Registered(KVR21R15D4/16) 硬盘:2x 2To ATA ST2000DM001-1ER1 在 Raid 0 中

我发现的唯一奇怪的事情是内核日志中的这些行:

Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17386.894665] CMCI storm detected: switching to poll mode
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.299974] EDAC MC0: 4 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x1042 offset:0x100 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:0)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.299989] EDAC MC0: 4 CE memory read error on CPU_SrcID#0_Ha#0_Chan#0_DIMM#0 (channel:0 slot:0 page:0x85392b offset:0xa80 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:1 rank:1)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.299999] EDAC MC0: 2 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x850da9 offset:0x580 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:1)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300009] EDAC MC0: 3 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x85f599 offset:0x100 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:1)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300018] EDAC MC0: 3 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x11b2 offset:0x780 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:0)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300022] EDAC MC0: 2 CE Error at MMIOH area, on addr 0x000000087fd43a40 on any memory ( page:0x0 offset:0x0 grain:32 syndrome:0x0)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300032] EDAC MC0: 1 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x8474e2 offset:0xf00 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:0)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300042] EDAC MC0: 1 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x8476f8 offset:0xd80 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:1)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300051] EDAC MC0: 2 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x8466eb offset:0x500 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:1)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300060] EDAC MC0: 1 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x846b23 offset:0x7c0 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:0)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300070] EDAC MC0: 1 CE memory read error on CPU_SrcID#0_Ha#0_Chan#0_DIMM#0 (channel:0 slot:0 page:0x846b23 offset:0xcc0 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:1 rank:0)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300080] EDAC MC0: 1 CE memory read error on CPU_SrcID#0_Ha#0_Chan#0_DIMM#0 (channel:0 slot:0 page:0x846d32 offset:0xe40 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:1 rank:0)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300089] EDAC MC0: 2 CE memory read error on CPU_SrcID#0_Ha#0_Chan#0_DIMM#0 (channel:0 slot:0 page:0x5c251b offset:0x640 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:1 rank:1)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300099] EDAC MC0: 1 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x8474e3 offset:0x1c0 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:0)
Feb 15 18:46:02 XXXX-Z10PE-D8-WS kernel: [17387.300108] EDAC MC0: 1 CE memory read error on CPU_SrcID#0_Ha#0_Chan#1_DIMM#0 (channel:1 slot:0 page:0x847711 offset:0xf40 grain:32 syndrome:0x0 -  area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:2 rank:0)
Feb 15 18:46:03 XXXX-Z10PE-D8-WS kernel: [17387.891537] EDAC sbridge MC0: HANDLING MCE MEMORY ERROR
Feb 15 18:46:03 XXXX-Z10PE-D8-WS kernel: [17387.891561] EDAC sbridge MC0: CPU 0: Machine Check Event: 0 Bank 7: cc08388000010090
Feb 15 18:46:03 XXXX-Z10PE-D8-WS kernel: [17387.891566] EDAC sbridge MC0: TSC 0 
Feb 15 18:46:03 XXXX-Z10PE-D8-WS kernel: [17387.891569] EDAC sbridge MC0: ADDR 87fc60500 EDAC sbridge MC0: MISC 14032b286 
Feb 15 18:46:03 XXXX-Z10PE-D8-WS kernel: [17387.891576] EDAC sbridge MC0: PROCESSOR 0:306f2 TIME 1455579963 SOCKET 0 APIC 0
Feb 15 18:46:03 XXXX-Z10PE-D8-WS kernel: [17388.299184] EDAC MC0: 8418 CE Error at MMIOH area, on addr 0x000000087fc60500 on any memory ( page:0x0 offset:0x0 grain:32 syndrome:0x0)
Feb 15 18:51:03 XXXX-Z10PE-D8-WS kernel: [17687.707744] CMCI storm subsided: switching to interrupt mode

这些台词重复了很多次

Feb 15 19:07:47 XXXX-Z10PE-D8-WS kernel: [18691.236569] EDAC sbridge MC0: HANDLING MCE MEMORY ERROR
Feb 15 19:07:47 XXXX-Z10PE-D8-WS kernel: [18691.236586] EDAC sbridge MC0: CPU 0: Machine Check Event: 0 Bank 7: cc00064000010090
Feb 15 19:07:47 XXXX-Z10PE-D8-WS kernel: [18691.236589] EDAC sbridge MC0: TSC 0 
Feb 15 19:07:47 XXXX-Z10PE-D8-WS kernel: [18691.236592] EDAC sbridge MC0: ADDR 103fb00 EDAC sbridge MC0: MISC 4062e286 
Feb 15 19:07:47 XXXX-Z10PE-D8-WS kernel: [18691.236597] EDAC sbridge MC0: PROCESSOR 0:306f2 TIME 1455581267 SOCKET 0 APIC 0

间隔一些

Feb 15 19:07:48 XXXX-Z10PE-D8-WS kernel: [18692.381405] EDAC MC0: 26415 CE memory read error on CPU_SrcID#0_Ha#0_Chan#0_DIMM#0 (channel:0 slot:0 page:0x1042 offset:0xa00 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0001:0090 socket:0 ha:0 channel_mask:1 rank:0)
Feb 15 19:07:48 XXXX-Z10PE-D8-WS kernel: [18692.381481] EDAC MC0: 4 CE memory scrubbing error on CPU_SrcID#0_Ha#0_Chan#0_DIMM#0 (channel:0 slot:0 page:0x7c5acf offset:0x0 grain:32 syndrome:0x0 -  OVERFLOW area:DRAM err_code:0008:00c1 socket:0 ha:0 channel_mask:1 rank:1)

帮助 !

答案1

仍在跟踪此问题?似乎您的内存模块有问题,机器暂停等待硬件自行纠正此错误。您可能需要尝试移除或更换第一个 CPU、第二个通道和第一个插槽处的内存。请参考:https://serverfault.com/questions/569289/server-freezes-completely-in-unknown-condition

希望能帮助到你。

答案2

谢谢你提醒我完成这件事!

确实,看了这些行之后,我注意到:slot:0 是问题所在。我以为是内存坏了,就把它拔了出来(插槽是由主板分配的,或者至少在我的主板上,插槽 0 是主板的插槽 1)

于是我把它拿出来,测试了 48 小时,没有出现任何错误。将 RAM 送去保修,拿回了一个新的。

仙境里一切都是完美的!

相关内容