我有两台服务器，它们的设置方式完全相同（相同的 SSD、RAM 等）。它们都运行着 ESXI。其中一个服务器有一个任务主机，它对写入磁盘有很多抱怨。原来是主机没有足够的 RAM，而且正在大量使用交换。这可以解释为什么我们看到错误。SSD 被用作 RAM，无法跟上。我启动了几个实例
dd bs=2GB if=/dev/zero of=/dev/null
在盒子上使盒子进入交换状态，以便 SSD 用作内存。在盒子 1（我们遇到问题的地方）上，这花费了很多精力，但我们能够触发相同的错误。在盒子 2 上，我们一遍又一遍地运行相同的测试，只有一次我们成功得到错误。这让我相信问题出在盒子 1 上的磁盘上。我使用了 fdisk 并查看了 smartd，两个系统的报告似乎表明磁盘没有问题。我在这里遗漏了什么吗？
box1（坏盒子）：
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.4.0-26-generic] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 100 100 000 Pre-fail Always - 0
5 Reallocate_NAND_Blk_Cnt 0x0032 100 100 010 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 131
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 24
171 Program_Fail_Count 0x0032 100 100 000 Old_age Always - 0
172 Erase_Fail_Count 0x0032 100 100 000 Old_age Always - 0
173 Ave_Block-Erase_Count 0x0032 100 100 000 Old_age Always - 12
174 Unexpect_Power_Loss_Ct 0x0032 100 100 000 Old_age Always - 20
180 Unused_Reserve_NAND_Blk 0x0033 000 000 000 Pre-fail Always - 48
183 SATA_Interfac_Downshift 0x0032 100 100 000 Old_age Always - 0
184 Error_Correction_Count 0x0032 100 100 000 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
194 Temperature_Celsius 0x0022 064 049 000 Old_age Always - 36 (Min/Max 0/51)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0
197 Bogus_Current_Pend_Sect 0x0032 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 100 100 000 Old_age Always - 665
202 Percent_Lifetime_Remain 0x0030 100 100 001 Old_age Offline - 0
206 Write_Error_Rate 0x000e 100 100 000 Old_age Always - 0
210 Success_RAIN_Recov_Cnt 0x0032 100 100 000 Old_age Always - 0
246 Total_LBAs_Written 0x0032 100 100 000 Old_age Always - 3801713422
247 Host_Program_Page_Count 0x0032 100 100 000 Old_age Always - 62354066
248 FTL_Program_Page_Count 0x0032 100 100 000 Old_age Always - 39135457
root@ubuntu:~# smartctl --all /dev/sda
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.4.0-26-generic] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Crucial/Micron MX500 SSDs
Device Model: CT500MX500SSD1
Serial Number: 2002E2330303
LU WWN Device Id: 5 00a075 1e2330303
Firmware Version: M3CR023
User Capacity: 500,107,862,016 bytes [500 GB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-3 T13/2161-D revision 5
SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Jul 20 09:34:19 2020 UTC
==> WARNING: This firmware returns bogus raw values in attribute 197
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 30) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x0031) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 100 100 000 Pre-fail Always - 0
5 Reallocate_NAND_Blk_Cnt 0x0032 100 100 010 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 131
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 24
171 Program_Fail_Count 0x0032 100 100 000 Old_age Always - 0
172 Erase_Fail_Count 0x0032 100 100 000 Old_age Always - 0
173 Ave_Block-Erase_Count 0x0032 100 100 000 Old_age Always - 12
174 Unexpect_Power_Loss_Ct 0x0032 100 100 000 Old_age Always - 20
180 Unused_Reserve_NAND_Blk 0x0033 000 000 000 Pre-fail Always - 48
183 SATA_Interfac_Downshift 0x0032 100 100 000 Old_age Always - 0
184 Error_Correction_Count 0x0032 100 100 000 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
194 Temperature_Celsius 0x0022 064 049 000 Old_age Always - 36 (Min/Max 0/51)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0
197 Bogus_Current_Pend_Sect 0x0032 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 100 100 000 Old_age Always - 665
202 Percent_Lifetime_Remain 0x0030 100 100 001 Old_age Offline - 0
206 Write_Error_Rate 0x000e 100 100 000 Old_age Always - 0
210 Success_RAIN_Recov_Cnt 0x0032 100 100 000 Old_age Always - 0
246 Total_LBAs_Written 0x0032 100 100 000 Old_age Always - 3801713422
247 Host_Program_Page_Count 0x0032 100 100 000 Old_age Always - 62354066
248 FTL_Program_Page_Count 0x0032 100 100 000 Old_age Always - 39135459
SMART Error Log Version: 1
Warning: ATA error count 0 inconsistent with error log pointer 1
ATA Error Count: 0
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 0 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was in an unknown state.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
00 ec 00 00 00 00 00
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
c8 00 00 00 00 00 00 00 00:00:00.000 READ DMA
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 121 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute dela```
box2 (good box)
```root@ubuntu:/usr/src/gsmartcontrol-1.1.3# smartctl -A /dev/sda
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.4.0-26-generic] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 100 100 000 Pre-fail Always - 0
5 Reallocate_NAND_Blk_Cnt 0x0032 100 100 010 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 295
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 20
171 Program_Fail_Count 0x0032 100 100 000 Old_age Always - 0
172 Erase_Fail_Count 0x0032 100 100 000 Old_age Always - 0
173 Ave_Block-Erase_Count 0x0032 099 099 000 Old_age Always - 28
174 Unexpect_Power_Loss_Ct 0x0032 100 100 000 Old_age Always - 13
180 Unused_Reserve_NAND_Blk 0x0033 000 000 000 Pre-fail Always - 36
183 SATA_Interfac_Downshift 0x0032 100 100 000 Old_age Always - 0
184 Error_Correction_Count 0x0032 100 100 000 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
194 Temperature_Celsius 0x0022 059 049 000 Old_age Always - 41 (Min/Max 0/51)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0
197 Bogus_Current_Pend_Sect 0x0032 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 100 100 000 Old_age Always - 4
202 Percent_Lifetime_Remain 0x0030 099 099 001 Old_age Offline - 1
206 Write_Error_Rate 0x000e 100 100 000 Old_age Always - 0
210 Success_RAIN_Recov_Cnt 0x0032 100 100 000 Old_age Always - 0
246 Total_LBAs_Written 0x0032 100 100 000 Old_age Always - 8068464723
247 Host_Program_Page_Count 0x0032 100 100 000 Old_age Always - 136319482
248 FTL_Program_Page_Count 0x0032 100 100 000 Old_age Always - 81387530
root@ubuntu:/usr/src/gsmartcontrol-1.1.3# smartctl --all /dev/sda
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.4.0-26-generic] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Crucial/Micron MX500 SSDs
Device Model: CT500MX500SSD1
Serial Number: 2002E2330312
LU WWN Device Id: 5 00a075 1e2330312
Firmware Version: M3CR023
User Capacity: 500,107,862,016 bytes [500 GB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-3 T13/2161-D revision 5
SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Jul 20 19:57:11 2020 UTC
==> WARNING: This firmware returns bogus raw values in attribute 197
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x03) Offline data collection activity
is in progress.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 249) Self-test routine in progress...
90% of test remaining.
Total time to complete Offline
data collection: ( 1303) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 30) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x0031) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 100 100 000 Pre-fail Always - 0
5 Reallocate_NAND_Blk_Cnt 0x0032 100 100 010 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 295
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 20
171 Program_Fail_Count 0x0032 100 100 000 Old_age Always - 0
172 Erase_Fail_Count 0x0032 100 100 000 Old_age Always - 0
173 Ave_Block-Erase_Count 0x0032 099 099 000 Old_age Always - 28
174 Unexpect_Power_Loss_Ct 0x0032 100 100 000 Old_age Always - 13
180 Unused_Reserve_NAND_Blk 0x0033 000 000 000 Pre-fail Always - 36
183 SATA_Interfac_Downshift 0x0032 100 100 000 Old_age Always - 0
184 Error_Correction_Count 0x0032 100 100 000 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
194 Temperature_Celsius 0x0022 059 049 000 Old_age Always - 41 (Min/Max 0/51)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0
197 Bogus_Current_Pend_Sect 0x0032 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 100 100 000 Old_age Always - 4
202 Percent_Lifetime_Remain 0x0030 099 099 001 Old_age Offline - 1
206 Write_Error_Rate 0x000e 100 100 000 Old_age Always - 0
210 Success_RAIN_Recov_Cnt 0x0032 100 100 000 Old_age Always - 0
246 Total_LBAs_Written 0x0032 100 100 000 Old_age Always - 8068464723
247 Host_Program_Page_Count 0x0032 100 100 000 Old_age Always - 136319482
248 FTL_Program_Page_Count 0x0032 100 100 000 Old_age Always - 81387535
SMART Error Log Version: 1
Warning: ATA error count 0 inconsistent with error log pointer 1
ATA Error Count: 0
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 0 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was in an unknown state.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
00 ec 00 00 00 00 00
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
c8 00 00 00 00 00 00 00 00:00:00.000 READ DMA
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Self_test_in_progress [90% left] (0-65535)
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x10):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.```
答案1
我目前没有看到任何迹象表明“坏”驱动器出现故障，除了UDMA_CRC_Error_Count有点太高。请尝试更换 SSD 电缆。
- 如果仍然没有帮助，请运行以下命令：
sudo cat /dev/sda > /dev/null
echo $?
应该返回 0，并且不应该有sda相关错误或警告dmesg。
- 第二次运行
sudo smartctl -t long /dev/sda
# Some 20 minutes or later (you will be told how much it takes)
sudo smartctl -a /dev/sda
你应该看到类似
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 1234 -
如果不是“无错误完成”，您可能应该尽快更换驱动器。
请注意，SSD 供应商通常不完全支持 SMART 标准，因此smartctl -t long /dev/sda可能根本不起作用。
- 最后，如果你可以暂时停止使用驱动器，请重新启动进入恢复/LiveCD，然后运行
badblocks -b 4096 -s -n /dev/sda