我的问题是，我的 zfs raid 的硬盘在遭遇雷击后部分性能下降，部分损坏。
我能够检测到 zpool status 的问题：
zpool status myzfs
pool myzfs
state: DEGRADED (DESTROYED)
好消息。ZFS 似乎非常可靠，在我的案例中，我能够完全恢复 raid。我如何恢复请参见下面的答案。
恢复 ZFS Raid 我学到了两件事：
- 1 个驱动器故障导致 zpool 瘫痪。但是，基于条纹镜的突袭仍然可用。详细信息解释ZFS：您应该使用镜像 vdev，而不是 RAIDZ
- 恢复和基于 raidz2-0 重新同步 zpool 需要很长时间。条纹镜可能更好。条纹镜的优点和缺点在互联网陸軍
- 突袭不是备份！将备份转移到云或第二个位置是一个很大的优势，而且现在无需花费太多。大多数 Raid 允许备份到云或 ZFS 复制到另一个 NAS
原始调试信息
然而这对于检测和解决问题并不一定重要。
我的 freenas 9.2.1 出了问题。它今天崩溃了。它在 zfs jbod raid 2 上运行文件服务器。我不确定到底是什么导致了问题。系统正在启动，但反应相当慢。从日志中我无法找出任何完全错误的地方。因此，我不确定从哪里开始进行错误分析以及如何解决它们。
问题是系统崩溃了，响应速度很慢。由于 pyhon 死机，freenas 网页界面也崩溃了。
Freenas 安装在 USB 棒上，附加一个驱动器 (2TB) 用于备份。其他 4 个驱动器以 zfs raid 形式运行。
硬盘确实显示智能错误。我该如何修复它们？它们可能是问题的原因。
顶部
CPU: 0.1% user, 0.0% nice, 2.5% system, 0.1% interrupt, 97.3% idle
Mem: 131M Active, 11G Inact, 3689M Wired, 494M Cache, 3232M Buf, 16M Free
ARC: 3028K Total, 347K MFU, 1858K MRU, 16K Anon, 330K Header, 477K Other
Swap: 10G Total, 636K Used, 10G Free
自由度
Filesystem Size Used Avail Capacity Mounted on
/dev/ufs/FreeNASs2a 971M 866M 27M 97% /
devfs 1.0k 1.0k 0B 100% /dev
/dev/md0 4.8M 3.5M 918k 79% /etc
/dev/md1 843k 2.6k 773k 0% /mnt
/dev/md2 156M 40M 103M 28% /var
/dev/ufs/FreeNASs4 20M 3.4M 15M 18% /data
fink-zfs01 6.0T 249k 6.0T 0% /mnt/fink-zfs01
fink-zfs01/.system 6.0T 249k 6.0T 0% /mnt/fink-zfs01/.system
fink-zfs01/.system/cores 6.0T 14M 6.0T 0% /mnt/fink-zfs01/.system/cores
fink-zfs01/.system/samba4 6.0T 862k 6.0T 0% /mnt/fink-zfs01/.system/samba4
fink-zfs01/.system/syslog 6.0T 2.7M 6.0T 0% /mnt/fink-zfs01/.system/syslog
fink-zfs01/shares 6.0T 261k 6.0T 0% /mnt/fink-zfs01/shares
fink-zfs01/shares/fink-privat 6.4T 344G 6.0T 5% /mnt/fink-zfs01/shares/fink-privat
fink-zfs01/shares/gf 6.0T 214k 6.0T 0% /mnt/fink-zfs01/shares/gf
fink-zfs01/shares/kundendaten 6.6T 563G 6.0T 9% /mnt/fink-zfs01/shares/kundendaten
fink-zfs01/shares/zubehoer 6.6T 539G 6.0T 8% /mnt/fink-zfs01/shares/zubehoer
fink-zfs01/temp 6.2T 106G 6.0T 2% /mnt/fink-zfs01/temp
/dev/ufs/Backup 1.9T 114G 1.7T 6% /mnt/Backup
/var/log/消息
Jan 21 21:48:32 s-FreeNAS root: /etc/rc: WARNING: failed to start syslogd
Jan 21 21:48:32 s-FreeNAS kernel: .
Jan 21 21:48:32 s-FreeNAS root: /etc/rc: WARNING: failed to start watchdogd
Jan 21 21:48:32 s-FreeNAS root: /etc/rc: WARNING: failed precmd routine for vmware_guestd
Jan 21 21:48:34 s-FreeNAS ntpd[2589]: ntpd 4.2.4p5-a (1)
Jan 21 21:48:34 s-FreeNAS kernel: .
Jan 21 21:48:36 s-FreeNAS generate_smb4_conf.py: [common.pipesubr:58] Popen()ing: zfs list -H -o mountpoint,name
Jan 21 21:48:36 s-FreeNAS generate_smb4_conf.py: [common.pipesubr:58] Popen()ing: zfs list -H -o mountpoint
Jan 21 21:48:38 s-FreeNAS last message repeated 4 times
Jan 21 21:48:38 s-FreeNAS generate_smb4_conf.py: [common.pipesubr:58] Popen()ing: /usr/local/bin/pdbedit -d 0 -i smbpasswd:/tmp/tmpEKKZ2A -e tdbsam:/var/etc/private/passdb.tdb -s /usr/local/etc/smb4.conf
Jan 21 21:48:43 s-FreeNAS ntpd[2590]: time reset -0.194758 s
Jan 21 21:48:45 s-FreeNAS smartd[2867]: Device: /dev/ada3, FAILED SMART self-check. BACK UP DATA NOW!
Jan 21 21:48:45 s-FreeNAS smartd[2867]: Device: /dev/ada3, 164 Currently unreadable (pending) sectors
Jan 21 21:48:45 s-FreeNAS smartd[2867]: Device: /dev/ada3, Failed SMART usage Attribute: 5 Reallocated_Sector_Ct.
Jan 21 21:48:45 s-FreeNAS smartd[2867]: Device: /dev/ada3, previous self-test completed with error (unknown test element)
Jan 21 21:48:51 s-FreeNAS mDNSResponder: mDNSResponder (Engineering Build) (Mar 1 2014 18:12:24) starting
Jan 21 21:48:51 s-FreeNAS mDNSResponder: 8: Listening for incoming Unix Domain Socket client requests
Jan 21 21:48:51 s-FreeNAS mDNSResponder: mDNS_AddDNSServer: Lock not held! mDNS_busy (0) mDNS_reentrancy (0)
Jan 21 21:48:51 s-FreeNAS mDNSResponder: mDNS_AddDNSServer: Lock not held! mDNS_busy (0) mDNS_reentrancy (0)
Jan 21 21:48:53 s-FreeNAS netatalk[3142]: Netatalk AFP server starting
Jan 21 21:48:53 s-FreeNAS cnid_metad[3179]: CNID Server listening on localhost:4700
Jan 21 21:48:53 s-FreeNAS kernel: done.
Jan 21 21:48:54 s-FreeNAS mDNSResponder: mDNS_Register_internal: ERROR!! Tried to register AuthRecord 0000000800C2FD60 s-FreeNAS.local. (Addr) that's already in the list
...
Jan 21 21:48:54 s-FreeNAS mDNSResponder: mDNS_Register_internal: ERROR!! Tried to register AuthRecord 0000000800C30180 109.1.1.10.in-addr.arpa. (PTR) that's already in the list
Jan 21 22:04:44 s-FreeNAS kernel: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 1572950, size: 8192
...
Jan 21 22:05:25 s-FreeNAS kernel: GEOM_ELI: g_eli_read_done() failed ada0p1.eli[READ(offset=110592, length=4096)]
Jan 21 22:05:25 s-FreeNAS kernel: swap_pager: I/O error - pagein failed; blkno 1572894,size 4096, error 5
Jan 21 22:05:25 s-FreeNAS kernel: vm_fault: pager read error, pid 3020 (python2.7)
Jan 21 22:05:25 s-FreeNAS kernel: Failed to write core file for process python2.7 (error 14)
...
Jan 21 22:19:44 s-FreeNAS kernel: (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 70 02 00 40 00 00 00 00 00 00
Jan 21 22:19:44 s-FreeNAS kernel: (ada0:ahcich0:0:0:0): CAM status: ATA Status Error
Jan 21 22:19:44 s-FreeNAS kernel: (ada0:ahcich0:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC )
Jan 21 22:19:44 s-FreeNAS kernel: (ada0:ahcich0:0:0:0): RES: 41 40 70 02 00 40 00 00 00 00 00
Jan 21 22:19:44 s-FreeNAS kernel: (ada0:ahcich0:0:0:0): Error 5, Retries exhausted
Jan 21 22:19:44 s-FreeNAS kernel: GEOM_ELI: g_eli_read_done() failed ada0p1.eli[READ(offset=253952, length=4096)]
Jan 21 22:19:44 s-FreeNAS kernel: swap_pager: I/O error - pagein failed; blkno 1572929,size 4096, error 5
Jan 21 22:19:44 s-FreeNAS kernel: vm_fault: pager read error, pid 2869 (smartd)
Jan 21 22:19:44 s-FreeNAS kernel: Failed to write core file for process smartd (error 14)
Jan 21 22:19:44 s-FreeNAS kernel: pid 2869 (smartd), uid 0: exited on signal 11
smartctl——扫描
/dev/ada0 -d atacam # /dev/ada0, ATA device
/dev/ada1 -d atacam # /dev/ada1, ATA device
/dev/ada2 -d atacam # /dev/ada2, ATA device
/dev/pass3 -d atacam # /dev/pass3, ATA device
/dev/ada3 -d atacam # /dev/ada3, ATA device
/dev/ada4 -d atacam # /dev/ada4, ATA device
/dev/ada5 -d atacam # /dev/ada5, ATA device
smartctl -a /dev/ada3
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Device Model: WDC WD4000F9YZ-09N20L0
Serial Number: WD-WMC1F1211607
LU WWN Device Id: 5 0014ee 0ae5c0b4c
Firmware Version: 01.01A01
User Capacity: 4,000,787,030,016 bytes [4.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 7200 rpm
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ATA8-ACS (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Wed Jan 21 23:07:55 2015 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: FAILED!
See vendor-specific Attribute list for failed Attributes.
General SMART Values:
Offline data collection status: (0x85) Offline data collection activity
was aborted by an interrupting command from host.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 73) The previous self-test completed having
a test element that failed and the test
element that failed is not known.
Total time to complete Offline
data collection: (41640) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 451) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x70bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 187 187 051 Pre-fail Always - 553
3 Spin_Up_Time 0x0027 142 138 021 Pre-fail Always - 11900
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 93
5 Reallocated_Sector_Ct 0x0033 139 139 140 Pre-fail Always FAILING_NOW 1791
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 090 090 000 Old_age Always - 7553
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 93
183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 59
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 35
194 Temperature_Celsius 0x0022 108 098 000 Old_age Always - 44
196 Reallocated_Event_Count 0x0032 001 001 000 Old_age Always - 353
197 Current_Pending_Sector 0x0032 200 199 000 Old_age Always - 162
198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed: unknown failure 90% 7553 -
# 2 Short offline Completed: unknown failure 90% 7552 -
# 3 Short offline Completed: unknown failure 90% 7551 -
# 4 Short offline Completed: unknown failure 90% 7550 -
# 5 Short offline Completed: unknown failure 90% 7549 -
# 6 Short offline Completed: unknown failure 90% 7548 -
# 7 Short offline Completed: unknown failure 90% 7547 -
# 8 Short offline Completed: unknown failure 90% 7546 -
# 9 Short offline Completed: unknown failure 90% 7545 -
#10 Short offline Completed: unknown failure 90% 7544 -
#11 Short offline Completed: unknown failure 90% 7543 -
#12 Short offline Completed: unknown failure 90% 7542 -
#13 Short offline Completed without error 00% 7541 -
#14 Short offline Completed without error 00% 7540 -
#15 Short offline Completed: read failure 10% 7538 1148054536
#16 Short offline Completed: read failure 10% 7538 1148054536
#17 Short offline Completed: read failure 10% 7536 1148057328
#18 Short offline Completed: read failure 10% 7535 1148057328
#19 Short offline Completed without error 00% 7530 -
#20 Short offline Completed without error 00% 7529 -
#21 Short offline Completed: read failure 10% 7528 1148057328
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
答案1
FreeBSD 由于此错误而崩溃：
21 22:19:44 s-FreeNAS 内核：（ada0：ahcich0：0：0：0）：READ_FPDMA_QUEUED。 ACB：60 08 70 02 00 40 00 00 00 00 00 00 1 月 21 日 22:19:44 s-FreeNAS 内核：（ada0：ahcich0：0：0：0）：CAM 状态：ATA 状态错误 1 月 21 日 22:19:44 s-FreeNAS 内核：（ada0：ahcich0：0：0：0）：ATA 状态：41（DRDY ERR），错误：40（UNC） 1 月 21 日 22:19:44 s-FreeNAS 内核：（ada0：ahcich0：0：0：0）：RES：41 40 70 02 00 40 00 00 00 00 00 1 月 21 日 22:19:44 s-FreeNAS 内核：（ada0：ahcich0：0：0：0）：错误 5，重试已用尽 1 月 21 日 22:19:44 s-FreeNAS 内核：GEOM_ELI：g_eli_read_done() 失败 ada0p1.eli[READ(offset=253952, length=4096)] 1 月 21 日 22:19:44 s-FreeNAS 内核：swap_pager：I/O 错误 - 分页失败；blkno 1572929，大小 4096，错误 5
这意味着 SATA 电缆有问题，或者（假设您还有另一个坏掉的磁盘 ada3）可能表明电源存在问题。这是一个系统磁盘或交换空间，而且由于它只是普通的 UFS，没有冗余，因此系统无法应对这种情况。
至于 ZFS - 请发布zpool status输出。
答案2
免责声明：
修复已损坏的 ZFS 池并替换已降级或不可用的进程是危险的。对我来说，这是可行的。但是请仔细阅读所有文档并了解典型故障。否则，您可能会完全破坏您的 raid。您可能希望联系专业人士来帮助您恢复数据。
然而，仔细阅读后您应该能够自己恢复 zfs raid！
情况
进一步的研究表明，雷击产生的电流峰值会损坏硬盘并损坏其他硬盘。因此，zfs 池存在缺陷。
可以使用 zpool status 检测：
zpool status myzfs
pool myzfs
state: DEGRADED (DESTROYED)
恢复被毁坏的 ZFS 存储池
这个问题可能（至少对我来说）可以通过以下方式解决：
zpool destroy myzfs
zpool import -Df # this made the zpool accessible again
然而，由于 1 个驱动器被完全损坏，zpool 性能继续下降。
有关恢复已损坏的 zfs 池的完整文档，请参阅 Oracle 文档恢复已损坏的 zfs 存储池
更换降级的 ZFS 驱动器
恢复降级的 zpool 并不能完全解决问题，因为它仍然有降级/有缺陷的驱动器
zpool-status myzfs 配置：
NAME STATE READ WRITE CKSUM
myzfs DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
gptid/uuid1 ONLINE 0 0 0
gptid/uuid2 ONLINE 0 0 0
gptid/uuid3 ONLINE 0 0 0
778923478919345 UNAVAIL 0 0 0 /was /dev/ada4
互联网上也有关于驱动器更换的详细记录。不过，这取决于您是否使用热备用、RAID 级别等...
基本上它对我来说相当简单
zpool replace myzfs 778923478919345
Oracle 的替换文档也对此进行了很好的说明替换 ZFS 存储池中的设备