我的 zfs 池处于 UNAVAIL 状态，无法导入。我想知道是否有可能恢复它或者我可以从中取出任何数据？
背景
我接管了一台服务器的管理，该服务器在 VMware ESXi 上有一个 Ubuntu 18.04 VM，其中有 zfs 池，还有另外两台 VM。
由于 zfs 池已满，我确实zfs add添加了额外的 12 TB 磁盘 (sdi)。（考虑到其他磁盘为 4 TB，这种不对称可能并不理想，但该磁盘似乎已被识别并且 VM 运行正常。）我认为我将其磁盘模式设置为“独立持久”是一个问题，这似乎意味着该磁盘不会通过快照恢复。
然后，由于虚拟机 ( sda, lvm, nextcloud-vg) 的启动驱动器已满，我尝试将 lvm 扩展到sdj。lvextend（我太天真和愚蠢了，在那之前没有拍快照。）然后，虚拟机无法启动，Grub 找不到启动盘。我注意到这是因为sdj是 SCSI(0:10) 并且在启动时未被 BIOS 识别。
当我将之前的 SCSI ID 更改sdj为 (0:2) 时，Grub 找到了启动盘，但虚拟机仍然无法启动，因为 Ubuntu 文件不知何故大量丢失。然后，我sda/nextcloud-vg通过从旧快照复制来恢复文件。现在，虚拟机可以启动，但无法读取和导入 zfs 池。
我非常感谢任何评论和建议。非常感谢你的阅读。
截至 2019 年 11 月，zfs 池运行状况良好，可通过 VMware 快照访问。
root@nextcloud:~# zpool status
pool: ncdata
state: ONLINE
scan: scrub repaired 0B in 51h42m with 0 errors on Tue Nov 12 04:06:08 2019
config:
NAME STATE READ WRITE CKSUM
ncdata ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
sdg ONLINE 0 0 0
sdh ONLINE 0 0 0
errors: No known data errors
root@nextcloud:~# zdb
ncdata:
version: 5000
name: 'ncdata'
state: 0
txg: 6813984
pool_guid: 4115695041733501295
errata: 0
hostname: 'nextcloud'
com.delphix:has_per_vdev_zaps
vdev_children: 7
vdev_tree:
type: 'root'
id: 0
guid: 4115695041733501295
create_txg: 4
children[0]:
type: 'disk'
id: 0
guid: 12100899330530807388
path: '/dev/sdb1'
whole_disk: 1
metaslab_array: 131
metaslab_shift: 28
ashift: 12
asize: 42934468608
is_log: 0
create_txg: 4
com.delphix:vdev_zap_leaf: 129
com.delphix:vdev_zap_top: 130
children[1]:
type: 'disk'
id: 1
guid: 12620057802260459555
path: '/dev/sdc1'
whole_disk: 1
metaslab_array: 256
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1469
com.delphix:vdev_zap_leaf: 135
com.delphix:vdev_zap_top: 136
children[2]:
type: 'disk'
id: 2
guid: 3074381589662896744
path: '/dev/sdd1'
whole_disk: 1
metaslab_array: 384
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1472
com.delphix:vdev_zap_leaf: 137
com.delphix:vdev_zap_top: 138
children[3]:
type: 'disk'
id: 3
guid: 1759578235837374106
path: '/dev/sde1'
whole_disk: 1
metaslab_array: 140
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1475
com.delphix:vdev_zap_leaf: 512
com.delphix:vdev_zap_top: 513
children[4]:
type: 'disk'
id: 4
guid: 10518026248187884540
path: '/dev/sdf1'
whole_disk: 1
metaslab_array: 259
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1478
com.delphix:vdev_zap_leaf: 257
com.delphix:vdev_zap_top: 258
children[5]:
type: 'disk'
id: 5
guid: 8733140335356986716
path: '/dev/sdg1'
whole_disk: 1
metaslab_array: 640
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1481
com.delphix:vdev_zap_leaf: 386
com.delphix:vdev_zap_top: 387
children[6]:
type: 'disk'
id: 6
guid: 4551718362642717333
path: '/dev/sdh1'
whole_disk: 1
metaslab_array: 369
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1603266
com.delphix:vdev_zap_leaf: 349
com.delphix:vdev_zap_top: 368
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
现在的情况。
root@nextcloud:~# zpool import
pool: ncdata
id: 4115695041733501295
state: UNAVAIL
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
see: http://zfsonlinux.org/msg/ZFS-8000-5E
config:
ncdata UNAVAIL insufficient replicas
sdb ONLINE
sde ONLINE
sdf ONLINE
sdg ONLINE
sdh ONLINE
sdi ONLINE
sdj ONLINE
sdi UNAVAIL corrupted data
root@nextcloud:~# zdb -C
ncdata:
version: 5000
name: 'ncdata'
state: 0
txg: 6813824
pool_guid: 4115695041733501295
errata: 0
hostname: 'nextcloud'
com.delphix:has_per_vdev_zaps
vdev_children: 7
vdev_tree:
type: 'root'
id: 0
guid: 4115695041733501295
create_txg: 4
children[0]:
type: 'disk'
id: 0
guid: 12100899330530807388
path: '/dev/sdb1'
whole_disk: 1
metaslab_array: 131
metaslab_shift: 28
ashift: 12
asize: 42934468608
is_log: 0
create_txg: 4
com.delphix:vdev_zap_leaf: 129
com.delphix:vdev_zap_top: 130
children[1]:
type: 'disk'
id: 1
guid: 12620057802260459555
path: '/dev/sdc1'
whole_disk: 1
metaslab_array: 256
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1469
com.delphix:vdev_zap_leaf: 135
com.delphix:vdev_zap_top: 136
children[2]:
type: 'disk'
id: 2
guid: 3074381589662896744
path: '/dev/sdd1'
whole_disk: 1
metaslab_array: 384
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1472
com.delphix:vdev_zap_leaf: 137
com.delphix:vdev_zap_top: 138
children[3]:
type: 'disk'
id: 3
guid: 1759578235837374106
path: '/dev/sde1'
whole_disk: 1
metaslab_array: 140
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1475
com.delphix:vdev_zap_leaf: 512
com.delphix:vdev_zap_top: 513
children[4]:
type: 'disk'
id: 4
guid: 10518026248187884540
path: '/dev/sdf1'
whole_disk: 1
metaslab_array: 259
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1478
com.delphix:vdev_zap_leaf: 257
com.delphix:vdev_zap_top: 258
children[5]:
type: 'disk'
id: 5
guid: 8733140335356986716
path: '/dev/sdg1'
whole_disk: 1
metaslab_array: 640
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1481
com.delphix:vdev_zap_leaf: 386
com.delphix:vdev_zap_top: 387
children[6]:
type: 'disk'
id: 6
guid: 4551718362642717333
path: '/dev/sdh1'
whole_disk: 1
metaslab_array: 369
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
create_txg: 1603266
com.delphix:vdev_zap_leaf: 349
com.delphix:vdev_zap_top: 368
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
root@nextcloud:/dev/disk# zdb -d ncdata
zdb: can't open 'ncdata': Value too large for defined data type
root@nextcloud:/dev/disk# zdb -e -p /dev -d ncdata
zdb: can't open 'ncdata': File exists
root@nextcloud:/mytmpdev# zdb -e -p /dev -AAA -d ncdata
zdb: can't open 'ncdata': File exists
root@nextcloud:/dev/disk# zdb -e ncdata
Configuration for import:
version: 5000
pool_guid: 4115695041733501295
name: 'ncdata'
state: 0
vdev_children: 8
vdev_tree:
type: 'root'
id: 0
guid: 4115695041733501295
children[0]:
type: 'disk'
id: 0
guid: 12100899330530807388
whole_disk: 1
metaslab_array: 131
metaslab_shift: 28
ashift: 12
asize: 42934468608
is_log: 0
DTL: 1102
create_txg: 4
path: '/dev/sdb1'
children[1]:
type: 'disk'
id: 1
guid: 12620057802260459555
whole_disk: 1
metaslab_array: 256
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
DTL: 1109
create_txg: 1469
path: '/dev/sde1'
children[2]:
type: 'disk'
id: 2
guid: 3074381589662896744
whole_disk: 1
metaslab_array: 384
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
DTL: 1108
create_txg: 1472
path: '/dev/sdf1'
children[3]:
type: 'disk'
id: 3
guid: 1759578235837374106
whole_disk: 1
metaslab_array: 140
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
DTL: 1107
create_txg: 1475
path: '/dev/sdg1'
children[4]:
type: 'disk'
id: 4
guid: 10518026248187884540
whole_disk: 1
metaslab_array: 259
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
DTL: 1103
create_txg: 1478
path: '/dev/sdh1'
children[5]:
type: 'disk'
id: 5
guid: 8733140335356986716
whole_disk: 1
metaslab_array: 640
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
DTL: 1106
create_txg: 1481
path: '/dev/sdi1'
children[6]:
type: 'disk'
id: 6
guid: 4551718362642717333
whole_disk: 1
metaslab_array: 369
metaslab_shift: 35
ashift: 12
asize: 4398031306752
is_log: 0
DTL: 1105
create_txg: 1603266
path: '/dev/sdj1'
children[7]:
type: 'disk'
id: 7
guid: 5107684954562511391
whole_disk: 1
metaslab_array: 1058
metaslab_shift: 36
ashift: 12
asize: 13194124328960
is_log: 0
create_txg: 6813878
path: '/dev/sdk1'
rewind-policy:
rewind-request-txg: 18446744073709551615
rewind-request: 2
zdb: can't open 'ncdata': File exists
ZFS_DBGMSG(zdb):
root@nextcloud:/mytmpdev# zdb -l /dev/sdb1
------------------------------------
LABEL 0
------------------------------------
version: 5000
name: 'ncdata'
state: 0
txg: 9549126
pool_guid: 4115695041733501295
errata: 0
hostname: 'nextcloud'
top_guid: 12100899330530807388
guid: 12100899330530807388
vdev_children: 8
vdev_tree:
type: 'disk'
id: 0
guid: 12100899330530807388
path: '/dev/sdb1'
whole_disk: 1
metaslab_array: 131
metaslab_shift: 28
ashift: 12
asize: 42934468608
is_log: 0
DTL: 1102
create_txg: 4
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
labels = 0 1 2 3
root@nextcloud:/mytmpdev# zdb -l /dev/sdk1
------------------------------------
LABEL 0
------------------------------------
version: 5000
name: 'ncdata'
state: 0
txg: 6813880
pool_guid: 4115695041733501295
errata: 0
hostname: 'nextcloud'
top_guid: 5107684954562511391
guid: 5107684954562511391
vdev_children: 8
vdev_tree:
type: 'disk'
id: 7
guid: 5107684954562511391
path: '/dev/sdi1'
whole_disk: 1
metaslab_array: 1058
metaslab_shift: 36
ashift: 12
asize: 13194124328960
is_log: 0
create_txg: 6813878
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
labels = 0 1 2 3
root@nextcloud:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 40G 0 disk
└─sda1 8:1 0 40G 0 part
├─nextcloud--vg-root 253:0 0 1T 0 lvm /
└─nextcloud--vg-swap_1 253:1 0 976M 0 lvm [SWAP]
sdb 8:16 0 40G 0 disk
├─sdb1 8:17 0 40G 0 part
└─sdb9 8:25 0 8M 0 part
sdc 8:32 0 4T 0 disk
└─nextcloud--vg-root 253:0 0 1T 0 lvm /
sdd 8:48 0 4T 0 disk
├─sdd1 8:49 0 1M 0 part
├─sdd2 8:50 0 976.8G 0 part
└─sdd3 8:51 0 976.8G 0 part
sde 8:64 0 4T 0 disk
├─sde1 8:65 0 4T 0 part
└─sde9 8:73 0 8M 0 part
sdf 8:80 0 4T 0 disk
├─sdf1 8:81 0 4T 0 part
└─sdf9 8:89 0 8M 0 part
sdg 8:96 0 4T 0 disk
├─sdg1 8:97 0 4T 0 part
└─sdg9 8:105 0 8M 0 part
sdh 8:112 0 4T 0 disk
├─sdh1 8:113 0 4T 0 part
└─sdh9 8:121 0 8M 0 part
sdi 8:128 0 4T 0 disk
├─sdi1 8:129 0 4T 0 part
└─sdi9 8:137 0 8M 0 part
sdj 8:144 0 4T 0 disk
├─sdj1 8:145 0 4T 0 part
└─sdj9 8:153 0 8M 0 part
sdk 8:160 0 12T 0 disk
├─sdk1 8:161 0 12T 0 part
└─sdk9 8:169 0 8M 0 part
sr0 11:0 1 1024M 0 rom
root@nextcloud:/dev/disk# parted -l
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 42.9GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 1049kB 42.9GB 42.9GB primary boot, lvm
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 42.9GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 42.9GB 42.9GB zfs zfs-7c215dc258f390a2
9 42.9GB 42.9GB 8389kB
Error: /dev/sdc: unrecognised disk label
Model: VMware Virtual disk (scsi)
Disk /dev/sdc: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
Model: VMware Virtual disk (scsi)
Disk /dev/sdd: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 2097kB 1049kB bios_grub
2 2097kB 1049GB 1049GB ext4
3 1049GB 2098GB 1049GB ext4
Model: VMware Virtual disk (scsi)
Disk /dev/sde: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 4398GB 4398GB zfs zfs-bb77ffc8c56c704c
9 4398GB 4398GB 8389kB
Model: VMware Virtual disk (scsi)
Disk /dev/sdf: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 4398GB 4398GB zfs zfs-e5f9cec454077158
9 4398GB 4398GB 8389kB
Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/nextcloud--vg-swap_1: 1023MB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:
Number Start End Size File system Flags
1 0.00B 1023MB 1023MB linux-swap(v1)
Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/nextcloud--vg-root: 1100GB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:
Number Start End Size File system Flags
1 0.00B 1100GB 1100GB ext4
Model: VMware Virtual disk (scsi)
Disk /dev/sdk: 13.2TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 13.2TB 13.2TB zfs zfs-e5495635496c5b84
9 13.2TB 13.2TB 8389kB
Model: VMware Virtual disk (scsi)
Disk /dev/sdi: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 4398GB 4398GB zfs zfs-b18c76425dec00d4
9 4398GB 4398GB 8389kB
Model: VMware Virtual disk (scsi)
Disk /dev/sdg: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 4398GB 4398GB zfs zfs-7662c1d580329ece
9 4398GB 4398GB 8389kB
Model: VMware Virtual disk (scsi)
Disk /dev/sdj: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 4398GB 4398GB zfs zfs-45572a27b3ca3dd5
9 4398GB 4398GB 8389kB
Model: VMware Virtual disk (scsi)
Disk /dev/sdh: 4398GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 4398GB 4398GB zfs zfs-a3d1bd676c04ef35
9 4398GB 4398GB 8389kB
答案1
如果您查看失败的 zfs 导入的链接：https://zfsonlinux.org/msg/ZFS-8000-5E/， 在底部
如果在“zpool import”期间遇到此错误，则操作相同[如下所示]无法导入池 - 所有数据丢失，必须从适当的备份源恢复。
上面说了一点
ZFS 将无法使用该池，并且池中的所有数据都将不可挽回地丢失。必须销毁该池并从适当的备份源重新创建。
如果您有数据快照，那么您可以重新创建 ZFS 池，然后手动将文件从快照复制到新池。