背景:
我在 Box 1 上有一个 6x 4TB MDADM 阵列。Box 1 出现 MB/CPU 问题,导致运行约 15-25 分钟后断电,但在正常运行期间,阵列可以正常访问。我需要从该阵列中获取数据。
我热接线了 PSU,为没有 MB/CPU 的 HDD 供电(回形针,针脚 4+5),并将 HDD 连接到 Box 2 中的控制器,该控制器使用 Live Ubuntu 启动。这样我就可以使用 Box 2 查看所有驱动器,它们都能正常显示lsblk
。
问题:
阵列未重组。它说所有设备都缺少 MBR,尽管gdisk
它们说它们有 MBR。我之前重组过这个阵列,没有任何问题……为什么现在?
root@ubuntu:~# mdadm --assemble /dev/md0 /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh
mdadm: Cannot assemble mbr metadata on /dev/sdc
mdadm: /dev/sdc has no superblock - assembly aborted
root@ubuntu:~# mdadm --assemble --scan
mdadm: No arrays found in config file or automatically
root@ubuntu:~# mdadm --examine /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh
/dev/sdc:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
/dev/sdd:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
/dev/sde:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
/dev/sdf:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
/dev/sdg:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
/dev/sdh:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
每个 HDD 上的 Gdisk
root@ubuntu:~# gdisk /dev/sdc
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help): q
root@ubuntu:~# gdisk /dev/sdd
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help): q
root@ubuntu:~# gdisk /dev/sde
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help): q
root@ubuntu:~# gdisk /dev/sdf
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help): q
root@ubuntu:~# gdisk /dev/sdg
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help): q
root@ubuntu:~# gdisk /dev/sdh
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help): q
所有驱动器的 SMART 指标基本相同/dev/sdc
(自第 1 天起它们就一直位于阵列中):
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 184 175 021 Pre-fail Always - 7766
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 489
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0
9 Power_On_Hours 0x0032 008 008 000 Old_age Always - 67555
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 489
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 439
193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 2155317
194 Temperature_Celsius 0x0022 124 098 000 Old_age Always - 28
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
SMART Error Log Version: 1
No Errors Logged
我这里漏掉了什么?为什么我不能重新组装这个阵列?!我只需要把它组装起来,让机器运行大约 2 天来复制内容。服务器升级时,我曾两次移动过这个阵列,没有问题……
为所有驱动器添加 Gdisk -l:
root@ubuntu:/nfs/spruce# gdisk -l /dev/sdc
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sdc: 7814037168 sectors, 3.6 TiB
Model: WDC WD40EZRX-00S
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): 4BE9E7EF-6811-4225-B6A1-36DA90DE9CA4
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 7814037101 sectors (3.6 TiB)
Number Start (sector) End (sector) Size Code Name
root@ubuntu:/nfs/spruce# gdisk -l /dev/sdd
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sdd: 7814037168 sectors, 3.6 TiB
Model: WDC WD40EZRX-00S
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): 504EAA64-CB71-409A-8409-17C2CAE09FEB
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 7814037101 sectors (3.6 TiB)
Number Start (sector) End (sector) Size Code Name
root@ubuntu:/nfs/spruce# gdisk -l /dev/sde
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sde: 7814037168 sectors, 3.6 TiB
Model: WDC WD40EZRX-00S
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): F516176B-1DA8-4870-B765-F1D5D1C82C28
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 7814037101 sectors (3.6 TiB)
Number Start (sector) End (sector) Size Code Name
root@ubuntu:/nfs/spruce# gdisk -l /dev/sdf
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sdf: 7814037168 sectors, 3.6 TiB
Model: WDC WD40EZRX-00S
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): EE127477-19DD-40B6-9012-D75A78914456
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 7814037101 sectors (3.6 TiB)
Number Start (sector) End (sector) Size Code Name
root@ubuntu:/nfs/spruce# gdisk -l /dev/sdg
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sdg: 7814037168 sectors, 3.6 TiB
Model: WDC WD40EZRX-00S
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): D10CA92C-66B9-47D5-989C-D8A46740A2D0
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 7814037101 sectors (3.6 TiB)
Number Start (sector) End (sector) Size Code Name
root@ubuntu:/nfs/spruce# gdisk -l /dev/sdh
GPT fdisk (gdisk) version 1.0.8
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sdh: 7814037168 sectors, 3.6 TiB
Model: WDC WD40EZRX-00S
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): 4439E365-A85A-428A-8A18-1056662A8195
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 7814037101 sectors (3.6 TiB)
Number Start (sector) End (sector) Size Code Name
答案1
您的每个磁盘都有一个单独的分区(可能)覆盖整个 LBA 跨度。尝试使用以下命令组装阵列:
mdadm --assemble /dev/md0 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1
如果不起作用,则显示--examine
(而不是--assemble
)的输出,并使用与上面相同的分区。
编辑:嗯,这里似乎有些奇怪。gdisk -l
没有显示任何分区,但mdadm --examine /dev/sdc
没有识别任何 RAID 磁盘。如果您确定这些磁盘真的用于 RAID 阵列,您可能有一个无元数据的阵列 - 即:需要通过 手动联机的阵列mdadm --build
。但是,要使此命令起作用,您需要完全了解阵列几何形状(级别、块大小等),而这里缺少这一点。
另一种可能性是旧主板使用了一些自定义 RAID 元数据(您是否使用了 BIOS 伪 RAID 功能?)。这应该但是,要向磁盘写入一些内容。
我强烈建议将磁盘重新连接到旧主板,并收集有关 RAID 阵列的尽可能多的信息(即:)cat /proc/mdstat; mdadm --examine; etc
。