今天我遇到了一个对我来说真正的谜团,如果有人能告诉我这件事是怎么发生的我将非常高兴。
我们当天早些时候从软件 RAID1(如果重要的话,是 Ubuntu 14.04)中物理移除了一个磁盘,然后重新添加它。(/dev/sda2 是 RAID 中的活动分区,我们想要重新添加 /dev/sdb2 并让其重新同步,/dev/md1 是有问题的 RAID 阵列)
我们发出命令:mdadm /dev/md1 --add /dev/sdb2
我们几乎立即注意到驱动器似乎没有同步,因为检查/proc/mdstat
表明驱动器已经合并到阵列中。
一分钟之内,应用程序崩溃,ext4 失败并以只读方式重新挂载。看来驱动器被重新添加到阵列中,但未与现有驱动器同步。系统日志的输出似乎证实了这一点:
[ 1607.017583] md: bind<sdb2>
[ 1607.019831] RAID1 conf printout:
[ 1607.019839] --- wd:1 rd:3
[ 1607.019842] disk 0, wo:1, o:1, dev:sdb2
[ 1607.019845] disk 1, wo:0, o:1, dev:sda2
[ 1607.019911] md: recovery of RAID array md1
[ 1607.019915] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[ 1607.019917] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
[ 1607.019923] md: using 128k window, over a total of 341795704k.
[ 1607.020187] md: md1: recovery done.
[ 1607.022356] RAID1 conf printout:
[ 1607.022359] --- wd:2 rd:3
[ 1607.022361] disk 0, wo:0, o:1, dev:sdb2
[ 1607.022364] disk 1, wo:0, o:1, dev:sda2
...
[ 1651.932310] init: mysql main process (1192) terminated with status 1
[ 1651.932329] init: mysql main process ended, respawning
[ 1654.691865] init: mysql main process (9271) terminated with status 1
[ 1654.691883] init: mysql main process ended, respawning
[ 1655.004970] init: mysql post-start process (9272) terminated with status 1
[ 1656.937186] EXT4-fs error (device md1): ext4_mb_generate_buddy:756: group 494, block bitmap and bg descriptor inconsistent: 1284 vs 8190 free clusters
[ 1656.937268] Aborting journal on device md1-8.
[ 1656.937718] EXT4-fs (md1): Remounting filesystem read-only
[ 1656.937921] EXT4-fs error (device md1): ext4_mb_generate_buddy:756: group 495, block bitmap and bg descriptor inconsistent: 479 vs 10240 free clusters
[ 1656.938181] EXT4-fs error (device md1): ext4_mb_generate_buddy:756: group 496, block bitmap and bg descriptor inconsistent: 23892 vs 23866 free clusters
[ 1656.938411] EXT4-fs error (device md1): ext4_mb_generate_buddy:756: group 497, block bitmap and bg descriptor inconsistent: 11610 vs 16699 free clusters
[ 1656.938639] EXT4-fs error (device md1): ext4_mb_generate_buddy:756: group 498, block bitmap and bg descriptor inconsistent: 2560 vs 10642 free clusters
[ 1656.938782] EXT4-fs error (device md1): ext4_mb_generate_buddy:756: group 499, block bitmap and bg descriptor inconsistent: 3830 vs 14336 free clusters
特别值得注意的是,有一行说需要同步 341795704k,一毫秒后显示“恢复完成”。
这是怎么发生的?我们做错了什么?导致这种事情发生?
答案1
首先使用:“mdadm --zero-superblock /dev/sdb2”,然后使用“mdadm /dev/md1 --add /dev/sdb2”