☰

Raid 10 – 非常慢

tag-icon • tag-icon linux centos raid software-raid raid10
Raid 10 – 非常慢

我已经设置了软件 Raid 10，它几个月来一直运行良好。

当我进行快速高清速度测试时：

dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync

它花了很长时间，我不得不终止它，结果是：

复制了 1073741824 字节（1.1 GB），耗时 151.27 秒，7.1 MB/s

是什么原因造成这个的？

[root@host ~]# cat /proc/mdstat
Personalities : [raid1] [raid10]
md2 : active raid1 sdc1[4] sdd1[3] sdb1[1] sda1[0]
      204788 blocks super 1.0 [4/4] [UUUU]

md127 : active raid10 sdc4[4] sdb4[1] sdd4[3] sda4[0]
      1915357184 blocks super 1.2 256K chunks 2 near-copies [4/4] [UUUU]

md1 : active raid1 sdc3[4] sdb3[1] sdd3[3] sda3[0]
      8387576 blocks super 1.1 [4/4] [UUUU]

md0 : active raid1 sdc2[4] sda2[0] sdb2[1] sdd2[3]
      10484668 blocks super 1.1 [4/4] [UUUU]
      bitmap: 1/1 pages [4KB], 65536KB chunk

正常运行时间：

[root@host ~]# uptime
 18:50:28 up 105 days, 11:34,  1 user,  load average: 0.04, 0.05, 0.00

记忆

[root@host ~]# free -m
              total       used       free     shared    buffers   cached 
 Mem:         15893      15767        125          0        461    14166
 -/+ buffers/cache:       1139      14753 
 Swap:         8190          9       8181

服务器规格：

至强 E3-1230

16GB DDR-3 ECC

4 x 1TB 7200 RPM SATA（软件 RAID）

当我执行命令时，我注意到fdisk -l- 得到一堆无效分区，这与它有什么关系吗？如果是的话，如何解决？

[root@host ~]# fdisk -l

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x000d1a79

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          26      204800   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2              26        1332    10485760   fd  Linux raid autodetect
/dev/sda3            1332        2376     8388608   fd  Linux raid autodetect
/dev/sda4            2376      121601   957679840+  fd  Linux raid autodetect

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x000303b7

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          26      204800   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdb2              26        1332    10485760   fd  Linux raid autodetect
/dev/sdb3            1332        2376     8388608   fd  Linux raid autodetect
/dev/sdb4            2376      121601   957679840+  fd  Linux raid autodetect

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *           1          26      204800   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdc2              26        1332    10485760   fd  Linux raid autodetect
/dev/sdc3            1332        2376     8388608   fd  Linux raid autodetect
/dev/sdc4            2376      121601   957679840+  fd  Linux raid autodetect

Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x0006436c

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1   *           1          26      204800   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdd2              26        1332    10485760   fd  Linux raid autodetect
/dev/sdd3            1332        2376     8388608   fd  Linux raid autodetect
/dev/sdd4            2376      121601   957679840+  fd  Linux raid autodetect

Disk /dev/md0: 10.7 GB, 10736300032 bytes
2 heads, 4 sectors/track, 2621167 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/md1: 8588 MB, 8588877824 bytes
2 heads, 4 sectors/track, 2096894 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table

Disk /dev/md127: 1961.3 GB, 1961325756416 bytes
2 heads, 4 sectors/track, 478839296 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 524288 bytes
Disk identifier: 0x00000000

Disk /dev/md127 doesn't contain a valid partition table

Disk /dev/md2: 209 MB, 209702912 bytes
2 heads, 4 sectors/track, 51197 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md2 doesn't contain a valid partition table

答案1

我认为造成这种结果的几个可能原因如下：

  1. 操作太多fdatasync()：在这种情况下，吞吐量受限于硬盘在特定时间范围内可以执行的事务数量。对于每个表面只有一个磁头的传统旋转介质，绝对最大值由旋转速度决定。例如，对于 64K 的数据块，您只能获得

    7200rpm * 64K/rotation = 7,680 K/sec

    我怀疑，由于 RAID 1+0 的语义，每次写入只会命中一个镜像，因为dd块大小小于阵列块大小。这会将阵列的性能限制为一个镜像的性能，对于写入而言，也就是单个驱动器的性能。

    在我的系统上dd，使用这些选项在退出前只执行一次fdatasync()调用 - 也许你的系统每个块都执行一次调用？运行ddunderstrace会告诉你是否是这种情况。

  2. 使用数据选项安装的文件系统sync也会表现出类似的行为。

  3. 硬件问题 - 我经常看到由于电缆故障和硬盘故障而导致的此类延迟。我建议仔细阅读您的系统日志并smartctl在所有驱动器上运行。读取测试也很有用，因为它也应该受到硬件问题的影响 - 您的阵列在读取方面表现如何？

相关内容