我刚刚安装了两个三星 970 EVO Plus 硬盘，并将最大的分区配置为 RAID 10（远 2 布局）。问题是菲奥仅报告读取速度为 72 MB/s。我使用以下命令来构建阵列；
mdadm --create --verbose --level=10 --metadata=1.2 --chunk=512 --raid-devices=2 --layout=f2 /dev/md/MyRAID10Array /dev/nvme0n1p3 /dev/nvme1n1p3
这应该能让我接近标称读取速度 3500 MB/s 的两倍。我已使用默认选项格式化了它西弗斯，
mkfs.xfs /dev/md127
磁盘布局如下
fdisk -l /dev/nvme0n1
输出：
Disk /dev/nvme0n1: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: Samsung SSD 970 EVO Plus 1TB
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: FF148851-E4A9-41D2-96C6-418280164BC6
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 821247 819200 400M Linux RAID
/dev/nvme0n1p2 821248 67930111 67108864 32G Linux swap
/dev/nvme0n1p3 67930112 1953523711 1885593600 899.1G Linux RAID
mdstat 报告
md127：活动 raid10 nvme1n1p3[0] nvme0n1p33 942664704 块超级 1.2 512K 块 2 远拷贝 [2/2] [UU] 位图：0/8 页 [0KB]，65536KB 块
但当我通过 FIO 运行它时，
; Random read of 128 MB of data
[random-read]
rw=randread
size=128m
directory=/backup_mount_point
我只得到非常糟糕的数字；
random-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.35
Starting 1 process
Jobs: 1 (f=1)
random-read: (groupid=0, jobs=1): err= 0: pid=214183: Sun Oct 1 06:28:06 2023
read: IOPS=18.2k, BW=71.3MiB/s (74.7MB/s)(128MiB/1796msec)
clat (usec): min=14, max=6880, avg=54.09, stdev=53.44
lat (usec): min=14, max=6880, avg=54.12, stdev=53.45
clat percentiles (usec):
| 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 52],
| 30.00th=[ 52], 40.00th=[ 53], 50.00th=[ 53], 60.00th=[ 53],
| 70.00th=[ 54], 80.00th=[ 55], 90.00th=[ 57], 95.00th=[ 58],
| 99.00th=[ 75], 99.50th=[ 76], 99.90th=[ 78], 99.95th=[ 80],
| 99.99th=[ 123]
bw ( KiB/s): min=70720, max=74920, per=100.00%, avg=73509.33, stdev=2415.69, samples=3
iops : min=17680, max=18730, avg=18377.33, stdev=603.92, samples=3
lat (usec) : 20=0.03%, 50=0.02%, 100=99.93%, 250=0.02%
lat (msec) : 10=0.01%
cpu : usr=0.39%, sys=9.25%, ctx=32778, majf=0, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=32768,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=71.3MiB/s (74.7MB/s), 71.3MiB/s-71.3MiB/s (74.7MB/s-74.7MB/s), io=128MiB (134MB), run=1796-1796msec
Disk stats (read/write):
md127: ios=30225/0, merge=0/0, ticks=1407/0, in_queue=1407, util=93.46%, aggrios=16384/0, aggrmerge=0/0, aggrticks=784/0, aggrin_queue=784, aggrutil=93.19%
nvme0n1: ios=16384/0, merge=0/0, ticks=768/0, in_queue=768, util=93.04%
nvme1n1: ios=16384/0, merge=0/0, ticks=800/0, in_queue=800, util=93.19%
如果我转到顺序 (rw=read)，速度会提高到 1803 MB/s，但仍不到我预期的三分之一。我正在运行Arch Linux在配备 64 GB RAM 的 AMD Ryzen 系统上。主板是微星X570S Edge Wifi 最大。
编辑；在 Batistuta9 的回答之后，我改变了我的测试，并获得了明显更快的结果。如果您感兴趣的话，以下是 FIO 摘要；
random-read: (groupid=0, jobs=1): err= 0: pid=143332: Thu Oct 5 23:23:49 2023
read: IOPS=15.0k, BW=58.7MiB/s (61.5MB/s)(20.0GiB/348908msec)
clat (usec): min=48, max=2746, avg=66.21, stdev=16.90
lat (usec): min=48, max=2746, avg=66.24, stdev=16.90
clat percentiles (usec):
| 1.00th=[ 49], 5.00th=[ 50], 10.00th=[ 51], 20.00th=[ 51],
| 30.00th=[ 52], 40.00th=[ 54], 50.00th=[ 69], 60.00th=[ 72],
| 70.00th=[ 74], 80.00th=[ 81], 90.00th=[ 88], 95.00th=[ 91],
| 99.00th=[ 120], 99.50th=[ 130], 99.90th=[ 149], 99.95th=[ 157],
| 99.99th=[ 184]
bw ( KiB/s): min=57064, max=61112, per=14.93%, avg=60131.86, stdev=374.32, samples=697
iops : min=14266, max=15278, avg=15032.97, stdev=93.58, samples=697
lat (usec) : 50=6.97%, 100=90.22%, 250=2.81%, 500=0.01%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.74%, sys=5.61%, ctx=5246560, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=5242880,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
random-read: (groupid=0, jobs=1): err= 0: pid=143333: Thu Oct 5 23:23:49 2023
read: IOPS=15.2k, BW=59.3MiB/s (62.2MB/s)(20.0GiB/345469msec)
clat (usec): min=48, max=2828, avg=65.55, stdev=16.90
lat (usec): min=48, max=2828, avg=65.58, stdev=16.90
clat percentiles (usec):
| 1.00th=[ 49], 5.00th=[ 50], 10.00th=[ 51], 20.00th=[ 51],
| 30.00th=[ 52], 40.00th=[ 53], 50.00th=[ 60], 60.00th=[ 71],
| 70.00th=[ 73], 80.00th=[ 79], 90.00th=[ 88], 95.00th=[ 90],
| 99.00th=[ 120], 99.50th=[ 130], 99.90th=[ 149], 99.95th=[ 157],
| 99.99th=[ 184]
bw ( KiB/s): min=59472, max=61528, per=15.08%, avg=60729.74, stdev=344.11, samples=690
iops : min=14868, max=15382, avg=15182.43, stdev=86.02, samples=690
lat (usec) : 50=6.62%, 100=90.63%, 250=2.75%, 500=0.01%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.76%, sys=5.63%, ctx=5246568, majf=0, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=5242880,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
random-read: (groupid=0, jobs=1): err= 0: pid=143334: Thu Oct 5 23:23:49 2023
read: IOPS=14.3k, BW=55.8MiB/s (58.5MB/s)(20.0GiB/366867msec)
clat (usec): min=48, max=2742, avg=69.63, stdev=16.68
lat (usec): min=48, max=2742, avg=69.66, stdev=16.69
clat percentiles (usec):
| 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 51], 20.00th=[ 52],
| 30.00th=[ 54], 40.00th=[ 70], 50.00th=[ 72], 60.00th=[ 73],
| 70.00th=[ 75], 80.00th=[ 86], 90.00th=[ 88], 95.00th=[ 92],
| 99.00th=[ 123], 99.50th=[ 133], 99.90th=[ 151], 99.95th=[ 159],
| 99.99th=[ 184]
bw ( KiB/s): min=55232, max=58272, per=14.20%, avg=57191.21, stdev=322.36, samples=733
iops : min=13808, max=14568, avg=14297.80, stdev=80.58, samples=733
lat (usec) : 50=3.98%, 100=92.88%, 250=3.14%, 500=0.01%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.74%, sys=5.33%, ctx=5246807, majf=0, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=5242880,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
random-read: (groupid=0, jobs=1): err= 0: pid=143335: Thu Oct 5 23:23:49 2023
read: IOPS=12.6k, BW=49.2MiB/s (51.6MB/s)(20.0GiB/416494msec)
clat (usec): min=56, max=2742, avg=79.09, stdev=12.05
lat (usec): min=56, max=2742, avg=79.12, stdev=12.05
clat percentiles (usec):
| 1.00th=[ 64], 5.00th=[ 70], 10.00th=[ 71], 20.00th=[ 72],
| 30.00th=[ 73], 40.00th=[ 73], 50.00th=[ 74], 60.00th=[ 78],
| 70.00th=[ 86], 80.00th=[ 88], 90.00th=[ 90], 95.00th=[ 95],
| 99.00th=[ 128], 99.50th=[ 137], 99.90th=[ 155], 99.95th=[ 163],
| 99.99th=[ 190]
bw ( KiB/s): min=48912, max=51584, per=12.51%, avg=50377.18, stdev=326.60, samples=832
iops : min=12228, max=12896, avg=12594.28, stdev=81.63, samples=832
lat (usec) : 100=96.05%, 250=3.95%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.64%, sys=4.86%, ctx=5247122, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=5242880,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
random-read: (groupid=0, jobs=1): err= 0: pid=143336: Thu Oct 5 23:23:49 2023
read: IOPS=12.6k, BW=49.2MiB/s (51.6MB/s)(20.0GiB/416501msec)
clat (usec): min=57, max=2889, avg=79.08, stdev=12.27
lat (usec): min=57, max=2889, avg=79.12, stdev=12.27
clat percentiles (usec):
| 1.00th=[ 64], 5.00th=[ 70], 10.00th=[ 71], 20.00th=[ 72],
| 30.00th=[ 73], 40.00th=[ 73], 50.00th=[ 74], 60.00th=[ 78],
| 70.00th=[ 86], 80.00th=[ 88], 90.00th=[ 90], 95.00th=[ 95],
| 99.00th=[ 128], 99.50th=[ 137], 99.90th=[ 155], 99.95th=[ 163],
| 99.99th=[ 188]
bw ( KiB/s): min=48640, max=51608, per=12.51%, avg=50376.18, stdev=314.71, samples=832
iops : min=12160, max=12902, avg=12594.03, stdev=78.66, samples=832
lat (usec) : 100=96.08%, 250=3.92%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.68%, sys=4.83%, ctx=5247298, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=5242880,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
random-read: (groupid=0, jobs=1): err= 0: pid=143337: Thu Oct 5 23:23:49 2023
read: IOPS=12.6k, BW=49.3MiB/s (51.7MB/s)(20.0GiB/415249msec)
clat (usec): min=14, max=3163, avg=78.85, stdev=12.35
lat (usec): min=14, max=3163, avg=78.88, stdev=12.35
clat percentiles (usec):
| 1.00th=[ 60], 5.00th=[ 70], 10.00th=[ 71], 20.00th=[ 72],
| 30.00th=[ 73], 40.00th=[ 73], 50.00th=[ 74], 60.00th=[ 77],
| 70.00th=[ 86], 80.00th=[ 88], 90.00th=[ 90], 95.00th=[ 95],
| 99.00th=[ 128], 99.50th=[ 137], 99.90th=[ 153], 99.95th=[ 161],
| 99.99th=[ 190]
bw ( KiB/s): min=49104, max=51799, per=12.54%, avg=50526.68, stdev=317.81, samples=830
iops : min=12276, max=12949, avg=12631.66, stdev=79.43, samples=830
lat (usec) : 20=0.01%, 50=0.04%, 100=96.03%, 250=3.93%, 500=0.01%
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.61%, sys=4.90%, ctx=5246732, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=5242880,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
random-read: (groupid=0, jobs=1): err= 0: pid=143338: Thu Oct 5 23:23:49 2023
read: IOPS=12.7k, BW=49.5MiB/s (51.9MB/s)(20.0GiB/413718msec)
clat (usec): min=48, max=2834, avg=78.55, stdev=12.65
lat (usec): min=48, max=2834, avg=78.59, stdev=12.65
clat percentiles (usec):
| 1.00th=[ 52], 5.00th=[ 70], 10.00th=[ 71], 20.00th=[ 72],
| 30.00th=[ 72], 40.00th=[ 73], 50.00th=[ 74], 60.00th=[ 77],
| 70.00th=[ 86], 80.00th=[ 88], 90.00th=[ 89], 95.00th=[ 95],
| 99.00th=[ 127], 99.50th=[ 137], 99.90th=[ 153], 99.95th=[ 161],
| 99.99th=[ 190]
bw ( KiB/s): min=49776, max=51912, per=12.59%, avg=50713.30, stdev=309.76, samples=827
iops : min=12444, max=12978, avg=12678.31, stdev=77.41, samples=827
lat (usec) : 50=0.20%, 100=95.92%, 250=3.87%, 500=0.01%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.65%, sys=4.89%, ctx=5246845, majf=0, minf=12
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=5242880,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
random-read: (groupid=0, jobs=1): err= 0: pid=143339: Thu Oct 5 23:23:49 2023
read: IOPS=13.3k, BW=52.0MiB/s (54.5MB/s)(20.0GiB/394165msec)
clat (usec): min=47, max=2904, avg=74.83, stdev=15.26
lat (usec): min=47, max=2904, avg=74.86, stdev=15.26
clat percentiles (usec):
| 1.00th=[ 50], 5.00th=[ 51], 10.00th=[ 53], 20.00th=[ 70],
| 30.00th=[ 71], 40.00th=[ 72], 50.00th=[ 73], 60.00th=[ 75],
| 70.00th=[ 85], 80.00th=[ 87], 90.00th=[ 89], 95.00th=[ 94],
| 99.00th=[ 126], 99.50th=[ 135], 99.90th=[ 153], 99.95th=[ 161],
| 99.99th=[ 188]
bw ( KiB/s): min=52096, max=54312, per=13.21%, avg=53229.10, stdev=312.71, samples=788
iops : min=13024, max=13578, avg=13307.26, stdev=78.16, samples=788
lat (usec) : 50=1.65%, 100=94.73%, 250=3.62%, 500=0.01%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.67%, sys=5.07%, ctx=5246945, majf=0, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=5242880,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=393MiB/s (412MB/s), 49.2MiB/s-59.3MiB/s (51.6MB/s-62.2MB/s), io=160GiB (172GB), run=345469-416501msec
Disk stats (read/write):
md127: ios=41940854/14, merge=0/0, ticks=2850618/100, in_queue=2850718, util=100.00%, aggrios=20971520/47, aggrmerge=0/2, aggrticks=1428779/139, aggrin_queue=1428932, aggrutil=100.00%
nvme0n1: ios=21179809/47, merge=0/2, ticks=1427626/139, in_queue=1427779, util=100.00%
nvme1n1: ios=20763231/47, merge=0/2, ticks=1429932/139, in_queue=1430086, util=100.00%
答案1
你得到了预期的结果单身的线4k 块大小性能基准队列深度 1. 在此处查看驱动器的规格 - https://semiconductor.samsung.com/consumer-storage/internal-ssd/970evoplus/。对于此类模式，结果应高达 19000 IOPS（您得到了 18200 IOPS）。在 fio 基准测试文件（随机读取）中添加以下参数并重新运行测试：
numjobs=8
iodepth=64
direct=1
ioengine=libaio
我还建议将基准测试文件的大小从 128MB 增加到至少 20GB。