☰

采用 xfs 文件系统的硬盘在子文件夹中的写入性能不佳

tag-icon • tag-icon performance xfs write
采用 xfs 文件系统的硬盘在子文件夹中的写入性能不佳

我们服务器有raid6阵列（10+2个盘，每个盘6TB/SATA/7200rpm），打算做成NFS存储，和其他客户端共享，但是写入速度很慢：

[root@Server ~]# mkfs.xfs /dev/sdd1
meta-data=/dev/sdd1              isize=512    agcount=55, agsize=268435455 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=14640739328, imaxpct=1
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=521728, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

[root@Server ~]# mount /dev/sdd1 /share/
[root@Server ~]# mkdir -p /share/aa/bb/cc/dd/ee
[root@Server ~]# cd /share/aa/bb/cc/dd/ee
[root@Server ~]# dd if=/dev/zero of=tempfile bs=1M count=30000 conv=fsync
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 1186.26 s, 26.5 MB/s
[root@Server ~]# cd /share
[root@Server ~]# dd if=/dev/zero of=tempfile bs=1M count=30000 conv=fsync
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 109.495 s, 287 MB/s

奇怪的是写入速度/share（287MB/s）比写入更深的子文件夹（26MB/s）快得多。我测试了很多次，结果都差不多。

我也尝试将其格式化为其他文件系统，例如 ext4 和 brtfs，但它们在子文件夹中均表现正常（592MB/s 和 226MB/s）。

ext4：

[root@Server ~]# mkfs.ext4 /dev/sdd1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
915046400 inodes, 14640739328 blocks
732036966 blocks (5.00%) reserved for the super user
First data block=0
446800 block groups
32768 blocks per group, 32768 fragments per group
2048 inodes per group
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
    102400000, 214990848, 512000000, 550731776, 644972544, 1934917632, 
    2560000000, 3855122432, 5804752896, 12800000000

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done         

[root@Server ~]# mount /dev/sdd1 /share/
[root@Server ~]# mkdir -p /share/aa/bb/cc/dd/ee
[root@Server ~]# cd /share/aa/bb/cc/dd/ee
[root@Server ~]# dd if=/dev/zero of=tempfile bs=1M count=30000 conv=fsync
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 53.1054 s, 592 MB/s

btrfs：

[root@Server ~]# mkfs.btrfs /dev/sdd1
btrfs-progs v4.9.1
See http://btrfs.wiki.kernel.org for more information.

Label:              (null)
UUID:               32aa3677-7898-4f14-9158-b698bd11abbc
Node size:          16384
Sector size:        4096
Filesystem size:    54.54TiB
Block group profiles:
  Data:             single            8.00MiB
  Metadata:         DUP               1.00GiB
  System:           DUP               8.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata
Number of devices:  1
Devices:
   ID        SIZE  PATH
    1    54.54TiB  /dev/sdd1

[root@Server ~]# mount /dev/sdd1 /share/
[root@Server ~]# mkdir -p /share/aa/bb/cc/dd/ee
[root@Server ~]# cd /share/aa/bb/cc/dd/ee
[root@Server ~]# dd if=/dev/zero of=tempfile bs=1M count=30000 conv=fsync
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 139.226 s, 226 MB/s

我在 Google 上搜索了几个星期，但没有找到解决方案。有人能给我一些提示吗？

PS：操作系统是CentOS 7.4

相关内容