我有一个 16 核虚拟化 FreeNAS 11.3-U4.1 实例,在 Hyper-V 上运行,具有 32GB RAM 和 18TB iscsi 目标,在 DELL T630 服务器级硬件上。
HDD 处于 RAID 卡的 IT 模式,因此作为单独的驱动器呈现给 FreeNAS 盒。
我没有运行任何重复数据删除,只有默认压缩。
Windows 共享在 Windows Server 2019 上以 iscsi 启动器的形式运行 ReFS,并且未满,只有 1TB 左右,用于传输测试电影文件。
从测试的多台机器(每次一台)来看,它在任何传输上最初都很快 - 读取或写入@130MB/秒 - 但是大约一分钟后就会减慢到40Mbps,有时甚至会停滞。
网络测试速度很快,无论是通过同一机器上的内部机器,还是外部机器:
root@freenas[~]# iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
^Ciperf3: interrupt - the server has terminated
root@freenas[~]# iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.69.139, port 49435
[ 5] local 192.168.69.4 port 5201 connected to 192.168.69.139 port 49437
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 105 MBytes 878 Mbits/sec
[ 5] 1.00-2.00 sec 112 MBytes 941 Mbits/sec
[ 5] 2.00-3.00 sec 112 MBytes 938 Mbits/sec
[ 5] 3.00-4.00 sec 112 MBytes 939 Mbits/sec
[ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec
[ 5] 5.00-6.00 sec 111 MBytes 935 Mbits/sec
[ 5] 6.00-7.00 sec 112 MBytes 939 Mbits/sec
[ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec
[ 5] 8.00-9.00 sec 112 MBytes 942 Mbits/sec
[ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec
[ 5] 10.00-10.00 sec 339 KBytes 955 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 1.09 GBytes 933 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.69.5, port 50601
[ 5] local 192.168.69.4 port 5201 connected to 192.168.69.5 port 50602
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 392 MBytes 3.29 Gbits/sec
[ 5] 1.00-2.00 sec 407 MBytes 3.41 Gbits/sec
[ 5] 2.00-3.00 sec 426 MBytes 3.57 Gbits/sec
[ 5] 3.00-4.00 sec 430 MBytes 3.60 Gbits/sec
[ 5] 4.00-5.00 sec 413 MBytes 3.46 Gbits/sec
[ 5] 5.00-6.00 sec 429 MBytes 3.60 Gbits/sec
[ 5] 6.00-7.00 sec 372 MBytes 3.12 Gbits/sec
[ 5] 7.00-8.00 sec 429 MBytes 3.60 Gbits/sec
[ 5] 8.00-9.00 sec 424 MBytes 3.56 Gbits/sec
[ 5] 9.00-10.00 sec 425 MBytes 3.56 Gbits/sec
[ 5] 10.00-10.00 sec 125 KBytes 2.29 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
通过 top 进行的 CPU 负载最小:
last pid: 15377; load averages: 1.15, 0.90, 1.06 up 0+12:42:50 03:48:22
44 processes: 1 running, 43 sleeping
CPU: 0.1% user, 0.0% nice, 15.7% system, 0.0% interrupt, 84.2% idle
Mem: 14M Active, 488M Inact, 1060M Laundry, 28G Wired, 1584M Free
ARC: 25G Total, 852M MFU, 19G MRU, 4773M Anon, 580M Header, 75M Other
20G Compressed, 23G Uncompressed, 1.12:1 Ratio
Swap: 6144M Total, 46M Used, 6097M Free
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
104 root 41 20 0 329M 258M kqread 5 3:15 0.77% python3.7
15347 root 1 20 0 7892K 3452K CPU5 5 0:00 0.16% top
1917 www 1 20 0 30920K 10268K kqread 6 0:01 0.03% nginx
93 root 1 20 0 9164K 5548K select 7 0:04 0.02% devd
5597 root 8 20 0 40284K 17892K select 6 1:25 0.00% rrdcached
1323 root 1 20 0 12484K 12580K select 2 0:02 0.00% ntpd
1522 root 1 20 0 9012K 5024K select 5 0:01 0.00% zfsd
1433 root 1 20 0 127M 105M kqread 2 0:03 0.00% uwsgi-3.7
5622 root 11 20 0 354M 261M nanslp 7 6:57 0.00% collectd
zpools 如下:
root@freenas[~]# zpool list -v
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
freenas-boot 110G 1.25G 109G - - 0% 1% 1.00x ONLINE -
da0p2 110G 1.25G 109G - - 0% 1%
pool1 32.5T 2.60T 29.9T - - 1% 7% 1.00x ONLINE /mnt
raidz2 32.5T 2.60T 29.9T - - 1% 7%
gptid/e7aba92d-dc27-11ea-b1e2-00155d450302 - - - - - - -
gptid/ea0339c1-dc27-11ea-b1e2-00155d450302 - - - - - - -
gptid/eadd3d73-dc27-11ea-b1e2-00155d450302 - - - - - - -
gptid/eb18ed64-dc27-11ea-b1e2-00155d450302 - - - - - - -
gptid/eb80f3cc-dc27-11ea-b1e2-00155d450302 - - - - - - -
gptid/eb99fe09-dc27-11ea-b1e2-00155d450302 - - - - - - -
使用 root@freenas[~]# ./solnet-array-test-v2.sh 我可以看到(写入显示将花费 467 分钟,我确信写入速度没问题,因为最初它们很快):
Disk Disk Size MB/sec %ofAvg
------- ---------- ------ ------
da0 130048MB 658 268 ++FAST++
da1 5723166MB 204 83 --SLOW--
da2 5723166MB 176 72 --SLOW--
da3 5723166MB 164 67 --SLOW--
da4 5723166MB 161 66 --SLOW--
da5 5723166MB 186 76 --SLOW--
da6 5723166MB 167 68 --SLOW--
Performing initial parallel array read
Wed Aug 12 04:06:35 PDT 2020
The disk da0 appears to be 130048 MB.
Disk is reading at about 1075 MB/sec
This suggests that this pass may take around 2 minutes
Serial Parall % of
Disk Disk Size MB/sec MB/sec Serial
------- ---------- ------ ------ ------
da0 130048MB 658 0 0 --SLOW--
da1 5723166MB 204 204 100
da2 5723166MB 176 164 93
da3 5723166MB 164 166 101
da4 5723166MB 161 159 99
另外,具有相同硬件的 Windows 存储空间的恒定速度为 150MB/s(与我在上面测试中看到的差不多),但在实际使用 Freenas 时,1 分钟后速度不会减慢到 30-50MB/秒。
我需要运行任何其他诊断程序或知道这可能是什么吗?
答案1
FreeNAS 和 BSD 内核上的整个 ZFS 故事正在消亡,采用 Ubuntu 和 ZoL,无论如何都有更广泛的社区覆盖和更好的硬件支持。