今天早上,我的 FreeNAS 9.3 机器上出现了“空间不足”的警告
warning: The capacity for the volume 'SeanVolume' is currently at 85%, while the recommended value is below 80%.
scrub
一段时间后运行该命令就没有用了。
我使用 SSH 登录到 FreeNAS 主机,并得到一些信息:
[root@freenas] ~# zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
SeanVolume 21.8T 18.6T 3.16T - 38% 85% 1.00x ONLINE /mnt
freenas-boot 111G 680M 110G - - 0% 1.00x ONLINE -
显示它的ALLOC空间只有18.6T容量,但是通过du -sh
命令它仍然有6.7T:
[root@freenas] ~# du -sh /mnt/SeanVolume/
6.7T /mnt/SeanVolume/
附加信息:
[root@freenas] ~# zpool status
pool: SeanVolume
state: ONLINE
scan: scrub repaired 0 in 20h32m with 0 errors on Thu Jul 16 07:48:34 2015
config:
NAME STATE READ WRITE CKSUM
SeanVolume ONLINE 0 0 0
raidz3-0 ONLINE 0 0 0
gptid/f4986ea8-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
gptid/f51a05f7-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
gptid/f595d78e-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
gptid/f62490d4-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
gptid/f6a8e41e-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
gptid/f7266471-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
gptid/f7f3bf28-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
gptid/f8b8fa7b-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
cache
gptid/f901ec00-f822-11e4-a7d4-d05099265144 ONLINE 0 0 0
gptid/dfa53351-1baa-11e5-ba0e-d05099265144 ONLINE 0 0 0
errors: No known data errors
命令结果df -TH
:
[root@freenas] ~# df -TH
Filesystem Type Size Used Avail Capacity Mounted on
freenas-boot/ROOT/default zfs 115G 698M 114G 1% /
devfs devfs 1.0k 1.0k 0B 100% /dev
tmpfs tmpfs 33M 5.5M 28M 16% /etc
tmpfs tmpfs 4.2M 8.2k 4.2M 0% /mnt
tmpfs tmpfs 11G 67M 11G 1% /var
freenas-boot/grub zfs 114G 8.1M 114G 0% /boot/grub
SeanVolume zfs 1.6T 822k 1.6T 0% /mnt/SeanVolume
SeanVolume/CandyDataset zfs 1.6T 299k 1.6T 0% /mnt/SeanVolume/CandyDataset
SeanVolume/CandyDataset/Applications zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Applications
SeanVolume/CandyDataset/Documents zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Documents
SeanVolume/CandyDataset/Downloads zfs 2.9T 1.3T 1.6T 46% /mnt/SeanVolume/CandyDataset/Downloads
SeanVolume/CandyDataset/Music zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Music
SeanVolume/CandyDataset/Pictures zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Pictures
SeanVolume/CandyDataset/Temporary zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/CandyDataset/Temporary
SeanVolume/CandyDataset/Videos zfs 1.8T 221G 1.6T 12% /mnt/SeanVolume/CandyDataset/Videos
SeanVolume/PublicDataset zfs 1.6T 243k 1.6T 0% /mnt/SeanVolume/PublicDataset
SeanVolume/PublicDataset/Applications zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Applications
SeanVolume/PublicDataset/Documents zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Documents
SeanVolume/PublicDataset/Downloads zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Downloads
SeanVolume/PublicDataset/Music zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Music
SeanVolume/PublicDataset/Pictures zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Pictures
SeanVolume/PublicDataset/Temporary zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Temporary
SeanVolume/PublicDataset/Videos zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/PublicDataset/Videos
SeanVolume/SeanDataset zfs 3.9T 2.3T 1.6T 60% /mnt/SeanVolume/SeanDataset
SeanVolume/SeanDataset/AppData zfs 1.6T 224k 1.6T 0% /mnt/SeanVolume/SeanDataset/AppData
SeanVolume/SeanDataset/Applications zfs 1.6T 261k 1.6T 0% /mnt/SeanVolume/SeanDataset/Applications
SeanVolume/SeanDataset/Documents zfs 3.0T 1.5T 1.6T 48% /mnt/SeanVolume/SeanDataset/Documents
SeanVolume/SeanDataset/Downloads zfs 2.2T 647G 1.6T 29% /mnt/SeanVolume/SeanDataset/Downloads
SeanVolume/SeanDataset/Music zfs 1.6T 411k 1.6T 0% /mnt/SeanVolume/SeanDataset/Music
SeanVolume/SeanDataset/Pictures zfs 1.6T 2.2M 1.6T 0% /mnt/SeanVolume/SeanDataset/Pictures
SeanVolume/SeanDataset/Saved Games zfs 1.6T 243k 1.6T 0% /mnt/SeanVolume/SeanDataset/Saved Games
SeanVolume/SeanDataset/Temporary zfs 1.6T 533k 1.6T 0% /mnt/SeanVolume/SeanDataset/Temporary
SeanVolume/SeanDataset/Videos zfs 2.4T 850G 1.6T 35% /mnt/SeanVolume/SeanDataset/Videos
SeanVolume/jails zfs 1.6T 710M 1.6T 0% /mnt/SeanVolume/jails
SeanVolume/jails/.warden-template-pluginjail zfs 1.6T 619M 1.6T 0% /mnt/SeanVolume/jails/.warden-template-pluginjail
SeanVolume/jails/.warden-template-pluginjail-9.3 zfs 1.6T 619M 1.6T 0% /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3
SeanVolume/jails/pluginjail zfs 1.6T 660M 1.6T 0% /mnt/SeanVolume/jails/pluginjail
SeanVolume/ownCloudDataset zfs 1.8T 259G 1.6T 14% /mnt/SeanVolume/ownCloudDataset
SeanVolume/.system zfs 1.6T 3.4M 1.6T 0% /var/db/system
SeanVolume/.system/cores zfs 1.6T 1.4M 1.6T 0% /var/db/system/cores
SeanVolume/.system/samba4 zfs 1.6T 6M 1.6T 0% /var/db/system/samba4
SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261 zfs 1.6T 6M 1.6T 0% /var/db/system/syslog-cd1fc29ce94d4a81a24df77359252261
SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261 zfs 1.6T 224k 1.6T 0% /var/db/system/rrd-cd1fc29ce94d4a81a24df77359252261
devfs devfs 1.0k 1.0k 0B 100% /mnt/SeanVolume/jails/pluginjail/dev
procfs procfs 4.1k 4.1k 0B 100% /mnt/SeanVolume/jails/pluginjail/proc
SeanVolume/jails/customplugin_2 zfs 1.6T 1.4G 1.6T 0% /mnt/SeanVolume/jails/customplugin_2
SeanVolume/SeanDataset/ISOImage zfs 1.6T 318k 1.6T 0% /mnt/SeanVolume/SeanDataset/ISOImage
devfs devfs 1.0k 1.0k 0B 100% /mnt/SeanVolume/jails/customplugin_2/dev
procfs procfs 4.1k 4.1k 0B 100% /mnt/SeanVolume/jails/customplugin_2/proc
/mnt/SeanVolume/ownCloudDataset nullfs 1.8T 259G 1.6T 14% /mnt/SeanVolume/jails/customplugin_2/media
运行清理状态(Web GUI):
Scrub status: Completed
Errors: 0 Repaired: 0 Date: Thu Jul 16 07:48:34 2015
我该怎么做才能尝试修复此问题(除了重启,因为它正在在线运行 ownCloud 插件)?
附言:2015/07/17新增:
[root@freenas] ~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
SeanVolume 10.7T 1.34T 803K /mnt/SeanVolume
SeanVolume/.system 304M 1.34T 3.21M legacy
SeanVolume/.system/cores 5.53M 1.34T 1.32M legacy
SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261 219K 1.34T 219K legacy
SeanVolume/.system/samba4 218M 1.34T 5.59M legacy
SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261 61.4M 1.34T 5.62M legacy
SeanVolume/CandyDataset 1.42T 1.34T 292K /mnt/SeanVolume/CandyDataset
SeanVolume/CandyDataset/Applications 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Applications
SeanVolume/CandyDataset/Documents 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Documents
SeanVolume/CandyDataset/Downloads 1.22T 1.34T 1.22T /mnt/SeanVolume/CandyDataset/Downloads
SeanVolume/CandyDataset/Music 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Music
SeanVolume/CandyDataset/Pictures 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Pictures
SeanVolume/CandyDataset/Temporary 365K 1.34T 219K /mnt/SeanVolume/CandyDataset/Temporary
SeanVolume/CandyDataset/Videos 206G 1.34T 206G /mnt/SeanVolume/CandyDataset/Videos
SeanVolume/PublicDataset 3.03M 1.34T 237K /mnt/SeanVolume/PublicDataset
SeanVolume/PublicDataset/Applications 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Applications
SeanVolume/PublicDataset/Documents 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Documents
SeanVolume/PublicDataset/Downloads 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Downloads
SeanVolume/PublicDataset/Music 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Music
SeanVolume/PublicDataset/Pictures 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Pictures
SeanVolume/PublicDataset/Temporary 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Temporary
SeanVolume/PublicDataset/Videos 365K 1.34T 219K /mnt/SeanVolume/PublicDataset/Videos
SeanVolume/SeanDataset 8.04T 1.34T 2.14T /mnt/SeanVolume/SeanDataset
SeanVolume/SeanDataset/AppData 2.64M 1.34T 219K /mnt/SeanVolume/SeanDataset/AppData
SeanVolume/SeanDataset/Applications 178G 1.34T 256K /mnt/SeanVolume/SeanDataset/Applications
SeanVolume/SeanDataset/Documents 2.58T 1.34T 1.33T /mnt/SeanVolume/SeanDataset/Documents
SeanVolume/SeanDataset/Downloads 1.07T 1.34T 661G /mnt/SeanVolume/SeanDataset/Downloads
SeanVolume/SeanDataset/ISOImage 103G 1.34T 310K /mnt/SeanVolume/SeanDataset/ISOImage
SeanVolume/SeanDataset/Music 51.7G 1.34T 402K /mnt/SeanVolume/SeanDataset/Music
SeanVolume/SeanDataset/Pictures 131G 1.34T 2.08M /mnt/SeanVolume/SeanDataset/Pictures
SeanVolume/SeanDataset/Saved Games 75.7G 1.34T 237K /mnt/SeanVolume/SeanDataset/Saved Games
SeanVolume/SeanDataset/Temporary 9.63G 1.34T 520K /mnt/SeanVolume/SeanDataset/Temporary
SeanVolume/SeanDataset/Videos 846G 1.34T 792G /mnt/SeanVolume/SeanDataset/Videos
SeanVolume/jails 3.80G 1.34T 678M /mnt/SeanVolume/jails
SeanVolume/jails/.warden-template-pluginjail 6.33M 1.34T 591M /mnt/SeanVolume/jails/.warden-template-pluginjail
SeanVolume/jails/.warden-template-pluginjail-9.3 597M 1.34T 591M /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3
SeanVolume/jails/customplugin_2 1.17G 1.34T 1.27G /mnt/SeanVolume/jails/customplugin_2
SeanVolume/jails/pluginjail 1.28G 1.34T 630M /mnt/SeanVolume/jails/pluginjail
SeanVolume/ownCloudDataset 1.21T 1.34T 255G /mnt/SeanVolume/ownCloudDataset
freenas-boot 680M 107G 144K none
freenas-boot/ROOT 670M 107G 144K none
freenas-boot/ROOT/Initial-Install 8K 107G 659M legacy
freenas-boot/ROOT/default 670M 107G 665M legacy
freenas-boot/grub 7.76M 107G 7.76M legacy
2015/07/17 添加:
我使用zfs list -t snapshot
命令后,它显示了许多微小的快照(总共 2,534 个文件......)。
我正在检查,并删除那些快照文件。
删除后,df -TH
再次运行!
2015/07/20 添加:
我删除了许多快照文件后得到了一些空间!
但...
FreeNAS 9.3 Web GUI 崩溃了...
在我清除大量快照(约 1500 个)后,FreeNAS 9.3 Web GUI 显示“发生错误”。两天后
[root@freenas] /etc/defaults# df -TH
Filesystem Type Size Used Avail Capacity Mounted on
freenas-boot/ROOT/default zfs 115G 698M 114G 1% /
devfs devfs 1.0k 1.0k 0B 100% /dev
tmpfs tmpfs 33M 5.5M 28M 16% /etc
tmpfs tmpfs 4.2M 8.2k 4.2M 0% /mnt
tmpfs tmpfs 11G 67M 11G 1% /var
freenas-boot/grub zfs 114G 8.1M 114G 0% /boot/grub
SeanVolume zfs 4T 822k 4T 0% /mnt/SeanVolume
SeanVolume/CandyDataset zfs 4T 299k 4T 0% /mnt/SeanVolume/CandyDataset
SeanVolume/CandyDataset/Downloads zfs 5.3T 1.3T 4T 25% /mnt/SeanVolume/CandyDataset/Downloads
SeanVolume/CandyDataset/Videos zfs 4.2T 221G 4T 5% /mnt/SeanVolume/CandyDataset/Videos
SeanVolume/PublicDataset zfs 4T 224k 4T 0% /mnt/SeanVolume/PublicDataset
SeanVolume/SeanDataset zfs 9.6T 5.6T 4T 59% /mnt/SeanVolume/SeanDataset
SeanVolume/jails zfs 4T 710M 4T 0% /mnt/SeanVolume/jails
SeanVolume/jails/.warden-template-pluginjail zfs 4T 619M 4T 0% /mnt/SeanVolume/jails/.warden-template-pluginjail
SeanVolume/jails/.warden-template-pluginjail-9.3 zfs 4T 619M 4T 0% /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3
SeanVolume/jails/pluginjail zfs 4T 661M 4T 0% /mnt/SeanVolume/jails/pluginjail
SeanVolume/ownCloudDataset zfs 4.3T 336G 4T 8% /mnt/SeanVolume/ownCloudDataset
SeanVolume/.system zfs 4T 3.4M 4T 0% /var/db/system
SeanVolume/.system/cores zfs 4T 1.4M 4T 0% /var/db/system/cores
SeanVolume/.system/samba4 zfs 4T 6.1M 4T 0% /var/db/system/samba4
SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261 zfs 4T 11M 4T 0% /var/db/system/syslog-cd1fc29ce94d4a81a24df77359252261
SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261 zfs 4T 224k 4T 0% /var/db/system/rrd-cd1fc29ce94d4a81a24df77359252261
devfs devfs 1.0k 1.0k 0B 100% /mnt/SeanVolume/jails/pluginjail/dev
procfs procfs 4.1k 4.1k 0B 100% /mnt/SeanVolume/jails/pluginjail/proc
SeanVolume/jails/customplugin_2 zfs 4T 1.4G 4T 0% /mnt/SeanVolume/jails/customplugin_2
devfs devfs 1.0k 1.0k 0B 100% /mnt/SeanVolume/jails/customplugin_2/dev
procfs procfs 4.1k 4.1k 0B 100% /mnt/SeanVolume/jails/customplugin_2/proc
/mnt/SeanVolume/ownCloudDataset nullfs 4.3T 336G 4T 8% /mnt/SeanVolume/jails/customplugin_2/media
答案1
我尝试总结给出的信息并给出一些解释(我知道的,它意味着什么):
关于zpool 列表输出:
SIZE 是所有磁盘的大小,包括用于冗余的磁盘(因此与zpool 状态这意味着,您已将 8 个 3TB 驱动器组合成 RAID-Z3 配置)
ALLOC 是已使用空间,还包括用于冗余的磁盘(因此它至少是实际数据大小的 (5+3)/5=1.6 倍,因为 8 个磁盘中有 3 个用于冗余)
Free 是可用空间,还包括用于冗余的磁盘(因此可用空间最多为所示可用大小的 5/(5+3)=0.625
)zfs 列表命令说,有 1,34T 可用,zpool 列表说有 3,16T 可用,减去我预期的冗余空间大约是 1,975T。
我知道 zfs 需要一些空间用于元数据,但在我的 Linux 系统上,RAID-Z1 中有 4 个 2TB 驱动器,差异约为 6%,你这里的差异约为 47%,我说不出差异来自哪里。
关于输出df-钍
df 找不到 zfs 快照,因此无法计算它们,因此使用的空间不是 zfs 文件系统中使用的空间。对于 zfs 文件系统中分配的空间,请使用zfs 列表
删除快照可能需要一些时间,这似乎是设计使然,你可以看看http://nex7.blogspot.de/2013/03/readme1st.html
删除快照后,如果我计算您的 df 输出,您的池中大约有 7.2T 的数据,大约有 4T 的可用空间,总共 11.2T。
如果我将您的完整大小(21.8T)减去冗余磁盘,我预计大约有 5/(5+3)*21.8T=13.6T(相差约 20%)
我认为 20% 对于 zfs 所需的元数据来说有点太多了,但我无法告诉您它在哪里丢失了。也许您有许多非常小的文件,它们使用的元数据空间比没有那么多大文件要多。
很抱歉,但我对 zfs 内部结构了解不多,所以我无法告诉您从哪里获取元数据空间使用情况。
您可以再试一次zfs 列表并将那里的已用空间与您从 df 获得的已用空间进行比较。也许它给出了提示。
编辑于 2015 年 7 月 21 日
关于大小:3T 驱动器实际上有 2.7 TiB(因为驱动器制造商使用的是 1TB = 1.000.000.000.000 字节,其中使用 1TiB 的计算机 = 1*1024*1024*1024*1024=1.099.511.627.776 字节。2.7
* 8 个驱动器 = 21.8T
在正常的 RAID 系统中,最小的驱动器决定了较大驱动器的可用大小。