FreeNAS 9.3 上没有更多空间。但根据“du -sh”命令,实际使用的空间只有一半

FreeNAS 9.3 上没有更多空间。但根据“du -sh”命令,实际使用的空间只有一半

今天早上,我的 FreeNAS 9.3 机器上出现了“空间不足”的警告

warning: The capacity for the volume 'SeanVolume' is currently at 85%, while the recommended value is below 80%.

scrub一段时间后运行该命令就没有用了。

我使用 SSH 登录到 FreeNAS 主机,并得到一些信息:

[root@freenas] ~# zpool list
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
SeanVolume    21.8T  18.6T  3.16T         -    38%    85%  1.00x  ONLINE  /mnt
freenas-boot   111G   680M   110G         -      -     0%  1.00x  ONLINE  -

显示它的ALLOC空间只有18.6T容量,但是通过du -sh命令它仍然有6.7T:

[root@freenas] ~# du -sh /mnt/SeanVolume/
6.7T    /mnt/SeanVolume/

附加信息:

[root@freenas] ~# zpool status
  pool: SeanVolume
 state: ONLINE
  scan: scrub repaired 0 in 20h32m with 0 errors on Thu Jul 16 07:48:34 2015
config:

        NAME                                            STATE     READ WRITE CKSUM
        SeanVolume                                      ONLINE       0     0     0
          raidz3-0                                      ONLINE       0     0     0
            gptid/f4986ea8-f822-11e4-a7d4-d05099265144  ONLINE       0     0     0
            gptid/f51a05f7-f822-11e4-a7d4-d05099265144  ONLINE       0     0     0
            gptid/f595d78e-f822-11e4-a7d4-d05099265144  ONLINE       0     0     0
            gptid/f62490d4-f822-11e4-a7d4-d05099265144  ONLINE       0     0     0
            gptid/f6a8e41e-f822-11e4-a7d4-d05099265144  ONLINE       0     0     0
            gptid/f7266471-f822-11e4-a7d4-d05099265144  ONLINE       0     0     0
            gptid/f7f3bf28-f822-11e4-a7d4-d05099265144  ONLINE       0     0     0
            gptid/f8b8fa7b-f822-11e4-a7d4-d05099265144  ONLINE       0     0     0
        cache
          gptid/f901ec00-f822-11e4-a7d4-d05099265144    ONLINE       0     0     0
          gptid/dfa53351-1baa-11e5-ba0e-d05099265144    ONLINE       0     0     0

errors: No known data errors

命令结果df -TH

[root@freenas] ~# df -TH
Filesystem                                                  Type      Size    Used   Avail Capacity  Mounted on
freenas-boot/ROOT/default                                   zfs       115G    698M    114G     1%    /
devfs                                                       devfs     1.0k    1.0k      0B   100%    /dev
tmpfs                                                       tmpfs      33M    5.5M     28M    16%    /etc
tmpfs                                                       tmpfs     4.2M    8.2k    4.2M     0%    /mnt
tmpfs                                                       tmpfs      11G     67M     11G     1%    /var
freenas-boot/grub                                           zfs       114G    8.1M    114G     0%    /boot/grub
SeanVolume                                                  zfs       1.6T    822k    1.6T     0%    /mnt/SeanVolume
SeanVolume/CandyDataset                                     zfs       1.6T    299k    1.6T     0%    /mnt/SeanVolume/CandyDataset
SeanVolume/CandyDataset/Applications                        zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/CandyDataset/Applications
SeanVolume/CandyDataset/Documents                           zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/CandyDataset/Documents
SeanVolume/CandyDataset/Downloads                           zfs       2.9T    1.3T    1.6T    46%    /mnt/SeanVolume/CandyDataset/Downloads
SeanVolume/CandyDataset/Music                               zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/CandyDataset/Music
SeanVolume/CandyDataset/Pictures                            zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/CandyDataset/Pictures
SeanVolume/CandyDataset/Temporary                           zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/CandyDataset/Temporary
SeanVolume/CandyDataset/Videos                              zfs       1.8T    221G    1.6T    12%    /mnt/SeanVolume/CandyDataset/Videos
SeanVolume/PublicDataset                                    zfs       1.6T    243k    1.6T     0%    /mnt/SeanVolume/PublicDataset
SeanVolume/PublicDataset/Applications                       zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/PublicDataset/Applications
SeanVolume/PublicDataset/Documents                          zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/PublicDataset/Documents
SeanVolume/PublicDataset/Downloads                          zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/PublicDataset/Downloads
SeanVolume/PublicDataset/Music                              zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/PublicDataset/Music
SeanVolume/PublicDataset/Pictures                           zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/PublicDataset/Pictures
SeanVolume/PublicDataset/Temporary                          zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/PublicDataset/Temporary
SeanVolume/PublicDataset/Videos                             zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/PublicDataset/Videos
SeanVolume/SeanDataset                                      zfs       3.9T    2.3T    1.6T    60%    /mnt/SeanVolume/SeanDataset
SeanVolume/SeanDataset/AppData                              zfs       1.6T    224k    1.6T     0%    /mnt/SeanVolume/SeanDataset/AppData
SeanVolume/SeanDataset/Applications                         zfs       1.6T    261k    1.6T     0%    /mnt/SeanVolume/SeanDataset/Applications
SeanVolume/SeanDataset/Documents                            zfs       3.0T    1.5T    1.6T    48%    /mnt/SeanVolume/SeanDataset/Documents
SeanVolume/SeanDataset/Downloads                            zfs       2.2T    647G    1.6T    29%    /mnt/SeanVolume/SeanDataset/Downloads
SeanVolume/SeanDataset/Music                                zfs       1.6T    411k    1.6T     0%    /mnt/SeanVolume/SeanDataset/Music
SeanVolume/SeanDataset/Pictures                             zfs       1.6T    2.2M    1.6T     0%    /mnt/SeanVolume/SeanDataset/Pictures
SeanVolume/SeanDataset/Saved Games                          zfs       1.6T    243k    1.6T     0%    /mnt/SeanVolume/SeanDataset/Saved Games
SeanVolume/SeanDataset/Temporary                            zfs       1.6T    533k    1.6T     0%    /mnt/SeanVolume/SeanDataset/Temporary
SeanVolume/SeanDataset/Videos                               zfs       2.4T    850G    1.6T    35%    /mnt/SeanVolume/SeanDataset/Videos
SeanVolume/jails                                            zfs       1.6T    710M    1.6T     0%    /mnt/SeanVolume/jails
SeanVolume/jails/.warden-template-pluginjail                zfs       1.6T    619M    1.6T     0%    /mnt/SeanVolume/jails/.warden-template-pluginjail
SeanVolume/jails/.warden-template-pluginjail-9.3            zfs       1.6T    619M    1.6T     0%    /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3
SeanVolume/jails/pluginjail                                 zfs       1.6T    660M    1.6T     0%    /mnt/SeanVolume/jails/pluginjail
SeanVolume/ownCloudDataset                                  zfs       1.8T    259G    1.6T    14%    /mnt/SeanVolume/ownCloudDataset
SeanVolume/.system                                          zfs       1.6T    3.4M    1.6T     0%    /var/db/system
SeanVolume/.system/cores                                    zfs       1.6T    1.4M    1.6T     0%    /var/db/system/cores
SeanVolume/.system/samba4                                   zfs       1.6T      6M    1.6T     0%    /var/db/system/samba4
SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261  zfs       1.6T      6M    1.6T     0%    /var/db/system/syslog-cd1fc29ce94d4a81a24df77359252261
SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261     zfs       1.6T    224k    1.6T     0%    /var/db/system/rrd-cd1fc29ce94d4a81a24df77359252261
devfs                                                       devfs     1.0k    1.0k      0B   100%    /mnt/SeanVolume/jails/pluginjail/dev
procfs                                                      procfs    4.1k    4.1k      0B   100%    /mnt/SeanVolume/jails/pluginjail/proc
SeanVolume/jails/customplugin_2                             zfs       1.6T    1.4G    1.6T     0%    /mnt/SeanVolume/jails/customplugin_2
SeanVolume/SeanDataset/ISOImage                             zfs       1.6T    318k    1.6T     0%    /mnt/SeanVolume/SeanDataset/ISOImage
devfs                                                       devfs     1.0k    1.0k      0B   100%    /mnt/SeanVolume/jails/customplugin_2/dev
procfs                                                      procfs    4.1k    4.1k      0B   100%    /mnt/SeanVolume/jails/customplugin_2/proc
/mnt/SeanVolume/ownCloudDataset                             nullfs    1.8T    259G    1.6T    14%    /mnt/SeanVolume/jails/customplugin_2/media

运行清理状态(Web GUI):

Scrub status: Completed
Errors: 0     Repaired: 0     Date: Thu Jul 16 07:48:34 2015

我该怎么做才能尝试修复此问题(除了重启,因为它正在在线运行 ownCloud 插件)?


附言:2015/07/17新增:

[root@freenas] ~# zfs list
NAME                                                         USED  AVAIL  REFER  MOUNTPOINT
SeanVolume                                                  10.7T  1.34T   803K  /mnt/SeanVolume
SeanVolume/.system                                           304M  1.34T  3.21M  legacy
SeanVolume/.system/cores                                    5.53M  1.34T  1.32M  legacy
SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261      219K  1.34T   219K  legacy
SeanVolume/.system/samba4                                    218M  1.34T  5.59M  legacy
SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261  61.4M  1.34T  5.62M  legacy
SeanVolume/CandyDataset                                     1.42T  1.34T   292K  /mnt/SeanVolume/CandyDataset
SeanVolume/CandyDataset/Applications                         365K  1.34T   219K  /mnt/SeanVolume/CandyDataset/Applications
SeanVolume/CandyDataset/Documents                            365K  1.34T   219K  /mnt/SeanVolume/CandyDataset/Documents
SeanVolume/CandyDataset/Downloads                           1.22T  1.34T  1.22T  /mnt/SeanVolume/CandyDataset/Downloads
SeanVolume/CandyDataset/Music                                365K  1.34T   219K  /mnt/SeanVolume/CandyDataset/Music
SeanVolume/CandyDataset/Pictures                             365K  1.34T   219K  /mnt/SeanVolume/CandyDataset/Pictures
SeanVolume/CandyDataset/Temporary                            365K  1.34T   219K  /mnt/SeanVolume/CandyDataset/Temporary
SeanVolume/CandyDataset/Videos                               206G  1.34T   206G  /mnt/SeanVolume/CandyDataset/Videos
SeanVolume/PublicDataset                                    3.03M  1.34T   237K  /mnt/SeanVolume/PublicDataset
SeanVolume/PublicDataset/Applications                        365K  1.34T   219K  /mnt/SeanVolume/PublicDataset/Applications
SeanVolume/PublicDataset/Documents                           365K  1.34T   219K  /mnt/SeanVolume/PublicDataset/Documents
SeanVolume/PublicDataset/Downloads                           365K  1.34T   219K  /mnt/SeanVolume/PublicDataset/Downloads
SeanVolume/PublicDataset/Music                               365K  1.34T   219K  /mnt/SeanVolume/PublicDataset/Music
SeanVolume/PublicDataset/Pictures                            365K  1.34T   219K  /mnt/SeanVolume/PublicDataset/Pictures
SeanVolume/PublicDataset/Temporary                           365K  1.34T   219K  /mnt/SeanVolume/PublicDataset/Temporary
SeanVolume/PublicDataset/Videos                              365K  1.34T   219K  /mnt/SeanVolume/PublicDataset/Videos
SeanVolume/SeanDataset                                      8.04T  1.34T  2.14T  /mnt/SeanVolume/SeanDataset
SeanVolume/SeanDataset/AppData                              2.64M  1.34T   219K  /mnt/SeanVolume/SeanDataset/AppData
SeanVolume/SeanDataset/Applications                          178G  1.34T   256K  /mnt/SeanVolume/SeanDataset/Applications
SeanVolume/SeanDataset/Documents                            2.58T  1.34T  1.33T  /mnt/SeanVolume/SeanDataset/Documents
SeanVolume/SeanDataset/Downloads                            1.07T  1.34T   661G  /mnt/SeanVolume/SeanDataset/Downloads
SeanVolume/SeanDataset/ISOImage                              103G  1.34T   310K  /mnt/SeanVolume/SeanDataset/ISOImage
SeanVolume/SeanDataset/Music                                51.7G  1.34T   402K  /mnt/SeanVolume/SeanDataset/Music
SeanVolume/SeanDataset/Pictures                              131G  1.34T  2.08M  /mnt/SeanVolume/SeanDataset/Pictures
SeanVolume/SeanDataset/Saved Games                          75.7G  1.34T   237K  /mnt/SeanVolume/SeanDataset/Saved Games
SeanVolume/SeanDataset/Temporary                            9.63G  1.34T   520K  /mnt/SeanVolume/SeanDataset/Temporary
SeanVolume/SeanDataset/Videos                                846G  1.34T   792G  /mnt/SeanVolume/SeanDataset/Videos
SeanVolume/jails                                            3.80G  1.34T   678M  /mnt/SeanVolume/jails
SeanVolume/jails/.warden-template-pluginjail                6.33M  1.34T   591M  /mnt/SeanVolume/jails/.warden-template-pluginjail
SeanVolume/jails/.warden-template-pluginjail-9.3             597M  1.34T   591M  /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3
SeanVolume/jails/customplugin_2                             1.17G  1.34T  1.27G  /mnt/SeanVolume/jails/customplugin_2
SeanVolume/jails/pluginjail                                 1.28G  1.34T   630M  /mnt/SeanVolume/jails/pluginjail
SeanVolume/ownCloudDataset                                  1.21T  1.34T   255G  /mnt/SeanVolume/ownCloudDataset
freenas-boot                                                 680M   107G   144K  none
freenas-boot/ROOT                                            670M   107G   144K  none
freenas-boot/ROOT/Initial-Install                              8K   107G   659M  legacy
freenas-boot/ROOT/default                                    670M   107G   665M  legacy
freenas-boot/grub                                           7.76M   107G  7.76M  legacy

2015/07/17 添加:

我使用zfs list -t snapshot命令后,它显示了许多微小的快照(总共 2,534 个文件......)。

我正在检查,并删除那些快照文件。

删除后,df -TH再次运行!


2015/07/20 添加:

我删除了许多快照文件后得到了一些空间!

但...

FreeNAS 9.3 Web GUI 崩溃了...

在我清除大量快照(约 1500 个)后,FreeNAS 9.3 Web GUI 显示“发生错误”。两天后

[root@freenas] /etc/defaults# df -TH
Filesystem                                                  Type      Size    Used   Avail Capacity  Mounted on
freenas-boot/ROOT/default                                   zfs       115G    698M    114G     1%    /
devfs                                                       devfs     1.0k    1.0k      0B   100%    /dev
tmpfs                                                       tmpfs      33M    5.5M     28M    16%    /etc
tmpfs                                                       tmpfs     4.2M    8.2k    4.2M     0%    /mnt
tmpfs                                                       tmpfs      11G     67M     11G     1%    /var
freenas-boot/grub                                           zfs       114G    8.1M    114G     0%    /boot/grub
SeanVolume                                                  zfs         4T    822k      4T     0%    /mnt/SeanVolume
SeanVolume/CandyDataset                                     zfs         4T    299k      4T     0%    /mnt/SeanVolume/CandyDataset
SeanVolume/CandyDataset/Downloads                           zfs       5.3T    1.3T      4T    25%    /mnt/SeanVolume/CandyDataset/Downloads
SeanVolume/CandyDataset/Videos                              zfs       4.2T    221G      4T     5%    /mnt/SeanVolume/CandyDataset/Videos
SeanVolume/PublicDataset                                    zfs         4T    224k      4T     0%    /mnt/SeanVolume/PublicDataset
SeanVolume/SeanDataset                                      zfs       9.6T    5.6T      4T    59%    /mnt/SeanVolume/SeanDataset
SeanVolume/jails                                            zfs         4T    710M      4T     0%    /mnt/SeanVolume/jails
SeanVolume/jails/.warden-template-pluginjail                zfs         4T    619M      4T     0%    /mnt/SeanVolume/jails/.warden-template-pluginjail
SeanVolume/jails/.warden-template-pluginjail-9.3            zfs         4T    619M      4T     0%    /mnt/SeanVolume/jails/.warden-template-pluginjail-9.3
SeanVolume/jails/pluginjail                                 zfs         4T    661M      4T     0%    /mnt/SeanVolume/jails/pluginjail
SeanVolume/ownCloudDataset                                  zfs       4.3T    336G      4T     8%    /mnt/SeanVolume/ownCloudDataset
SeanVolume/.system                                          zfs         4T    3.4M      4T     0%    /var/db/system
SeanVolume/.system/cores                                    zfs         4T    1.4M      4T     0%    /var/db/system/cores
SeanVolume/.system/samba4                                   zfs         4T    6.1M      4T     0%    /var/db/system/samba4
SeanVolume/.system/syslog-cd1fc29ce94d4a81a24df77359252261  zfs         4T     11M      4T     0%    /var/db/system/syslog-cd1fc29ce94d4a81a24df77359252261
SeanVolume/.system/rrd-cd1fc29ce94d4a81a24df77359252261     zfs         4T    224k      4T     0%    /var/db/system/rrd-cd1fc29ce94d4a81a24df77359252261
devfs                                                       devfs     1.0k    1.0k      0B   100%    /mnt/SeanVolume/jails/pluginjail/dev
procfs                                                      procfs    4.1k    4.1k      0B   100%    /mnt/SeanVolume/jails/pluginjail/proc
SeanVolume/jails/customplugin_2                             zfs         4T    1.4G      4T     0%    /mnt/SeanVolume/jails/customplugin_2
devfs                                                       devfs     1.0k    1.0k      0B   100%    /mnt/SeanVolume/jails/customplugin_2/dev
procfs                                                      procfs    4.1k    4.1k      0B   100%    /mnt/SeanVolume/jails/customplugin_2/proc
/mnt/SeanVolume/ownCloudDataset                             nullfs    4.3T    336G      4T     8%    /mnt/SeanVolume/jails/customplugin_2/media

答案1

我尝试总结给出的信息并给出一些解释(我知道的,它意味着什么):

关于zpool 列表输出:
SIZE 是所有磁盘的大小,包括用于冗余的磁盘(因此与zpool 状态这意味着,您已将 8 个 3TB 驱动器组合成 RAID-Z3 配置)
ALLOC 是已使用空间,还包括用于冗余的磁盘(因此它至少是实际数据大小的 (5+3)/5=1.6 倍,因为 8 个磁盘中有 3 个用于冗余)
Free 是可用空间,还包括用于冗余的磁盘(因此可用空间最多为所示可用大小的 5/(5+3)=0.625

zfs 列表命令说,有 1,34T 可用,zpool 列表说有 3,16T 可用,减去我预期的冗余空间大约是 1,975T。
我知道 zfs 需要一些空间用于元数据,但在我的 Linux 系统上,RAID-Z1 中有 4 个 2TB 驱动器,差异约为 6%,你这里的差异约为 47%,我说不出差异来自哪里。

关于输出df-钍
df 找不到 zfs 快照,因此无法计算它们,因此使用的空间不是 zfs 文件系统中使用的空间。对于 zfs 文件系统中分配的空间,请使用zfs 列表

删除快照可能需要一些时间,这似乎是设计使然,你可以看看http://nex7.blogspot.de/2013/03/readme1st.html

删除快照后,如果我计算您的 df 输出,您的池中大约有 7.2T 的数据,大约有 4T 的可用空间,总共 11.2T。
如果我将您的完整大小(21.8T)减去冗余磁盘,我预计大约有 5/(5+3)*21.8T=13.6T(相差约 20%)

我认为 20% 对于 zfs 所需的元数据来说有点太多了,但我无法告诉您它在哪里丢失了。也许您有许多非常小的文件,它们使用的元数据空间比没有那么多大文件要多。
很抱歉,但我对 zfs 内部结构了解不多,所以我无法告诉您从哪里获取元数据空间使用情况。
您可以再试一次zfs 列表并将那里的已用空间与您从 df 获得的已用空间进行比较。也许它给出了提示。

编辑于 2015 年 7 月 21 日
关于大小:3T 驱动器实际上有 2.7 TiB(因为驱动器制造商使用的是 1TB = 1.000.000.000.000 字节,其中使用 1TiB 的计算机 = 1*1024*1024*1024*1024=1.099.511.627.776 字节。2.7
* 8 个驱动器 = 21.8T
在正常的 RAID 系统中,最小的驱动器决定了较大驱动器的可用大小。

相关内容