CEPH 卷已满,很难了解数据的使用位置

CEPH 卷已满,很难了解数据的使用位置

我们使用 CEPH 作为 OpenStack 安装的后端。

自两周以来,我们发现我们的卷几乎(现在)已满:

$ ceph df
GLOBAL:
    SIZE       AVAIL      RAW USED     %RAW USED
    84887G     34052G       50834G         59.88
POOLS:
    NAME                    ID     USED       %USED      MAX AVAIL     OBJECTS
    .rgw.root               1        1113     100.00             0           4
    images                  2        229G     100.00             0       38486
    volumes                 3      15668G     100.00             0     4021432
    backups                 4          19     100.00             0           2
    vms                     5        841G     100.00             0      218124
    default.rgw.control     6           0          0             0           8
    default.rgw.meta        7           0          0             0           0
    default.rgw.log         8           0          0             0         207
    volumes-SSD             9        235G      35.15          434G       60442

我发现我们无法使用所有的磁盘,因为我们有磁盘很少的服务器和磁盘很多的服务器,所以 Ceph 无法像希望的那样均匀分布(我们有 3 个副本)。

3次的总和POOLS USED大致相同RAW USED(因为有3个副本)。

现在我尝试深入挖掘这个巨大的volumes池子来查看哪一个(些)是贪婪的:

rbd du -p volumes
NAME                                 PROVISIONED   USED
01d01dc5-3a6f-4618-85f2-b9eaf5b09c16      20480M  2624M
0211d80a-2b6b-4723-a6b6-64c6ddfd6fd3      61440M 11332M
035c8313-cc4f-4464-aa11-3980ba565210      61440M 10708M
0362cad8-4ecb-4861-a172-cb3ea5c8e669      40960M  3320M
049ad9c7-f19b-4316-a19a-9c4b70ad67f5      81920M 16948M
06760df4-1832-4203-89d1-2b1cd102a1c4       5120M  1560M
07a6e824-6a32-478c-b216-c869b3aa14b7      61440M 34596M
08ec14e2-3128-43a1-8b94-dd129aedaff6      61440M 31700M
0b63ef53-d9a8-4d8f-80c8-7e991ba9f56b      61440M  8116M
1128e783-1781-49cd-9094-d503c02d1d4e      61440M 17328M
1b1cc7e8-b696-448d-a7b6-9171b45f2ddd       1024M 57344k
1cf1d860-6f31-4914-ba47-081e6c93dae6      81920M 15372M
1f48e6bc-8c14-4032-a054-e67b31fb4a1d      61440M 28796M
1f6c62f2-d475-420a-b92d-341218b2c7c5       5120M  1104M
22e087ec-7bff-47a7-9a7e-cb641ff5f1ed      61440M  8204M
23a6a911-14c8-4110-b539-74c0ce243b72      61440M  8092M
25c292e5-87cd-41c5-baab-566c2dda2a15      61440M  6556M
28fd8021-4c9e-4879-ad5c-b1532e6520bb      40960M  1660M
2f8c06d1-0055-4ee8-ac35-15febac7f8d6       5120M  1088M
32180db8-7eff-42f4-9f43-3c43a7ae913c      40960M  4356M
333df83d-ac47-4562-b8ff-c0e9d56a95a1      61440M 11696M
3465e12c-f52a-4ca6-835b-414b154ba419      81920M 12644M
35d6422c-9ba1-465c-b608-4ca4ecfe8b4f       1024M 61440k
36e4b104-6b98-4167-b7a1-036e597493f3      61440M  9304M
37520afd-2668-47fa-a45c-e6600106769f       1024M 57344k
3b218839-eb0b-43dd-9c65-f2b66586e421      61440M 27716M
3d5cf531-5b07-4b06-bad5-82a31a794a73      61440M 34696M
3fcccb51-bc38-4355-ab50-cc35ed73920e      61440M 11580M
418a5a71-1413-4504-8064-bb2085409a41      61440M 12016M
4196ac78-6a1f-4413-bced-66d041fc2e1f      40960M 40904M
41a1d494-8de8-47d2-83d0-c39b3d6f6b74      81920M  1276M
422d3105-eb07-47e4-ab4b-3ccea7b2e55d      61440M  7172M
4371ca6c-3c02-4b17-89d5-a2f327a5cd9a      40960M      0
43bd81da-7b51-47e4-899e-9fc1dc8f1ab2       5120M  1104M
4431ff9e-d710-46df-b454-5dc80ca21ade      40960M 40960M
45b0759f-fe7c-40d5-9f39-360806a60c06      61440M 11944M
473ad663-8d1a-4f67-b7c7-e7a5b739729a      40960M 19596M
4bbb9b2e-e298-49a8-978e-4f3986395b9c      61440M 12988M
5173177a-322e-4d56-ac4f-b04acd190285      61440M 13068M
56899172-13b8-4d87-b372-c7237ecb8b23       1024M 57344k
56fe0f14-e75d-44ac-82bc-d2c8c72195c5      61440M 40736M
58f28005-081a-4046-b723-7fd69a2cf84f       1024M 61440k
644f97db-6cd4-4855-83c2-95f914be69ea        500G   378G
6a12916d-f707-41aa-8b29-67abde7f4845      61440M  9304M
6b7e2d3d-2354-456b-8e69-b0d5d08acb45      61440M 11328M
6ce54dae-162e-415f-a27d-cfb900c9b30e      81920M 33492M
6d03ad58-7db8-48b1-96f3-70d408579a8a      61440M 41556M
6d076e6c-2cae-4796-8a4f-42920c94ddfc      61440M 12052M
6e96a285-5191-4fdc-a181-e98df04c2722      61440M 13920M
7065dacc-f4af-4a4c-8429-82a3c06d9bf8      61440M 12052M
73d135f1-83a2-45fd-9856-7b16dadd7ac9     102400M 19940M
73d198f1-bee2-416d-9826-81e7340393c1     102400M 43908M
746d2101-84ab-4afb-93bf-23000b1c24f7       5120M  2052M
7c2f605d-dd85-4426-a843-287cab2eb233       1024M 57344k
7f935336-25c7-449b-8e31-b6f9cf8de1fe      61440M  7684M
82cf1bae-83ba-4183-9fd5-be6fe5fdbca0      61440M  8104M
8666a7c9-123b-4ebd-aafe-a262e542069d      81920M 11484M
87876fc1-9be4-44fd-8b8e-b9a413b4ea5e      10240M 69632k
8a108ce7-ecf1-4d39-a65a-74f72f30ef36       5120M   996M
8ba11bdb-85a0-46ac-8358-1f0f0cda4900      81920M  1192M
8c10cc4b-9ceb-4a22-930c-17d88a4577e7      61440M  9808M
8cad9cc2-80e5-42f1-a146-b93fe7fb544a       5120M  1112M
8d11685e-2cf4-477d-8d27-b595e73dade6      61440M  8072M
91b3cd3c-e26b-4b9c-84dc-26bbf2c1ef5f      61440M 37808M
96441ac6-cd7f-4a3e-b42f-97bcb0c1e2db       1024M 57344k
9919e429-ea9e-4447-b0e5-506c9453c1a7     102400M      0
99e71d07-2da4-4cf2-b12d-e1305be15659      61440M 12956M
9ed969a8-7b42-486f-8de6-1205f393c8ab      61440M 11064M
a139ea0f-b7bd-4148-a8b7-80f77497838e      40960M      0
a2b46610-1919-4ac0-a8e0-e76b7ce1221f       5120M   996M
a41cce51-99c4-4ccd-948c-ceefd2bbbc1a      81920M 27236M
a4d51863-d4b3-487d-87eb-37bcb71719c7      61440M 32988M
a601b2b6-b413-423d-9210-4afae5115d1f       1024M 57344k
a803d575-6539-4f9f-8ce8-d2058602a6bf      40960M 24660M
a989949e-9377-4929-b4cf-961779c76ca8      81920M 19156M
a9e37a84-4ae5-47b9-9c81-34a34a411c48      81920M 25104M
aa2e258a-8d49-4cbe-ab00-545b3c564b83       1024M 61440k
aaff1b72-c59a-478a-85be-da412630765b      10240M 69632k
afd36bea-8dbb-407e-a14a-0db6f9189b4b       2048M      0
b0ae7c4e-1ca4-4015-a461-d4d43feafff5      81920M 14444M
b4109743-3e11-4938-9e99-b3c86161a7dd      81920M 13880M
b421adae-53e5-48f9-9b9d-d7ecaf6de8b7      61440M 11092M
b57d1fe9-7604-4945-b7c3-39c6221388bc      61440M  8384M
bb72bc84-e521-4c92-93a2-2f5646ff7f38      61440M 27564M
bc802e71-a5b0-4ba7-aa15-237685bd3dd8      40960M      0
becca2bd-4a01-4c89-889c-d28c803ff0bd      10240M 69632k
c399ab72-f783-4b4d-b484-b50ed59c4509       5120M  1044M
c3b010a5-3861-40c9-bb93-92d18ac6d6b2      61440M  9140M
c47666fc-952f-4ee3-ab81-ddfd235be4af       1024M 57344k
c4e4867a-83b2-4df4-acaa-f5ef25889d32      61440M 15668M
c65bf903-7e2c-4f05-91bf-fe995075b017      61440M  9560M
cc6cdd4d-b515-4dc6-ab73-636911953e0d       5120M  1636M
d268485d-b7fb-49b5-9ff6-e889e86938c0      61440M  9696M
d5a13ee4-65c9-440d-8656-96820d99ec78      61440M 36332M
d88349c6-ae0f-4229-8324-9cadca9178d3      61440M 11724M
d8f38e2e-f84e-4f7b-acde-043b1e7bd492       5120M  1072M
d906b45e-b596-4501-92d6-c859d481f20e      61440M 11100M
dadc9e91-1c23-443b-bd0e-fa58cee61863      40960M 40960M
ddc18e8c-11cf-4d5b-a632-81f93afc8bc1      40960M      0
e051dd8d-e91b-4f38-85af-c83ca0c3da79      40960M 31756M
e16d18fb-c79d-49db-8145-4680cf0924c6      61440M  3376M
e2631386-ca49-4be0-b650-3e9cff9f4a20      81920M 13132M
e733a6e7-373f-4109-9330-531eee5b029c       1024M 57344k
ea386235-ee6c-4061-87c0-afdf9a70aecd      81920M 13580M
f0f43900-9db5-453c-b0e9-ed7668b66bf4      61440M 12580M
f49a5c36-992d-4cb6-b95c-20288e6f84dd       5120M  1544M
f7998111-8a8e-4c9b-a0e7-d75de7a7571d      61440M 35464M
f9de8283-7c10-459b-a42f-6858763e4493       1024M 57344k
faeb9623-4911-4c3d-ad01-5c4c2add3352      40960M 32924M
<TOTAL>                                    5484G  1678G

这里du告诉我总使用量1678G5484G配置。

那么我怎样才能1678G一方面有用途,15668G另一方面又有用途呢?

相关内容