前段时间我搜索了一下，发现了一些关于使用循环设备（我们没有使用）以及使用 fstrim 释放空间的内容。在阅读了一些关于 XFS 使用精简配置释放空间的潜在问题后，我最终只是重建了我的 kubernetes 工作节点以清除上次的空间并从 XFS 切换到 EXT4。不过，我们现在似乎又回到了同样的情况，我想知道发生了什么。
我怀疑设备映射器被用来创建精简卷，然后它们就被孤立了，但我无法使用 dmsetup 命令看到它们。这个问题似乎是 kubernetes 工作节点所特有的，我猜可能是 Rancher 和 Kubernetes 以及它如何与 Docker 交互？
docker_host:~ # cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.0 (Maipo)
docker_host:~ # docker info
Containers: 101
Running: 83
Paused: 0
Stopped: 18
Images: 56
Server Version: 17.03.2-ce
Storage Driver: devicemapper
Pool Name: k8s_vol1-docker_thinpool
Pool Blocksize: 131.1 kB
Base Device Size: 10.74 GB
Backing Filesystem: ext4
Data file:
Metadata file:
Data Space Used: 48.32 GB
Data Space Total: 214.7 GB
Data Space Available: 166.4 GB
Metadata Space Used: 862.3 MB
Metadata Space Total: 2.252 GB
Metadata Space Available: 1.39 GB
Thin Pool Minimum Free Space: 21.47 GB
Udev Sync Supported: true
Deferred Removal Enabled: true
Deferred Deletion Enabled: true
Deferred Deleted Device Count: 0
Library Version: 1.02.135-RHEL7 (2016-11-16)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 4ab9917febca54791c5f071a9d1f404867857fcc
runc version: 54296cf40ad8143b62dbcaa1d90e520a2136ddfe
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-514.10.2.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.0 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 62.76 GiB
Name: docker_host
ID: *******
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
###.###.###.###:####
127.0.0.0/8
Live Restore Enabled: false
docker_host:~ # lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
docker_containers k8s_vol1 -wi-ao---- 50.00g
docker_thinpool k8s_vol1 twi-aotz-- 200.00g 22.50 38.28
kubelet k8s_vol1 -wi-ao---- 10.00g
optlv k8s_vol1 -wi-ao---- 10.00g
rook_ceph k8s_vol1 -wi-ao---- 200.00g
root os -wi-ao---- 10.00g
swap os -wi-ao---- 2.00g
tmp os -wi-ao---- 2.00g
var os -wi-ao---- 5.00g
docker_host:~ # pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 os lvm2 a-- 29.47g 10.47g
/dev/sdb k8s_vol1 lvm2 a-- 1000.00g 527.80g
docker_host:~ # vgs
VG #PV #LV #SN Attr VSize VFree
k8s_vol1 1 5 0 wz--n- 1000.00g 527.80g
os 1 4 0 wz--n- 29.47g 10.47g
docker_host:~ # dmsetup ls
docker-253:3-101498017-72bcd76347e26fed9b0332842d42dab557262ef56d6abefb6faff3247efa1cf8 (253:79)
docker-253:3-101498017-cb9540890830b416bbc03da01f68a446604b55682b3e5e82740eeb37a8e8fb24 (253:12)
docker-253:3-101498017-692db6edc24c878f9caa45c89c930120e6c0b3897f62e468aa1ba56482743a8f (253:95)
docker-253:3-101498017-c31f527f16b1d4126a0b18c22b54ec1121d83f3f3a13a38f218a8956b700202e (253:74)
docker-253:3-101498017-2ca06d5355e51a41695b3a407c141d540ce3eaebfaf79d55c39e7bf5b14ec0fd (253:88)
docker-253:3-101498017-1e91e539f9a112909e7439bca13629864a2d7f1797c3cc42944dc01a6ed07001 (253:39)
docker-253:3-101498017-b72394e1b06d75059d7696e41cb92e2904dfb68c66b4d44bb7e5710c9d078734 (253:27)
docker-253:3-101498017-3bb55da27654a22e832141e1ef8f943ac22b3f06f96315033a99529b43a1fd7e (253:90)
docker-253:3-101498017-a2e755e978eeda8788df96e95eb811e40f1e747aa08b9dbdd3bd9affcbf4765f (253:14)
os-tmp (253:7)
docker-253:3-101498017-58aae6188dca08a773f7b0dd3c051ca315006054ae87a6864488750419183451 (253:59)
docker-253:3-101498017-7d23c8c042d3eca16b215cd90a244d931fa8c1f7c0d9f6354c8aba1442d185a9 (253:62)
docker-253:3-101498017-081c71e18b31c9bb8e92bba85e97768be3c9bbfc0d5fb9d0bdd0f9d5299278ef (253:76)
docker-253:3-101498017-57fe21d56db1e80144e67f29eea5d52eeb2140248f13ecdda24430f33a76b75f (253:68)
docker-253:3-101498017-9b4be0f839de894812e1918c7d46eecaff84f816d58ac17818b4b9069088081b (253:82)
docker-253:3-101498017-66c8b4df11fff26acb065b57fa1ead99cdbb91ca53af794b7d5f9fa93dc421ca (253:80)
docker-253:3-101498017-e8658773d4478b95145832dd4ce68ff8acb90e0128cf1a4cd1592b3d10b56c14 (253:40)
docker-253:3-101498017-83d87da21d2cc6717b3b9bf64f85552af572363cb75eaf6aa8a5443dc1815ede (253:16)
docker-253:3-101498017-d6f8df43759458ef8c66ee84a618e98d6b1e3f3664e69134136df5fa41b0f980 (253:46)
docker-253:3-101498017-861e4d8fd60e438a93589bf114df45fde0dde59e805287b477aef8fd30335ad0 (253:73)
docker-253:3-101498017-15a7b53ae901d70b709733315f70b3e7b77d48177bf719799436877b04661533 (253:72)
docker-253:3-101498017-82d44c0769935c8e64d28806b525f4daf2c9011fbb8566b10c7c357dee88b295 (253:70)
docker-253:3-101498017-cde50281f6d9a07d8d227d97ab1621c00dea5467dfca97f7cea8e668d6cd4195 (253:45)
docker-253:3-101498017-dc85b9e7171b469ba9f32e8a1f91671a03e2933606aa1b53edeadc3de13d4972 (253:61)
docker-253:3-101498017-7688a3352e7b0caa35b55a83c8f62fd1edb5b527bbce25bfa6374763bcc789f2 (253:60)
docker-253:3-101498017-0334a53c8f4afd225a961ca5e001cd0daed7d549027963e6b4af3601773315e6 (253:35)
docker-253:3-101498017-9a5f122b66aa832856a6cf9714517ccd2d5862fb29e71ce72af4c365f27a1520 (253:19)
docker-253:3-101498017-4a4e61a6cb0c02b3a918af44b98b554252d45cea5651acba3ec2f4e2d7d05cf0 (253:83)
docker-253:3-101498017-2d847821dc876c1cf9339e006f34562fdae9f4f5dae0219476045eada99c1b01 (253:15)
docker-253:3-101498017-c7f082591d74f0a1b2ded0e9c57ce5a76c3a92a2f119e63018b9d44aa5c5fb94 (253:65)
docker-253:3-101498017-d053d6aca55c7498de05048e17876f4081725e7c58a425950f465b6ec44702df (253:29)
os-var (253:8)
docker-253:3-101498017-e0d6285982adca5d02f380a8d34793cfee94da06543e751e6e65211a055291b4 (253:85)
docker-253:3-101498017-280657ed58b3c2b6ad2985cf810f78f0a5a1224e194357912176c0140de26701 (253:23)
k8s_vol1-optlv (253:2)
docker-253:3-101498017-94bf39e488c4de0ceb0547a49cbecc496ab7660113dafe3b0fd78bd7cb2aee51 (253:43)
docker-253:3-101498017-b16d5d13faec6ace9dee33ee736a720b265273d07514575f3e8c1815e3351035 (253:38)
docker-253:3-101498017-b9b8e8346860c24c9629241dd586e330c1948432a54f8d5f2efd114329be2dfd (253:57)
docker-253:3-101498017-a96d4b4b9aac6459b36ca93bd71092f60fc72a204fc13bd83c3efdd4b46e6f52 (253:17)
k8s_vol1-docker_containers (253:3)
docker-253:3-101498017-b8baaa0a194a40ffd4b165a1a381b84706cb6a60fd46d80306efec146e431f95 (253:77)
docker-253:3-101498017-c055e9a480a884f00333dd6a2c335da3409aefcc807ee52e6bf8f78362095bc9 (253:26)
k8s_vol1-rook_ceph (253:10)
docker-253:3-101498017-9ce68f7c91a3ed26264668f3de3ae558975a92b9f8d6233a82c7bff784d21ccb (253:49)
docker-253:3-101498017-c6d44daa361bc0439bba8346b193b04c4c8252e600ed722de7b7aa8cdc0b28d2 (253:63)
docker-253:3-101498017-5ce0e9df887903b61491b9ca51b49ebdd1cfa790ae010606d6098049683a86f3 (253:71)
docker-253:3-101498017-01e0ff2289f9d72c93d6cf94187d7682bec99e3a02e4cdc4916f6d1d4cbdf5dc (253:41)
docker-253:3-101498017-8a552cf69c2d25746795a0e5a3e9535c0c926246c865052252be88bd23980490 (253:92)
docker-253:3-101498017-9c74c5623eb36c4ffa7ced7ceb838ba227cd5b434a9ae06e3131a9c90c7a238c (253:22)
docker-253:3-101498017-19f400317e661c19ddd8ee6a6d7f25eeb48e54c67ed8c31667babbff70486899 (253:67)
docker-253:3-101498017-c7e0ad7332c64312fb79d4a3aebead2c842d5606f90caef786ba0ca6fd61d669 (253:54)
docker-253:3-101498017-aea189e064aa1820ce56a3155d5c949dca68008d22582c533b357654329d8a31 (253:66)
docker-253:3-101498017-26273229d473b48898bc0426694ade5010f36ddab8948529fbbdc7f5acc5841d (253:36)
docker-253:3-101498017-1cee6b102ce6671c0a44e1350c562208e1af3ea3e2b7d58b6967dffca32265d2 (253:37)
docker-253:3-101498017-f56c67d4d326547eb35a6c8e9170f74d23d137371312e53c78fef27abe6aba61 (253:24)
docker-253:3-101498017-9d3379bcb8edbb38e0db85fb8a54e46db953c2fa6fc5ebca18e2a4c3f6d47e84 (253:47)
docker-253:3-101498017-456b66ba8079c962fb9c6239d33440984d86a02ea6bfdc6718597a70307b529f (253:32)
k8s_vol1-docker_thinpool_tdata (253:6)
docker-253:3-101498017-5aec78cb52a7e922839edcb7b6307a5f2d8d1847fc66061fd0c3a84cc2330e0e (253:48)
os-swap (253:0)
docker-253:3-101498017-f12784005d2ba8707dfcbb8424921ad2721698079253a3121289dd39689de282 (253:87)
docker-253:3-101498017-c0663ce2cafe4a458da70bbc96b5068c8f62989a4e53f289874df2f724057e81 (253:69)
docker-253:3-101498017-7cf9e87a2f613f0c7d7ca8c827986091e8b8c8132c7f1326a6f3d30b385d1d17 (253:33)
os-root (253:1)
docker-253:3-101498017-54df327f48ba863ffd70b1c249aa7da4d913fa317af99af971a63439e8efe139 (253:13)
docker-253:3-101498017-458803691c98537e1c92371a1fa956cf968f791f2bf51bff7678595c5160ba4f (253:64)
docker-253:3-101498017-d07e3bfe5c48623aceb7625c9e2d66ddfe523a06c303801ab4c29f6c33c68cdd (253:50)
docker-253:3-101498017-d9e37c4a49eba4d82e22dd07f6317d3d5714037f19e2aa3f15ab86c0162c8ed2 (253:21)
k8s_vol1-docker_thinpool_tmeta (253:5)
k8s_vol1-docker_thinpool (253:9)
docker-253:3-101498017-3c8c62d635216ac3ff6421ccb8d4dc835f399a08ec6b660b3c8edbd5503e2903 (253:42)
k8s_vol1-kubelet (253:4)
docker-253:3-101498017-2d05d9275ecf718697c59aba19f67a3347fe4395dc37d89dd323d2889652d7ed (253:58)
docker-253:3-101498017-cb5813d7f1064e9b26fec5d91e1665205927769841df552bf1b72e65320d22ff (253:55)
docker-253:3-101498017-f697572e0551ec13f1c09d5e7a6e163b861ee08251a7e4d4e42fd4c589e418b2 (253:78)
docker-253:3-101498017-a61b78adab88a851dc75bb73daa289b8127d608e8f934b75f7689b2bac3924de (253:31)
docker-253:3-101498017-e36d383f3d2aa4e02c57374bbac54c9b998a22d2ac2cfc4bbbe2092b914cc05f (253:11)
docker-253:3-101498017-87f74f834c7eb9aed8b1d781638a375107237e9e0d32ad5ad1e22297a1ac6c4c (253:25)
docker-253:3-101498017-2fdea3314ebf45424145f8c67458de28ffb8e512919c2d4429a14be9a37e64f5 (253:18)
docker-253:3-101498017-cbf7504c9b084a6942c28c4944669b0e7d549cad9fb9b79b8d12e1a87c030a35 (253:75)
docker-253:3-101498017-e88e9db7e1fd3706918f7269ee38f62b7a9a5b12318991dbd19299cf2630d736 (253:89)
docker-253:3-101498017-a09c944b5dadb6aff47b31099cc0884af9996d0526403d9bbc8c86823051aaff (253:51)
docker-253:3-101498017-3c7a5d24808362d90d1b22074235943378b4488122733453e369e4509e5ee2a2 (253:20)
docker-253:3-101498017-4e1f157bba6ead05baa86d72fb8f7e90705090db89196cb84d1dab916c369a47 (253:84)
docker-253:3-101498017-1f7cb658dbbb2ab9635eb2dd6c29219fb84a659d60bb5fef6fdee7977b658cd7 (253:30)
docker-253:3-101498017-7b5a238eebb203422ab5daa6a8706950337124e2ce0bcc622b613231ed03f224 (253:34)
docker-253:3-101498017-8e564cde63bdb70ea017b2cef69b68cfa10e22ddea0451229fd1bb7ab67d05fa (253:28)
docker-253:3-101498017-7d0a3e631d18fc56e7d80b3132786c46cd2cf4efc24ea6ff367e17e208474da2 (253:81)
docker-253:3-101498017-3c897c44a0b7287dc91afcdf4e913dec6084e8816b7a0c2b1dd3bc832cf116f7 (253:94)
docker-253:3-101498017-7e8da0104cc1b0434c6d95cfdbb68a89f2c7a9ec8502467613bf1e61052de6f5 (253:56)
docker-253:3-101498017-9b3b2b78d0ed36d8c382520cb983f179777e936212e90b87edcec88a8adf91b9 (253:52)
docker-253:3-101498017-661cacf156942f9ffe9018140ceca192df12d33628b9d483a9a0439fe2e2402d (253:86)
docker-253:3-101498017-547a6a68db282fabcbf4377151b4d3df076a4b8497012d36e217187fcd6eaf57 (253:53)
docker-253:3-101498017-0e536306352fea1c007f71e8e87703ab79eefcd1ea8330b7b8e9f4ce6fc5d0ce (253:44)
docker_host:~ # df | grep dm- | wc -l
83
docker_host:~ # dmsetup ls | wc -l
94
docker_host:~ # dmsetup ls | grep docker | wc -l
87
docker_host:~ # df | grep dm-
/dev/dm-12 10190100 1129120 8520308 12% /var/lib/docker/devicemapper/mnt/cb9540890830b416bbc03da01f68a446604b55682b3e5e82740eeb37a8e8fb24
/dev/dm-13 10190100 1129448 8519980 12% /var/lib/docker/devicemapper/mnt/54df327f48ba863ffd70b1c249aa7da4d913fa317af99af971a63439e8efe139
/dev/dm-14 10190100 165092 9484336 2% /var/lib/docker/devicemapper/mnt/a2e755e978eeda8788df96e95eb811e40f1e747aa08b9dbdd3bd9affcbf4765f
/dev/dm-15 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/2d847821dc876c1cf9339e006f34562fdae9f4f5dae0219476045eada99c1b01
/dev/dm-17 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/a96d4b4b9aac6459b36ca93bd71092f60fc72a204fc13bd83c3efdd4b46e6f52
/dev/dm-18 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/2fdea3314ebf45424145f8c67458de28ffb8e512919c2d4429a14be9a37e64f5
/dev/dm-19 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/9a5f122b66aa832856a6cf9714517ccd2d5862fb29e71ce72af4c365f27a1520
/dev/dm-20 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/3c7a5d24808362d90d1b22074235943378b4488122733453e369e4509e5ee2a2
/dev/dm-21 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/d9e37c4a49eba4d82e22dd07f6317d3d5714037f19e2aa3f15ab86c0162c8ed2
/dev/dm-24 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/f56c67d4d326547eb35a6c8e9170f74d23d137371312e53c78fef27abe6aba61
/dev/dm-22 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/9c74c5623eb36c4ffa7ced7ceb838ba227cd5b434a9ae06e3131a9c90c7a238c
/dev/dm-25 10190100 319768 9329660 4% /var/lib/docker/devicemapper/mnt/87f74f834c7eb9aed8b1d781638a375107237e9e0d32ad5ad1e22297a1ac6c4c
/dev/dm-23 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/280657ed58b3c2b6ad2985cf810f78f0a5a1224e194357912176c0140de26701
/dev/dm-27 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/b72394e1b06d75059d7696e41cb92e2904dfb68c66b4d44bb7e5710c9d078734
/dev/dm-29 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/d053d6aca55c7498de05048e17876f4081725e7c58a425950f465b6ec44702df
/dev/dm-28 10190100 143968 9505460 2% /var/lib/docker/devicemapper/mnt/8e564cde63bdb70ea017b2cef69b68cfa10e22ddea0451229fd1bb7ab67d05fa
/dev/dm-26 10190100 281632 9367796 3% /var/lib/docker/devicemapper/mnt/c055e9a480a884f00333dd6a2c335da3409aefcc807ee52e6bf8f78362095bc9
/dev/dm-11 10190100 632460 9016968 7% /var/lib/docker/devicemapper/mnt/e36d383f3d2aa4e02c57374bbac54c9b998a22d2ac2cfc4bbbe2092b914cc05f
/dev/dm-16 10190100 64968 9584460 1% /var/lib/docker/devicemapper/mnt/83d87da21d2cc6717b3b9bf64f85552af572363cb75eaf6aa8a5443dc1815ede
/dev/dm-30 10190100 61484 9587944 1% /var/lib/docker/devicemapper/mnt/1f7cb658dbbb2ab9635eb2dd6c29219fb84a659d60bb5fef6fdee7977b658cd7
/dev/dm-31 10190100 546804 9102624 6% /var/lib/docker/devicemapper/mnt/a61b78adab88a851dc75bb73daa289b8127d608e8f934b75f7689b2bac3924de
/dev/dm-33 10190100 319768 9329660 4% /var/lib/docker/devicemapper/mnt/7cf9e87a2f613f0c7d7ca8c827986091e8b8c8132c7f1326a6f3d30b385d1d17
/dev/dm-34 10190100 104316 9545112 2% /var/lib/docker/devicemapper/mnt/7b5a238eebb203422ab5daa6a8706950337124e2ce0bcc622b613231ed03f224
/dev/dm-35 10190100 82192 9567236 1% /var/lib/docker/devicemapper/mnt/0334a53c8f4afd225a961ca5e001cd0daed7d549027963e6b4af3601773315e6
/dev/dm-38 10190100 536900 9112528 6% /var/lib/docker/devicemapper/mnt/b16d5d13faec6ace9dee33ee736a720b265273d07514575f3e8c1815e3351035
/dev/dm-37 10190100 536900 9112528 6% /var/lib/docker/devicemapper/mnt/1cee6b102ce6671c0a44e1350c562208e1af3ea3e2b7d58b6967dffca32265d2
/dev/dm-32 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/456b66ba8079c962fb9c6239d33440984d86a02ea6bfdc6718597a70307b529f
/dev/dm-36 10190100 632460 9016968 7% /var/lib/docker/devicemapper/mnt/26273229d473b48898bc0426694ade5010f36ddab8948529fbbdc7f5acc5841d
/dev/dm-41 10190100 536904 9112524 6% /var/lib/docker/devicemapper/mnt/01e0ff2289f9d72c93d6cf94187d7682bec99e3a02e4cdc4916f6d1d4cbdf5dc
/dev/dm-40 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/e8658773d4478b95145832dd4ce68ff8acb90e0128cf1a4cd1592b3d10b56c14
/dev/dm-42 10190100 536908 9112520 6% /var/lib/docker/devicemapper/mnt/3c8c62d635216ac3ff6421ccb8d4dc835f399a08ec6b660b3c8edbd5503e2903
/dev/dm-39 10190100 89256 9560172 1% /var/lib/docker/devicemapper/mnt/1e91e539f9a112909e7439bca13629864a2d7f1797c3cc42944dc01a6ed07001
/dev/dm-44 10190100 89256 9560172 1% /var/lib/docker/devicemapper/mnt/0e536306352fea1c007f71e8e87703ab79eefcd1ea8330b7b8e9f4ce6fc5d0ce
/dev/dm-43 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/94bf39e488c4de0ceb0547a49cbecc496ab7660113dafe3b0fd78bd7cb2aee51
/dev/dm-46 10190100 141136 9508292 2% /var/lib/docker/devicemapper/mnt/d6f8df43759458ef8c66ee84a618e98d6b1e3f3664e69134136df5fa41b0f980
/dev/dm-55 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/cb5813d7f1064e9b26fec5d91e1665205927769841df552bf1b72e65320d22ff
/dev/dm-56 10190100 85836 9563592 1% /var/lib/docker/devicemapper/mnt/7e8da0104cc1b0434c6d95cfdbb68a89f2c7a9ec8502467613bf1e61052de6f5
/dev/dm-57 10190100 76872 9572556 1% /var/lib/docker/devicemapper/mnt/b9b8e8346860c24c9629241dd586e330c1948432a54f8d5f2efd114329be2dfd
/dev/dm-54 10190100 78100 9571328 1% /var/lib/docker/devicemapper/mnt/c7e0ad7332c64312fb79d4a3aebead2c842d5606f90caef786ba0ca6fd61d669
/dev/dm-53 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/547a6a68db282fabcbf4377151b4d3df076a4b8497012d36e217187fcd6eaf57
/dev/dm-52 10190100 287272 9362156 3% /var/lib/docker/devicemapper/mnt/9b3b2b78d0ed36d8c382520cb983f179777e936212e90b87edcec88a8adf91b9
/dev/dm-50 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/d07e3bfe5c48623aceb7625c9e2d66ddfe523a06c303801ab4c29f6c33c68cdd
/dev/dm-51 10190100 1019208 8630220 11% /var/lib/docker/devicemapper/mnt/a09c944b5dadb6aff47b31099cc0884af9996d0526403d9bbc8c86823051aaff
/dev/dm-58 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/2d05d9275ecf718697c59aba19f67a3347fe4395dc37d89dd323d2889652d7ed
/dev/dm-59 10190100 177592 9471836 2% /var/lib/docker/devicemapper/mnt/58aae6188dca08a773f7b0dd3c051ca315006054ae87a6864488750419183451
/dev/dm-60 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/7688a3352e7b0caa35b55a83c8f62fd1edb5b527bbce25bfa6374763bcc789f2
/dev/dm-61 10190100 73788 9575640 1% /var/lib/docker/devicemapper/mnt/dc85b9e7171b469ba9f32e8a1f91671a03e2933606aa1b53edeadc3de13d4972
/dev/dm-63 10190100 141160 9508268 2% /var/lib/docker/devicemapper/mnt/c6d44daa361bc0439bba8346b193b04c4c8252e600ed722de7b7aa8cdc0b28d2
/dev/dm-68 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/57fe21d56db1e80144e67f29eea5d52eeb2140248f13ecdda24430f33a76b75f
/dev/dm-69 10190100 598648 9050780 7% /var/lib/docker/devicemapper/mnt/c0663ce2cafe4a458da70bbc96b5068c8f62989a4e53f289874df2f724057e81
/dev/dm-45 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/cde50281f6d9a07d8d227d97ab1621c00dea5467dfca97f7cea8e668d6cd4195
/dev/dm-48 10190100 205664 9443764 3% /var/lib/docker/devicemapper/mnt/5aec78cb52a7e922839edcb7b6307a5f2d8d1847fc66061fd0c3a84cc2330e0e
/dev/dm-67 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/19f400317e661c19ddd8ee6a6d7f25eeb48e54c67ed8c31667babbff70486899
/dev/dm-74 10190100 601884 9047544 7% /var/lib/docker/devicemapper/mnt/c31f527f16b1d4126a0b18c22b54ec1121d83f3f3a13a38f218a8956b700202e
/dev/dm-64 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/458803691c98537e1c92371a1fa956cf968f791f2bf51bff7678595c5160ba4f
/dev/dm-65 10190100 954536 8694892 10% /var/lib/docker/devicemapper/mnt/c7f082591d74f0a1b2ded0e9c57ce5a76c3a92a2f119e63018b9d44aa5c5fb94
/dev/dm-75 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/cbf7504c9b084a6942c28c4944669b0e7d549cad9fb9b79b8d12e1a87c030a35
/dev/dm-76 10190100 439296 9210132 5% /var/lib/docker/devicemapper/mnt/081c71e18b31c9bb8e92bba85e97768be3c9bbfc0d5fb9d0bdd0f9d5299278ef
/dev/dm-66 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/aea189e064aa1820ce56a3155d5c949dca68008d22582c533b357654329d8a31
/dev/dm-62 10190100 386920 9262508 5% /var/lib/docker/devicemapper/mnt/7d23c8c042d3eca16b215cd90a244d931fa8c1f7c0d9f6354c8aba1442d185a9
/dev/dm-47 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/9d3379bcb8edbb38e0db85fb8a54e46db953c2fa6fc5ebca18e2a4c3f6d47e84
/dev/dm-49 10190100 681736 8967692 8% /var/lib/docker/devicemapper/mnt/9ce68f7c91a3ed26264668f3de3ae558975a92b9f8d6233a82c7bff784d21ccb
/dev/dm-94 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/3c897c44a0b7287dc91afcdf4e913dec6084e8816b7a0c2b1dd3bc832cf116f7
/dev/dm-95 10190100 112708 9536720 2% /var/lib/docker/devicemapper/mnt/692db6edc24c878f9caa45c89c930120e6c0b3897f62e468aa1ba56482743a8f
/dev/dm-70 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/82d44c0769935c8e64d28806b525f4daf2c9011fbb8566b10c7c357dee88b295
/dev/dm-72 10190100 430968 9218460 5% /var/lib/docker/devicemapper/mnt/15a7b53ae901d70b709733315f70b3e7b77d48177bf719799436877b04661533
/dev/dm-79 10190100 319764 9329664 4% /var/lib/docker/devicemapper/mnt/72bcd76347e26fed9b0332842d42dab557262ef56d6abefb6faff3247efa1cf8
/dev/dm-78 10190100 319764 9329664 4% /var/lib/docker/devicemapper/mnt/f697572e0551ec13f1c09d5e7a6e163b861ee08251a7e4d4e42fd4c589e418b2
/dev/dm-77 10190100 319764 9329664 4% /var/lib/docker/devicemapper/mnt/b8baaa0a194a40ffd4b165a1a381b84706cb6a60fd46d80306efec146e431f95
/dev/dm-73 10190100 319764 9329664 4% /var/lib/docker/devicemapper/mnt/861e4d8fd60e438a93589bf114df45fde0dde59e805287b477aef8fd30335ad0
/dev/dm-71 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/5ce0e9df887903b61491b9ca51b49ebdd1cfa790ae010606d6098049683a86f3
/dev/dm-80 10190100 879836 8769592 10% /var/lib/docker/devicemapper/mnt/66c8b4df11fff26acb065b57fa1ead99cdbb91ca53af794b7d5f9fa93dc421ca
/dev/dm-86 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/661cacf156942f9ffe9018140ceca192df12d33628b9d483a9a0439fe2e2402d
/dev/dm-89 10190100 285304 9364124 3% /var/lib/docker/devicemapper/mnt/e88e9db7e1fd3706918f7269ee38f62b7a9a5b12318991dbd19299cf2630d736
/dev/dm-84 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/4e1f157bba6ead05baa86d72fb8f7e90705090db89196cb84d1dab916c369a47
/dev/dm-85 10190100 1019240 8630188 11% /var/lib/docker/devicemapper/mnt/e0d6285982adca5d02f380a8d34793cfee94da06543e751e6e65211a055291b4
/dev/dm-81 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/7d0a3e631d18fc56e7d80b3132786c46cd2cf4efc24ea6ff367e17e208474da2
/dev/dm-82 10190100 523004 9126424 6% /var/lib/docker/devicemapper/mnt/9b4be0f839de894812e1918c7d46eecaff84f816d58ac17818b4b9069088081b
/dev/dm-83 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/4a4e61a6cb0c02b3a918af44b98b554252d45cea5651acba3ec2f4e2d7d05cf0
/dev/dm-87 10190100 1329112 8320316 14% /var/lib/docker/devicemapper/mnt/f12784005d2ba8707dfcbb8424921ad2721698079253a3121289dd39689de282
/dev/dm-88 10190100 37648 9611780 1% /var/lib/docker/devicemapper/mnt/2ca06d5355e51a41695b3a407c141d540ce3eaebfaf79d55c39e7bf5b14ec0fd
/dev/dm-90 10190100 417500 9231928 5% /var/lib/docker/devicemapper/mnt/3bb55da27654a22e832141e1ef8f943ac22b3f06f96315033a99529b43a1fd7e
/dev/dm-92 10190100 319764 9329664 4% /var/lib/docker/devicemapper/mnt/8a552cf69c2d25746795a0e5a3e9535c0c926246c865052252be88bd23980490
docker_host:~ # total=0; for s in `df | grep dm- | awk '{print $3}'`; do total=$(expr $total + $s); done; echo $(expr $total / 1024)
21255
docker_host:~ # docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 43 41 11.58 GB 450.3 MB (3%)
Containers 101 83 1.665 GB 83 B (0%)
Local Volumes 7 7 124.9 MB 0 B (0%)
docker_host:~ # cat /etc/docker/
daemon.json key.json
docker_host:~ # cat /etc/docker/daemon.json
{
"storage-driver": "devicemapper",
"storage-opts": [
"dm.thinpooldev=/dev/mapper/k8s_vol1-docker_thinpool",
"dm.use_deferred_removal=true",
"dm.use_deferred_deletion=true",
"dm.fs=ext4"
],
"insecure-registries": ["###.###.###.###:####"]
}