我注意到一个日志文件的大小比 inode 中提到的要小。
$ ls -lhtr /var/log/test/fk-3p-kafka/kafkaServer-gc.log
-rw-r--r-- 1 fk-3p-kafka fk-3p 11G Apr 10 18:27 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
$ du -sh /var/log/test/fk-3p-kafka/kafkaServer-gc.log
2.4G /var/log/test/fk-3p-kafka/kafkaServer-gc.log
$# stat /var/log/test/fk-3p-kafka/kafkaServer-gc.log
File: `/var/log/test/fk-3p-kafka/kafkaServer-gc.log'
Size: 11038186491 Blocks: 4823840 IO Block: 4096 regular file
Device: fe01h/65025d Inode: 132380 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 7100/fk-3p-kafka) Gid: ( 7100/ fk-3p)
Access: 2016-03-09 22:47:06.822168512 +0530
Modify: 2016-04-10 18:28:11.022285512 +0530
Change: 2016-04-10 18:28:11.022285512 +0530
Birth: -
我所做的下一步是检查是否有任何已删除的文件处理程序。
$ lsof | grep kafkaServer-gc.log
java 22252 fk-3p-kafka 3w REG 254,1 11038227512 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 347 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 712 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 764 fk-3p-kafka 3w REG 254,1 11038224580 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 2538 fk-3p-kafka 3w REG 254,1 11038224580 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 3241 fk-3p-kafka 3w REG 254,1 11038224580 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 3498 fk-3p-kafka 3w REG 254,1 11038224580 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 4146 fk-3p-kafka 3w REG 254,1 11038224580 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 4147 fk-3p-kafka 3w REG 254,1 11038224580 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 4148 fk-3p-kafka 3w REG 254,1 11038224580 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 6935 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 7650 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 9530 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 9581 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 10816 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 11887 fk-3p-kafka 3w REG 254,1 11038227512 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 13732 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22255 fk-3p-kafka 3w REG 254,1 11038204079 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22256 fk-3p-kafka 3w REG 254,1 11038204079 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22257 fk-3p-kafka 3w REG 254,1 11038204079 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22258 fk-3p-kafka 3w REG 254,1 11038204079 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22259 fk-3p-kafka 3w REG 254,1 11038204079 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22260 fk-3p-kafka 3w REG 254,1 11038204079 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22261 fk-3p-kafka 3w REG 254,1 11038205544 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22262 fk-3p-kafka 3w REG 254,1 11038205544 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22263 fk-3p-kafka 3w REG 254,1 11038205544 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22264 fk-3p-kafka 3w REG 254,1 11038205544 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22265 fk-3p-kafka 3w REG 254,1 11038205544 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22266 fk-3p-kafka 3w REG 254,1 11038205544 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22267 fk-3p-kafka 3w REG 254,1 11038207009 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22268 fk-3p-kafka 3w REG 254,1 11038207009 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22269 fk-3p-kafka 3w REG 254,1 11038207009 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22270 fk-3p-kafka 3w REG 254,1 11038207009 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22271 fk-3p-kafka 3w REG 254,1 11038207009 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22272 fk-3p-kafka 3w REG 254,1 11038207009 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22273 fk-3p-kafka 3w REG 254,1 11038207009 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22274 fk-3p-kafka 3w REG 254,1 11038207009 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22275 fk-3p-kafka 3w REG 254,1 11038208473 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22276 fk-3p-kafka 3w REG 254,1 11038208473 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22277 fk-3p-kafka 3w REG 254,1 11038208473 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22278 fk-3p-kafka 3w REG 254,1 11038208473 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22279 fk-3p-kafka 3w REG 254,1 11038208473 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22280 fk-3p-kafka 3w REG 254,1 11038208473 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22281 fk-3p-kafka 3w REG 254,1 11038208473 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22282 fk-3p-kafka 3w REG 254,1 11038208473 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22283 fk-3p-kafka 3w REG 254,1 11038209937 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22284 fk-3p-kafka 3w REG 254,1 11038209937 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22285 fk-3p-kafka 3w REG 254,1 11038209937 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22286 fk-3p-kafka 3w REG 254,1 11038209937 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22287 fk-3p-kafka 3w REG 254,1 11038209937 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22288 fk-3p-kafka 3w REG 254,1 11038209937 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22289 fk-3p-kafka 3w REG 254,1 11038209937 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22290 fk-3p-kafka 3w REG 254,1 11038211401 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22291 fk-3p-kafka 3w REG 254,1 11038211401 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22292 fk-3p-kafka 3w REG 254,1 11038211401 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22293 fk-3p-kafka 3w REG 254,1 11038211401 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22294 fk-3p-kafka 3w REG 254,1 11038211401 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22295 fk-3p-kafka 3w REG 254,1 11038211401 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22296 fk-3p-kafka 3w REG 254,1 11038211401 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22297 fk-3p-kafka 3w REG 254,1 11038211401 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22298 fk-3p-kafka 3w REG 254,1 11038212865 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22299 fk-3p-kafka 3w REG 254,1 11038212865 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22300 fk-3p-kafka 3w REG 254,1 11038212865 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22301 fk-3p-kafka 3w REG 254,1 11038212865 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22302 fk-3p-kafka 3w REG 254,1 11038212865 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22303 fk-3p-kafka 3w REG 254,1 11038212865 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22304 fk-3p-kafka 3w REG 254,1 11038212865 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22305 fk-3p-kafka 3w REG 254,1 11038212865 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22306 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22307 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22308 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22309 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22310 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22311 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22312 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22313 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22317 fk-3p-kafka 3w REG 254,1 11038214329 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22318 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22319 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22320 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22321 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22322 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22323 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22324 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22325 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22326 fk-3p-kafka 3w REG 254,1 11038215793 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22327 fk-3p-kafka 3w REG 254,1 11038217258 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22328 fk-3p-kafka 3w REG 254,1 11038217258 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22329 fk-3p-kafka 3w REG 254,1 11038217258 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22330 fk-3p-kafka 3w REG 254,1 11038217258 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22331 fk-3p-kafka 3w REG 254,1 11038217258 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22332 fk-3p-kafka 3w REG 254,1 11038217258 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22333 fk-3p-kafka 3w REG 254,1 11038217258 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22334 fk-3p-kafka 3w REG 254,1 11038217258 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22335 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22336 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22337 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22338 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22339 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22340 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22341 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22342 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22343 fk-3p-kafka 3w REG 254,1 11038218724 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22344 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22345 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22349 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22352 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22367 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 22450 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 26510 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 26511 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 26740 fk-3p-kafka 3w REG 254,1 11038220188 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 26741 fk-3p-kafka 3w REG 254,1 11038221652 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 26749 fk-3p-kafka 3w REG 254,1 11038221652 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 26750 fk-3p-kafka 3w REG 254,1 11038221652 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 26758 fk-3p-kafka 3w REG 254,1 11038221652 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 26759 fk-3p-kafka 3w REG 254,1 11038221652 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 28172 fk-3p-kafka 3w REG 254,1 11038227512 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 28173 fk-3p-kafka 3w REG 254,1 11038227512 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 28174 fk-3p-kafka 3w REG 254,1 11038227512 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 31036 fk-3p-kafka 3w REG 254,1 11038221652 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 31187 fk-3p-kafka 3w REG 254,1 11038221652 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 31339 fk-3p-kafka 3w REG 254,1 11038221652 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 31389 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 31692 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32151 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32350 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32452 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32603 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32606 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32607 fk-3p-kafka 3w REG 254,1 11038226047 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32608 fk-3p-kafka 3w REG 254,1 11038227512 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32654 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
java 22252 32755 fk-3p-kafka 3w REG 254,1 11038223116 132380 /var/log/test/fk-3p-kafka/kafkaServer-gc.log
显然kafkaServer-gc.log没有删除的FD。我也尝试截断文件,但 inode 中的文件大小没有改变。这种行为的原因可能是什么?
答案1
鉴于这是一个日志文件,以下是最可能的解释:
- 有一个进程打开文件并不断向其追加内容。目前该进程为 22252,过去可能存在其他进程(同一应用程序的先前实例)。
- 在过去的某个时刻,有人截断了该文件。将文件截断到某个位置会删除该位置之后的所有数据,但不会影响附加到文件的进程的位置。
假设文件被截断到位置 0。然后,在截断操作之后,文件大小立即为 0。但是 Java 进程打开了该文件,并将其当前位置设置为n。 Java进程下次写入日志文件时,它从位置开始写入n到n+k(在哪里k是它写入的字节数),因此在此之后文件大小为n+k,并且随着 Java 进程编写更多内容,它继续增长。
磁盘使用率 1 小于文件大小,因为截断删除了用于文件内容的所有存储。当进程追加到截断后的文件时,即将该文件变成了稀疏文件:位置之前的所有字节n是空字节,但不会重新创建完全由空字节组成的块。这么说吧n= 1024567 并且块大小为 1024 字节,然后在截断后的第一次追加操作中,创建了位置 1024000–1025023 的块,其中 567 个空字节,后跟k该日志消息的字节数。
删除仍处于打开状态的文件不会影响其磁盘使用情况或文件大小,但会使该文件对 不可见,du
并且ls
仍然占用记录为正在使用的空间df
。看为什么有这么多不同的方法来测量磁盘使用情况?
¹文件使用的块数 - 它和文件大小都存储在 inode 中,因此不要将磁盘使用情况称为“inode 文件大小”!
答案2
这可能是一个稀疏文件,即一个大部分为空的文件,并且尚未分配完整大小所需的所有磁盘块。
使用dd if=/dev/zero bs=1 count=1 seek=1E of=sparse
我刚刚制作了一个表观大小为 1 EB 的文件,但我没有那么多磁盘空间:
grove@cassiopeia> ls -lh sparse
-rw-r--r-- 1 grove grove 1.1E Apr 10 23:53 sparse
grove@cassiopeia> du sparse
4 sparse
grove@cassiopeia> stat sparse
File: `sparse'
Size: 1152921504606846977 Blocks: 8 IO Block: 4096 regular file
Device: 902h/2306d Inode: 1922068 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ grove) Gid: ( 1000/ grove)
Access: 2016-04-10 23:53:00.925240922 +0200
Modify: 2016-04-10 23:53:00.925240922 +0200
Change: 2016-04-10 23:53:00.925240922 +0200
Birth: -