文件位置

文件位置

在 GCE 上的 docker 实现中,我们的根文件系统空间不足。

由于图像本身存储在单独的 1TB 卷上,因此图像本身不应该是问题。

一个候选者是 Docker 本身存储的集中日志文件(某个地方的 json 文件?),有人知道这些文件在哪里/位于哪里,以及我们如何记录/截断它们?

答案1

首先,我在客户端和服务器上都使用 docker 1.1.2,由于 docker 快速发展,这个答案对于较新版本的 docker 来说可能已经过时了。

文件位置

找到您的 docker 目录。在使用 apt/debian 样式系统的系统上,docker 存储库安装的软件包https://get.docker.com/ubuntu使用/var/lib/docker。该目录很可能在其他系统上位于同一位置(无法确认)。

您将在下面containers/**CONTAINER_ID**找到有关容器的信息。在该文件夹中的文件中**CONTAINER_ID**-json.log,您将找到一个包含该容器的所有日志的文件。它可能看起来像一个 json 文件,但实际上不是。它是一个 json 结构流,每行一个,每个包含一个日志行(每行以 结尾,}下一行以 开头{,因此它不是一个有效的 json 整体)。

示例位置:-/var/lib/docker/containers/05b6053c41a2130afd6fc3b158bda4e605b6053c41a2130afd6fc3b158bda4e6/05b6053c41a2130afd6fc3b158bda4e605b6053c41a2130afd6fc3b158bda4e6-json.log

编辑/修改该文件

我建议你使用这条路径无论这是否是您空间不足的原因,但不要记录轮换它们。

我宁愿确保容器不会记录太多行(通过在 dockerfile 中使用 CMD,将进程的输出重定向到卷中的文件或 /dev/null - 使用配置启用日志 - 然后我会用另一个容器对日志文件进行日志轮转)

答案2

这里有一个清理容器日志文件的简单命令:

echo "" > $(docker inspect --format='{{.LogPath}}' <container_name_or_id>)

答案3

如果你运行sudo lsof | grep '(deleted)',你可能会发现有很多被删除的容器日志文件被docker保留下来

docker    17511             root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511  2879       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511  6191       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511  8713       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511  8726       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511  9907       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511  9918       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 11093       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 11096       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 11110       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 11885       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 11890       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 11973       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 11976       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 11983       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 13105       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 16737       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17139       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17516       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17517       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17518       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17519       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17520       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17525       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17574       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17584       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17586       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17637       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17643       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17644       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17659       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17666       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17669       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17680       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17884       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 17895       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 21126       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 21474       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 21475       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 21519       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 21721       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 23829       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 23830       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 23833       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 24196       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 24197       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 26821       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 28597       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 28608       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 30305       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 30308       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 30594       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)
docker    17511 30595       root   25r      REG              202,1      105     456624 /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)

这些是属于已删除容器的日志。关闭这些日志的一种方法是重新启动 docker 守护进程本身。但这并不总是可行的。

释放该空间的另一种方法是找到相关的文件描述符并截断它:

ls -lh /proc/17511/fd/ | grep e49d81fc
lr-x------ 1 root root 64 Sep  8 08:06 25 -> /var/lib/docker/containers/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc/39ba30ffc0e218ee830576570222284a93ebb458eadf46933d2dd7afe49d81fc-json.log (deleted)

17511docker 的 pid在哪里,e49d81fc是有问题的日志文件名/容器 id 的一部分。这里,fd 是 /proc/17511/fd/25,所以我们截断它:

cat /dev/null > /proc/17511/fd/25

答案4

答案有点晚了,但任何处于同样境地的人最好探索将他们的 docker 日志导出到合理的地方进行管理的选项。 日志输出可用于从所有 docker 容器中收集日志并将它们发送到某个地方。如果您只需要考虑这一点,那么将它们发送到主机的 syslog 守护程序可能是一种好方法,并且您可以使用通常的机制将它们排列到文件中并从那里轮换它们。对于较大的多主机环境,您可以考虑运送到“ELK”堆栈。即 Elasticsearch、Logstash 和 Kibana。

相关内容