☰

CoreOS：`fleetctl stop` 总是以状态 137 退出

tag-icon • tag-icon docker systemd coreos
CoreOS：`fleetctl stop` 总是以状态 137 退出

我一直在尝试熟悉 CoreOS (v. 633.1.0)，并且一直在尝试使用 fleet (我在本地机器上使用推荐的 Vagrant 3 集群设置)。我创建了以下非常基本的服务 ( [email protected])：

[Unit]
Description="Dummy Apache service"
After="docker.service"
Requires="docker.service"

[Service]
TimeoutStartSec=0
TimeoutStopSec=30
ExecStartPre=-/usr/bin/docker kill apache1
ExecStartPre=-/usr/bin/docker rm apache1
ExecStartPre=/usr/bin/docker pull coreos/apache
ExecStart=/usr/bin/docker run --rm --name apache1 -p 80:80 coreos/apache /usr/sbin/apache2ctl -D FOREGROUND
ExecStop=/usr/bin/docker stop apache1

[X-Fleet]
Conflicts=apache@*.service

当我启动它时，它运行完美，但是，每当我尝试停止它时，它都会被标记为失败。这是命令开始执行fleetctl status apache@2时的输出：stop

core@core-01 ~ $ fleetctl stop apache@2
Unit [email protected] loaded on a91e28b0.../172.17.8.101
core@core-01 ~ $ fleetctl status apache@2
● [email protected] - "Dummy Apache container"
   Loaded: loaded (/run/fleet/units/[email protected]; linked-runtime; vendor preset: disabled)
   Active: deactivating (stop) since Wed 2015-04-15 18:45:46 UTC; 2s ago
  Process: 1038 ExecStartPre=/usr/bin/docker pull coreos/apache (code=exited, status=0/SUCCESS)
  Process: 1030 ExecStartPre=/usr/bin/docker rm apache1 (code=exited, status=1/FAILURE)
  Process: 1024 ExecStartPre=/usr/bin/docker kill apache1 (code=exited, status=1/FAILURE)
 Main PID: 1375 (docker);         : 1522 (docker)
   CGroup: /system.slice/system-apache.slice/[email protected]
           ├─1375 /usr/bin/docker run --rm --name apache1 -p 80:80 coreos/apache /usr/sbin/apache2ctl -D FOREGROUND
           └─control
             └─1522 /usr/bin/docker stop apache1

Apr 15 18:43:18 core-01 docker[1038]: 9cd978db300e: Pulling fs layer
Apr 15 18:44:26 core-01 docker[1038]: 9cd978db300e: Download complete
Apr 15 18:44:26 core-01 docker[1038]: 87026dcb0044: Pulling metadata
Apr 15 18:44:27 core-01 docker[1038]: 87026dcb0044: Pulling fs layer
Apr 15 18:44:53 core-01 docker[1038]: 87026dcb0044: Download complete
Apr 15 18:44:53 core-01 docker[1038]: 87026dcb0044: Download complete
Apr 15 18:44:53 core-01 docker[1038]: Status: Downloaded newer image for coreos/apache:latest
Apr 15 18:44:53 core-01 systemd[1]: Started "Dummy Apache container".
Apr 15 18:44:53 core-01 docker[1375]: apache2: Could not reliably determine the server's fully qualified domain name, using 10.1.0.2 for ServerName
Apr 15 18:45:46 core-01 systemd[1]: Stopping "Dummy Apache container"...

然而，几秒钟后：

core@core-01 ~ $ fleetctl status apache@2
● [email protected] - "Dummy Apache container"
   Loaded: loaded (/run/fleet/units/[email protected]; linked-runtime; vendor preset: disabled)
   Active: failed (Result: exit-code) since Wed 2015-04-15 18:45:56 UTC; 1s ago
  Process: 1522 ExecStop=/usr/bin/docker stop apache1 (code=exited, status=0/SUCCESS)
  Process: 1375 ExecStart=/usr/bin/docker run --rm --name apache1 -p 80:80 coreos/apache /usr/sbin/apache2ctl -D FOREGROUND (code=exited, status=137)
  Process: 1038 ExecStartPre=/usr/bin/docker pull coreos/apache (code=exited, status=0/SUCCESS)
  Process: 1030 ExecStartPre=/usr/bin/docker rm apache1 (code=exited, status=1/FAILURE)
  Process: 1024 ExecStartPre=/usr/bin/docker kill apache1 (code=exited, status=1/FAILURE)
 Main PID: 1375 (code=exited, status=137)

Apr 15 18:44:53 core-01 docker[1038]: 87026dcb0044: Download complete
Apr 15 18:44:53 core-01 docker[1038]: Status: Downloaded newer image for coreos/apache:latest
Apr 15 18:44:53 core-01 systemd[1]: Started "Dummy Apache container".
Apr 15 18:44:53 core-01 docker[1375]: apache2: Could not reliably determine the server's fully qualified domain name, using 10.1.0.2 for ServerName
Apr 15 18:45:46 core-01 systemd[1]: Stopping "Dummy Apache container"...
Apr 15 18:45:56 core-01 docker[1522]: apache1
Apr 15 18:45:56 core-01 systemd[1]: [email protected]: main process exited, code=exited, status=137/n/a
Apr 15 18:45:56 core-01 systemd[1]: Stopped "Dummy Apache container".
Apr 15 18:45:56 core-01 systemd[1]: Unit [email protected] entered failed state.
Apr 15 18:45:56 core-01 systemd[1]: [email protected] failed.

据我所见，docker 似乎正在终止容器（这是ExecStartPre服务启动时命令应该执行的操作）。我还使用 调查了日志journalctl，似乎确实如此。

更正：经过进一步检查（并将目光靠近屏幕），我注意到日志中有一行非常重要的内容：

Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="Container afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1 failed to exit within 10 seconds of SIGTERM -

因此，我可以看到 docker 无法正确停止服务（我不明白原因），但我仍然不明白为什么它会继续并尝试销毁容器。以下是其余相关日志：

Apr 15 18:45:46 core-01 systemd[1]: Stopping "Dummy Apache container"...
Apr 15 18:45:46 core-01 fleetd[880]: INFO manager.go:145: Triggered systemd unit [email protected] stop: job=2115
Apr 15 18:45:46 core-01 fleetd[880]: INFO reconcile.go:311: AgentReconciler completed task: type=StopUnit [email protected] reason="unit currently launched but desired state is loaded"
Apr 15 18:45:46 core-01 dockerd[881]: time="2015-04-15T18:45:46Z" level="info" msg="POST /v1.17/containers/apache1/stop?t=10"
Apr 15 18:45:46 core-01 dockerd[881]: time="2015-04-15T18:45:46Z" level="info" msg="+job stop(apache1)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="Container afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1 failed to exit within 10 seconds of SIGTERM -
Apr 15 18:45:56 core-01 kernel: docker0: port 1(veth87a841f) entered disabled state
Apr 15 18:45:56 core-01 kernel: device veth87a841f left promiscuous mode
Apr 15 18:45:56 core-01 kernel: docker0: port 1(veth87a841f) entered disabled state
Apr 15 18:45:56 core-01 systemd-networkd[830]: veth87a841f     : lost carrier
Apr 15 18:45:56 core-01 systemd-networkd[830]: veth87a841f     : could not find udev device: No such device
Apr 15 18:45:56 core-01 systemd-networkd[830]: docker0         : lost carrier
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="+job log(die, afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1, coreos/apache:latest)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job log(die, afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1, coreos/apache:latest) = OK (0)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="+job release_interface(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job attach(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1) = OK (0)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="POST /v1.17/containers/afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1/wait"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="+job wait(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job release_interface(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1) = OK (0)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="+job log(stop, afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1, coreos/apache:latest)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job log(stop, afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1, coreos/apache:latest) = OK (0)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job stop(apache1) = OK (0)"
Apr 15 18:45:56 core-01 docker[1522]: apache1
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job wait(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1) = OK (0)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="GET /v1.17/containers/afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1/json"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="+job container_inspect(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job container_inspect(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1) = OK (0)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="DELETE /v1.17/containers/afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1?v=1"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="+job rm(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="POST /v1.17/containers/afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1/kill?signal=TERM"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="+job kill(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1, TERM)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="+job log(destroy, afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1, coreos/apache:latest)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job log(destroy, afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1, coreos/apache:latest) = OK (0)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job rm(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1) = OK (0)"
Apr 15 18:45:56 core-01 systemd[1]: [email protected]: main process exited, code=exited, status=137/n/a
Apr 15 18:45:56 core-01 systemd[1]: Stopped "Dummy Apache container".
Apr 15 18:45:56 core-01 systemd[1]: Unit [email protected] entered failed state.
Apr 15 18:45:56 core-01 systemd[1]: [email protected] failed.
Apr 15 18:45:56 core-01 dockerd[881]: No such container: afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="info" msg="-job kill(afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1, TERM) = ERR (1)"
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="error" msg="Handler for POST /containers/{name:.*}/kill returned error: No such container: afa4e52d8ff2edaa53d2bae1177535d669a4758f
Apr 15 18:45:56 core-01 dockerd[881]: time="2015-04-15T18:45:56Z" level="error" msg="HTTP Error: statusCode=404 No such container: afa4e52d8ff2edaa53d2bae1177535d669a4758f1cc524056ba55a9da383ffd1"

我不太清楚这里发生了什么，所以非常感谢任何帮助。

更新：只是为了测试，我增加了服务文件中的超时时间（docker stop -t 300 apache1和TimeoutStopSec = 300），但容器即使等待 5 分钟后仍无法停止。为确保万无一失，我尝试直接在命令行上停止容器（docker stop apache1），当然它工作正常。所以似乎有一些东西阻止了容器通过正确停止fleetctl。

谢谢你的时间！

答案1

docker 停止容器的原因是因为容器中的 apache 出现故障。

答案2

你确定它运行完美吗？

我认为最有可能的问题是它根本没有运行，而是运行后停止并重新启动。很可能是因为端口 80 正在使用中，因为您在所有实例中都将其绑定到主机端口 80。由于您在同一台主机上运行 vagrant，因此至少有 2 个会与其他端口发生冲突。如果单元文件位于不同的机器上并且没有任何东西使用端口 80，则不会出现问题。

要解决这个问题，请改用 -P。然后在每台机器上，您必须运行 docker ps 来查看外部实际正在使用的端口。

相关内容