☰

CentOS7 升级后：Docker 服务不再可访问

tag-icon • tag-icon iptables docker centos7 firewalld
CentOS7 升级后：Docker 服务不再可访问

昨天对我的 CentOS7 系统进行了相当大的升级后，我的 Docker 服务不再可访问。

来自本地主机

curl localhost=> curl: (56) Recv failure: Connection reset by peer。

该 (haproxy) 服务之前运行正常，因此我认为容器没有问题。docker ps显示它已被绑定：0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp。

从远程

Firewalld 似乎阻止了连接：

启用防火墙：curl: (7) Failed to connect to my.example.com port 80: No route to host 禁用防火墙：curl: (56) Recv failure: Connection reset by peer

细节

$ docker info

Containers: 11
 Running: 8
 Paused: 0
 Stopped: 3
Images: 37
Server Version: 1.13.1
Storage Driver: devicemapper
 Pool Name: docker-252:1-270354-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/vg-root/docker-data
 Metadata file: /dev/vg-root/docker-meta
 Data Space Used: 20.71 GB
 Data Space Total: 96.64 GB
 Data Space Available: 75.93 GB
 Metadata Space Used: 33.51 MB
 Metadata Space Total: 4.295 GB
 Metadata Space Available: 4.261 GB
 Thin Pool Minimum Free Space: 9.664 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Library Version: 1.02.135-RHEL7 (2016-11-16)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
 selinux
Kernel Version: 3.10.0-327.10.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 11.58 GiB
Name: my.example.com
ID: TXXX:TXNO:X22W:4SMX:NEEE:DBZE:BYX3:XGAN:4UST:6TMM:3LBG:IICW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

# netstat -tulpen

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name
tcp6       0      0 :::587                  :::*                    LISTEN      0          29958267   23921/docker-proxy
tcp6       0      0 :::80                   :::*                    LISTEN      0          29956605   23743/docker-proxy
tcp6       0      0 :::25                   :::*                    LISTEN      0          15146      2495/master
tcp6       0      0 :::26                   :::*                    LISTEN      0          29956768   23932/docker-proxy
tcp6       0      0 :::443                  :::*                    LISTEN      0          29956581   23725/docker-proxy

# firewall-cmd --permanent --zone=trusted --change-interface=docker0

The interface is under control of NetworkManager, setting zone to 'trusted'.
success

# systemctl status firewalld -l

● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since Fr 2017-02-10 01:54:59 CET; 6h ago
     Docs: man:firewalld(1)
 Main PID: 4843 (firewalld)
   Memory: 24.0M
   CGroup: /system.slice/firewalld.service
           └─4843 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid

Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C POSTROUTING -p tcp -s 172.28.0.222 -d 172.28.0.222 --dport 993 -j MASQUERADE' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C DOCKER -p tcp -d 0/0 --dport 27389 -j DNAT --to-destination 172.28.0.222:389 ! -i br-56cc7d2b4e29' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t filter -C DOCKER ! -i br-56cc7d2b4e29 -o br-56cc7d2b4e29 -p tcp -d 172.28.0.222 --dport 389 -j ACCEPT' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C POSTROUTING -p tcp -s 172.28.0.222 -d 172.28.0.222 --dport 389 -j MASQUERADE' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C DOCKER -p tcp -d 0/0 --dport 587 -j DNAT --to-destination 172.28.0.222:587 ! -i br-56cc7d2b4e29' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t filter -C DOCKER ! -i br-56cc7d2b4e29 -o br-56cc7d2b4e29 -p tcp -d 172.28.0.222 --dport 587 -j ACCEPT' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C POSTROUTING -p tcp -s 172.28.0.222 -d 172.28.0.222 --dport 587 -j MASQUERADE' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C DOCKER -p tcp -d 0/0 --dport 26 -j DNAT --to-destination 172.28.0.222:25 ! -i br-56cc7d2b4e29' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t filter -C DOCKER ! -i br-56cc7d2b4e29 -o br-56cc7d2b4e29 -p tcp -d 172.28.0.222 --dport 25 -j ACCEPT' failed:
Feb 10 02:37:05 my.example.com firewalld[4843]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C POSTROUTING -p tcp -s 172.28.0.222 -d 172.28.0.222 --dport 25 -j MASQUERADE' failed:

# systemctl status docker -l

● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Fr 2017-02-10 02:37:08 CET; 6h ago
     Docs: https://docs.docker.com
 Main PID: 23358 (dockerd)
   Memory: 137.6M
   CGroup: /system.slice/docker.service
           ├─23358 /usr/bin/dockerd
           ├─23365 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libco$tainerd/containerd --shim docker-containerd-shim --runtime docker-runc
           ├─23653 docker-containerd-shim 57eeade86f9eb659887b59812c95666e7ee97d7dc987e066c597c45cf960271e /var/run/docker/libcontainerd/57eeade86f9eb659887b59812c95666e7ee9$d7dc987e066c597c45cf960271e docker-runc
           ├─23725 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 172.28.0.99 -container-port 443
           ├─23743 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.28.0.99 -container-port 80
           ├─23753 docker-containerd-shim c737be17ff53005fd2933a2b7cc5042175d03588ec7c1b2cd06b6be46147c832 /var/run/docker/libcontainerd/c737be17ff53005fd2933a2b7cc5042175d0$588ec7c1b2cd06b6be46147c832 docker-runc
           ├─23844 docker-containerd-shim e0fa228730474373a80129ce5326854cc4f464da89d334776ddb1e69e8e89403 /var/run/docker/libcontainerd/e0fa228730474373a80129ce5326854cc4f4$4da89d334776ddb1e69e8e89403 docker-runc
           ├─23864 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 27007 -container-ip 172.28.0.222 -container-port 4190
           ├─23880 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 27005 -container-ip 172.28.0.222 -container-port 993
           ├─23900 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 27389 -container-ip 172.28.0.222 -container-port 389
           ├─23921 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 587 -container-ip 172.28.0.222 -container-port 587
           ├─23932 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 26 -container-ip 172.28.0.222 -container-port 25
           ├─23993 docker-containerd-shim 29b5f9559d4d2406b81e8bf0c53cd639c5fd1ca7cb1cd21fa8f263ea33d3ab05 /var/run/docker/libcontainerd/29b5f9559d4d2406b81e8bf0c53cd639c5fd$ca7cb1cd21fa8f263ea33d3ab05 docker-runc
           ├─24021 docker-containerd-shim 28aab4f737f97547fad4a094d85e1e038c126432a269b55f59ba984d4c09e20d /var/run/docker/libcontainerd/28aab4f737f97547fad4a094d85e1e038c126432a269b55f59ba984d4c09e20d docker-runc
           ├─24086 docker-containerd-shim 854cba88b26b492cd6cf86d39779940d1f4a15c57c67a82cb5fd00a49dcd1513 /var/run/docker/libcontainerd/854cba88b26b492cd6cf86d39779940d1f4a15c57c67a82cb5fd00a49dcd1513 docker-runc
           ├─24202 docker-containerd-shim b5a376fa0b0ed84c384f6c417fb7dd1e88642fcd45a59b1b4b5b54bb4bbd3316 /var/run/docker/libcontainerd/b5a376fa0b0ed84c384f6c417fb7dd1e88642fcd45a59b1b4b5b54bb4bbd3316 docker-runc
           └─24276 docker-containerd-shim 15260ed80b685a369a5da640da09ec79c88b1844a5957d1fcfbe7d25356b6b40 /var/run/docker/libcontainerd/15260ed80b685a369a5da640da09ec79c88b1844a5957d1fcfbe7d25356b6b40 docker-runc

Feb 10 02:37:08 my.example.com dockerd[23358]: time="2017-02-10T02:37:08+01:00" level=info msg="Firewalld running: true"
Feb 10 02:37:08 my.example.com dockerd[23358]: time="2017-02-10T02:37:08.617724084+01:00" level=info msg="Loading containers: done."
Feb 10 02:37:08 my.example.com dockerd[23358]: time="2017-02-10T02:37:08.654889203+01:00" level=info msg="Daemon has completed initialization"
Feb 10 02:37:08 my.example.com dockerd[23358]: time="2017-02-10T02:37:08.654964541+01:00" level=info msg="Docker daemon" commit=092cba3 graphdriver=devicemapper version=1.13.1
Feb 10 02:37:08 my.example.com dockerd[23358]: time="2017-02-10T02:37:08.663899035+01:00" level=info msg="API listen on /var/run/docker.sock"
Feb 10 02:37:08 my.example.com systemd[1]: Started Docker Application Container Engine.

# cat /etc/firewalld/zones/trusted.xml

<?xml version="1.0" encoding="utf-8"?>
<zone target="ACCEPT">
  <short>Trusted</short>
  <description>All network connections are accepted.</description>
  <interface name="docker0"/>
</zone>

# cat /etc/sysconfig/network-scripts/ifcfg-docker0

DEVICE=docker0
STP=no
TYPE=Bridge
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=docker0
UUID=e4aeffc1-cc87-4fad-ac6c-37cdfaf72369
ONBOOT=no
ZONE=trusted
IPADDR=172.17.0.1
PREFIX=16

答案1

事实证明，问题出network在我的 docker-compose 文件中的配置上——当我删除它时，一切正常。

相关内容