KVM 客户端无法访问互联网

KVM 客户端无法访问互联网

我在 Ubuntu 16.04.6 kvm 主机上运行了许多 Ubuntu 客户机。

由于我不得不重启主机,网络不再正常工作。主机和客户机可以互相访问,但我的客户机无法访问互联网,我无法从网络上的任何其他计算机访问客户机。主机在重启前安装了 Docker(可能相关)。

以一个客户机为例:Ubuntu 16.04.4

  • 主机 IP:10.0.10.2
  • 访客 IP:10.0.10.13
  • 客户 KVM 网络设置:
    • 网络源:网桥br0:主机设备eno1
    • 设备型号:virtio

Ping 结果:

colin@host:~$ ping 10.0.10.13
PING 10.0.10.13 (10.0.10.13) 56(84) bytes of data.
64 bytes from 10.0.10.13: icmp_seq=1 ttl=64 time=0.207 ms
colin@host:~$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=58 time=119 ms
colin@guest:~$ ping 10.0.10.2
PING 10.0.10.2 (10.0.10.2) 56(84) bytes of data.
64 bytes from 10.0.10.2: icmp_seq=1 ttl=64 time=0.257 ms
colin@guest:~$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.

--- 1.1.1.1 ping statistics ---
9 packets transmitted, 0 received, 100% packet loss, time 7999ms

在测试期间在主机上运行 tcpdump(ping 主机 > 访客、访客 > 世界、访客 > 主机)显示无回复:

colin@host:~$ sudo tcpdump -nni br0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:57:52.684722 IP 10.0.10.2 > 10.0.10.13: ICMP echo request, id 29324, seq 1, length 64
15:57:52.684953 IP 10.0.10.13 > 10.0.10.2: ICMP echo reply, id 29324, seq 1, length 64
15:57:53.683713 IP 10.0.10.2 > 10.0.10.13: ICMP echo request, id 29324, seq 2, length 64
15:57:53.683891 IP 10.0.10.13 > 10.0.10.2: ICMP echo reply, id 29324, seq 2, length 64
15:58:10.697087 IP 10.0.10.13 > 1.1.1.1: ICMP echo request, id 1620, seq 1, length 64
15:58:11.705736 IP 10.0.10.13 > 1.1.1.1: ICMP echo request, id 1620, seq 2, length 64
15:58:18.482031 IP 10.0.10.13 > 10.0.10.2: ICMP echo request, id 1622, seq 1, length 64
15:58:18.482061 IP 10.0.10.2 > 10.0.10.13: ICMP echo reply, id 1622, seq 1, length 64
15:58:19.481024 IP 10.0.10.13 > 10.0.10.2: ICMP echo request, id 1622, seq 2, length 64
15:58:19.481052 IP 10.0.10.2 > 10.0.10.13: ICMP echo reply, id 1622, seq 2, length 64
^C
10 packets captured
10 packets received by filter
0 packets dropped by kernel

主机配置:

colin@host:~$ ifconfig
br0       Link encap:Ethernet  HWaddr 00:fd:45:fc:85:d8  
          inet addr:10.0.10.2  Bcast:10.0.255.255  Mask:255.255.0.0
          inet6 addr: fe80::2fd:45ff:fefc:85d8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:121153422 errors:0 dropped:715 overruns:0 frame:0
          TX packets:65107786 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:172509978566 (172.5 GB)  TX bytes:16134433818 (16.1 GB)

br-df15cee8a6f6 Link encap:Ethernet  HWaddr 02:42:74:28:3f:82  
          inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
          inet6 addr: fe80::42:74ff:fe28:3f82/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:158611 errors:0 dropped:0 overruns:0 frame:0
          TX packets:402715 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:14673553 (14.6 MB)  TX bytes:62211023 (62.2 MB)

docker0   Link encap:Ethernet  HWaddr 02:42:85:6b:5d:cd  
          inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eno1      Link encap:Ethernet  HWaddr 00:fd:45:fc:85:d8  
          inet6 addr: fe80::2fd:45ff:fefc:85d8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:122992881 errors:0 dropped:14310 overruns:0 frame:0
          TX packets:71313707 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:174820231625 (174.8 GB)  TX bytes:16827079661 (16.8 GB)
          Interrupt:16 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:5163328 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5163328 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:3075521109 (3.0 GB)  TX bytes:3075521109 (3.0 GB)

veth142ebf2 Link encap:Ethernet  HWaddr 72:71:cc:8a:71:ff  
          inet6 addr: fe80::7071:ccff:fe8a:71ff/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:111842 errors:0 dropped:0 overruns:0 frame:0
          TX packets:278641 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:11860403 (11.8 MB)  TX bytes:44222087 (44.2 MB)

veth4bb7c84 Link encap:Ethernet  HWaddr fe:17:53:a6:ea:ca  
          inet6 addr: fe80::fc17:53ff:fea6:eaca/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:232661 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:18124977 (18.1 MB)

veth7a2885b Link encap:Ethernet  HWaddr d6:1c:ff:48:9d:e6  
          inet6 addr: fe80::d41c:ffff:fe48:9de6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:232659 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:18124809 (18.1 MB)

vnet0     Link encap:Ethernet  HWaddr fe:54:00:7e:9d:e0  
          inet6 addr: fe80::fc54:ff:fe7e:9de0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2800 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1339 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:267912 (267.9 KB)  TX bytes:2975296 (2.9 MB)
colin@host:~$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#auto eno1
#iface eno1 inet static
#address 10.0.10.2
#netmask 255.255.0.0
#gateway 10.0.0.1
#dns-nameservers 10.0.0.1

# Bridge
auto br0
iface br0 inet static
        address 10.0.10.2
        netmask 255.255.0.0
        gateway 10.0.0.1
        dns-nameservers 1.1.1.1
        bridge_ports eno1
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0
colin@host:~$ sudo iptables -L -n -v
[sudo] password for colin: 
Chain INPUT (policy ACCEPT 87M packets, 121G bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 2630 packets, 264K bytes)
 pkts bytes target     prot opt in     out     source               destination         
 5346 1299K DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
 5346 1299K DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
 1278  872K ACCEPT     all  --  *      br-df15cee8a6f6  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-df15cee8a6f6  0.0.0.0/0            0.0.0.0/0           
 1438  164K ACCEPT     all  --  br-df15cee8a6f6 !br-df15cee8a6f6  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-df15cee8a6f6 br-df15cee8a6f6  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 49M packets, 16G bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  !br-df15cee8a6f6 br-df15cee8a6f6  0.0.0.0/0            172.20.0.3           tcp dpt:9000
    0     0 ACCEPT     tcp  --  !br-df15cee8a6f6 br-df15cee8a6f6  0.0.0.0/0            172.20.0.4           tcp dpt:8181

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
 1438  164K DOCKER-ISOLATION-STAGE-2  all  --  br-df15cee8a6f6 !br-df15cee8a6f6  0.0.0.0/0            0.0.0.0/0           
 5346 1299K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-df15cee8a6f6  0.0.0.0/0            0.0.0.0/0           
 1438  164K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 5346 1299K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0  

来宾配置:

colin@guest:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:7e:9d:e0  
          inet addr:10.0.10.13  Bcast:10.0.255.255  Mask:255.255.0.0
          inet6 addr: fe80::5054:ff:fe7e:9de0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1057 errors:0 dropped:256 overruns:0 frame:0
          TX packets:2384 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2950816 (2.9 MB)  TX bytes:228448 (228.4 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:141 errors:0 dropped:0 overruns:0 frame:0
          TX packets:141 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:22109 (22.1 KB)  TX bytes:22109 (22.1 KB)
colin@guest:~$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
address 10.0.10.13
netmask 255.255.0.0
network 10.0.0.0
broadcast 10.0.255.255
gateway 10.0.0.1
dns-nameservers 8.8.8.8 8.8.4.4

我已经尝试了数周来解决这个问题,但似乎没有取得任何进展。

答案1

我在 Reddit 上收到了一条评论,它为我指明了正确的方向:https://www.reddit.com/r/linuxadmin/comments/bdy6sz/kvm_guest_cant_access_internet/

问题在于Docker 将iptables中FORWARD链的默认策略更改为DROP。

必须添加允许流量的规则。运行此命令可添加所需规则并修复问题:

sudo iptables -I FORWARD -i br0 -o br0 -j ACCEPT

答案2

需要补充的是,iptables已被弃用并移至iptables-legacy,但docker的行为仍然存在:

问题在于Docker 将iptables中FORWARD链的默认策略更改为DROP。

我尝试使用iptables-persistent,但它似乎是在docker之前加载的,所以docker仍然违反了上面所需的转发规则。

我想到最好的解决方案是添加一个 systemd 服务,该服务将在 Docker 添加此条iptables目后运行。

注意今天我也在运行 docker 的 snap 版本,如果你足够明智地运行非 snap docker,你可以替换snap.docker.dockerd.service我认为它只是docker.service

/etc/systemd/system/iptables-after-docker.service

[Unit]
Description=iptables rule after docker
After=snap.docker.dockerd.service
BindsTo=snap.docker.dockerd.service
ReloadPropagatedFrom=snap.docker.dockerd.service

[Service]
Type=oneshot
ExecStart=/usr/local/bin/iptables-after-docker
ExecReload=/usr/local/bin/iptables-after-docker
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

/usr/local/bin/iptables-after-docker

#!/bin/bash

iptables-legacy -I FORWARD -i br0 -o br0 -j ACCEPT

不要忘记启用它:

sudo systemctl daemon-reload
sudo systemctl enable iptables-after-docker.service

重新启动并确保你br0在 iptables 中看到以下条目:

sudo reboot

# check that you now see the entry
sudo iptables-legacy-save | grep -i br0
# You should see you're `iptables` rule...
# Like: -A FORWARD -i br0 -o br0 -j ACCEPT

祝你好运!

相关内容