Openstack 实例在附加浮动 IP 后失去互联网访问

Openstack 实例在附加浮动 IP 后失去互联网访问

也许有人有同样的问题

我在运行 ubuntu 20.04 的两台虚拟机(1 个控制节点、1 个计算节点)上安装了 Openstack Victoria。每个节点都有两个网络接口,即管理网络和提供商网络。我创建了专用网络并将其连接到路由器。通过此配置,我可以访问互联网。

但是,当我将浮动 IP 附加到我的实例时,它失去了互联网连接。我可以从外部访问此实例,但实例无法访问网络网关。我用 进行了检查ip netns exec <qrouter-id> ping 8.8.8.8。它一直正常工作,直到我附加 FIP。

我认为这是路由问题,但我找不到问题出在哪里?你们有什么想法吗?

10.0.0.0/24 - 管理网络
10.0.2.0/24 - 外部(提供商)网络

Linuxbridge的配置:

root@compute1:/# grep -v "^#" /etc/neutron/plugins/ml2/linuxbridge_agent.ini | grep -v "^$"
[DEFAULT]
[agent]
extensions = qos
[linux_bridge]
physical_interface_mappings = provider:ens34
[network_log]
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan = true
local_ip = 10.0.0.131
l2_population = true

提供商网络:

root@controller1:/# openstack subnet show provider
| Field | Value |
| allocation_pools | 10.0.2.50-10.0.2.150 |
| cidr | 10.0.2.0/24|
| created_at | 2021-02-22T16:17:20Z |
| description | |
| dns_nameservers | 8.8.8.8|
| dns_publish_fixed_ip | None |
| enable_dhcp | True |
| gateway_ip | 10.0.2.1|
| host_routes | |
| id | 7d07101a-4696-4ff8-88bc-fa4ffde1622f |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | provider |
| network_id | d65d17fe-9829-44d5-bf07-1abb70f9d523 |
| prefix_length | None |
| project_id | 957f142f850240b5801023369eace69a |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |

路由器:

root@controller1:/# openstack router show router1
| Field | Value |
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2021-02-22T16:17:51Z |
| description | |
| distributed | False |
| external_gateway_info | {"network_id": "d65d17fe-9829-44d5-bf07-1abb70f9d523", "external_fixed_ips": [{"subnet_id": "7d07101a-4696-4ff8-88bc-fa4ffde1622f", "ip_address": "10.0.2.51"}], "enable_snat": true} |
| flavor_id | None |
| ha | False |
| id | fa11f06e-906c-4ae9-8176-20fb74e1cacd |
| interfaces_info | [{"port_id": "67d37c5f-1250-45e7-a003-78493921b4d6", "ip_address": "172.16.1.1", "subnet_id": "b0762924-6c7a-453f-a9b8-788e15e5f0c0"}] |
| name | router1 |
| project_id | 957f142f850240b5801023369eace69a |
| revision_number | 4 |
| routes | |
| status | ACTIVE |

网络命名空间:

root@controller1:/# ip netns
qrouter-fa11f06e-906c-4ae9-8176-20fb74e1cacd (id: 3)
qdhcp-d65d17fe-9829-44d5-bf07-1abb70f9d523 (id: 0)
qdhcp-f6a245eb-001d-47b1-8af5-38178585fe87 (id: 6)
qdhcp-0fb79928-ae24-4d85-8c58-b1acb9c8c9d2 (id: 2)
qdhcp-0ab1f94c-1e06-485c-b024-548a927a5e36 (id: 1)

root@controller1:/# ip netns exec qrouter-fa11f06e-906c-4ae9-8176-20fb74e1cacd ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=128 time=11.7 ms
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 11.679/11.679/11.679/0.000 ms

root@controller1:/# ip netns exec qrouter-fa11f06e-906c-4ae9-8176-20fb74e1cacd ip route
default via 10.0.2.1 dev qg-61a6ea6f-7e proto static
10.0.2.0/24 dev qg-61a6ea6f-7e proto kernel scope link src 10.0.2.51
172.16.1.0/24 dev qr-67d37c5f-12 proto kernel scope link src 172.16.1.1

所以一切都很好...现在我正在附加 FIP

root@controller1:/# openstack floating ip list
| ID | Floating IP Address | Fixed IP Address | Port | Floating Network | Project |
| 8a3333a9-345d-4b2a-9d63-420f09e4c020 | 10.0.2.106| 172.16.1.236| edef7b03-25a9-43b4-9953-831539056ac3 | d65d17fe-9829-44d5-bf07-1abb70f9d523 | 957f142f850240b5801023369eace69a |

我可以从本地电脑 ping 通它,也可以通过 SSH 访问实例,但是我无法从提供商网络访问互联网:

root@controller1:/# ip netns exec qrouter-fa11f06e-906c-4ae9-8176-20fb74e1cacd ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2040ms

这是来自计算节点的 tcpdump:

root@compute1:/# tcpdump -i ens34 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens34, link-type EN10MB (Ethernet), capture size 262144 bytes
17:30:00.258697 IP 10.0.2.106 > 8.8.8.8: ICMP echo request, id 41872, seq 0, length 64
17:30:01.259844 IP 10.0.2.106 > 8.8.8.8: ICMP echo request, id 41872, seq 1, length 64

因此数据包正在通过提供商接口ens34。我认为这是计算节点上的路由问题,但我找不到它在哪里。

编辑

命名空间中的接口

root@controller1:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: qr-67d37c5f-12@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:cb:0e:3a brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.16.1.1/24 brd 172.16.1.255 scope global qr-67d37c5f-12
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fecb:e3a/64 scope link
       valid_lft forever preferred_lft forever
3: qg-61a6ea6f-7e@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:60:e9:e4 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.0.2.51/24 brd 10.0.2.255 scope global qg-61a6ea6f-7e
       valid_lft forever preferred_lft forever
    inet 10.0.2.106/32 brd 10.0.2.106 scope global qg-61a6ea6f-7e
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe60:e9e4/64 scope link
       valid_lft forever preferred_lft forever

命名空间中的路由表

root@controller1:/# ip r
default via 10.0.2.1 dev qg-61a6ea6f-7e proto static
10.0.2.0/24 dev qg-61a6ea6f-7e proto kernel scope link src 10.0.2.51
172.16.1.0/24 dev qr-67d37c5f-12 proto kernel scope link src 172.16.1.1

命名空间中的 iptables

root@controller1:/# iptables-save
# Generated by iptables-save v1.8.4 on Tue Feb 23 12:51:04 2021
*raw
:PREROUTING ACCEPT [105:8611]
:OUTPUT ACCEPT [65:6090]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
COMMIT
# Completed on Tue Feb 23 12:51:04 2021
# Generated by iptables-save v1.8.4 on Tue Feb 23 12:51:04 2021
*nat
:PREROUTING ACCEPT [8:1322]
:INPUT ACCEPT [18:1372]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-POSTROUTING - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
:neutron-l3-agent-float-snat - [0:0]
:neutron-l3-agent-snat - [0:0]
:neutron-postrouting-bottom - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-l3-agent-OUTPUT -d 10.0.2.106/32 -j DNAT --to-destination 172.16.1.236
-A neutron-l3-agent-POSTROUTING ! -o qg-61a6ea6f-7e -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 10.0.2.106/32 -j DNAT --to-destination 172.16.1.236
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-l3-agent-float-snat -s 172.16.1.236/32 -j SNAT --to-source 10.0.2.106 --random-fully
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -o qg-61a6ea6f-7e -j SNAT --to-source 10.0.2.51 --random-fully
-A neutron-l3-agent-snat -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source 10.0.2.51 --random-fully
-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
COMMIT
# Completed on Tue Feb 23 12:51:04 2021
# Generated by iptables-save v1.8.4 on Tue Feb 23 12:51:04 2021
*mangle
:PREROUTING ACCEPT [105:8611]
:INPUT ACCEPT [99:7701]
:FORWARD ACCEPT [2:102]
:OUTPUT ACCEPT [65:6090]
:POSTROUTING ACCEPT [67:6192]
:neutron-l3-agent-FORWARD - [0:0]
:neutron-l3-agent-INPUT - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-POSTROUTING - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
:neutron-l3-agent-float-snat - [0:0]
:neutron-l3-agent-floatingip - [0:0]
:neutron-l3-agent-mark - [0:0]
:neutron-l3-agent-scope - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A INPUT -j neutron-l3-agent-INPUT
-A FORWARD -j neutron-l3-agent-FORWARD
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A neutron-l3-agent-POSTROUTING -o qg-61a6ea6f-7e -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-mark
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-scope
-A neutron-l3-agent-PREROUTING -m connmark ! --mark 0x0/0xffff0000 -j CONNMARK --restore-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-floatingip
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xffff
-A neutron-l3-agent-float-snat -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-mark -i qg-61a6ea6f-7e -j MARK --set-xmark 0x2/0xffff
-A neutron-l3-agent-scope -i qr-67d37c5f-12 -j MARK --set-xmark 0x4000000/0xffff0000
-A neutron-l3-agent-scope -i qg-61a6ea6f-7e -j MARK --set-xmark 0x4000000/0xffff0000
COMMIT
# Completed on Tue Feb 23 12:51:04 2021
# Generated by iptables-save v1.8.4 on Tue Feb 23 12:51:04 2021
*filter
:INPUT ACCEPT [3:741]
:FORWARD ACCEPT [2:102]
:OUTPUT ACCEPT [65:6090]
:neutron-filter-top - [0:0]
:neutron-l3-agent-FORWARD - [0:0]
:neutron-l3-agent-INPUT - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-local - [0:0]
:neutron-l3-agent-scope - [0:0]
-A INPUT -j neutron-l3-agent-INPUT
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-l3-agent-FORWARD
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A neutron-filter-top -j neutron-l3-agent-local
-A neutron-l3-agent-FORWARD -j neutron-l3-agent-scope
-A neutron-l3-agent-INPUT -m mark --mark 0x1/0xffff -j ACCEPT
-A neutron-l3-agent-INPUT -p tcp -m tcp --dport 9697 -j DROP
-A neutron-l3-agent-scope -o qr-67d37c5f-12 -m mark ! --mark 0x4000000/0xffff0000 -j DROP
COMMIT
# Completed on Tue Feb 23 12:51:04 2021

来自命名空间的 tcpdump

root@controller1:/# tcpdump -e -i any host 8.8.8.8
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
13:56:31.517183 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP                                                                                                                                                                           echo request, id 58998, seq 14, length 64
13:56:31.524754  In 00:50:56:f5:43:f8 (oui Unknown) ethertype IPv4 (0x0800), length 100: 8.8.8.8 > 10.0.2.131: ICMP                                                                                                                                                                           echo reply, id 58998, seq 14, length 64
13:56:31.524803 Out fa:16:3e:e3:79:16 (oui Unknown) ethertype IPv4 (0x0800), length 100: 8.8.8.8 > 172.16.1.236: ICM                                                                                                                                                                          P echo reply, id 34561, seq 14, length 64
13:56:32.518197  In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM                                                                                                                                                                          P echo request, id 34561, seq 15, length 64
13:56:32.518237 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP                                                                                                                                                                           echo request, id 58998, seq 15, length 64
13:56:33.519420  In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM                                                                                                                                                                          P echo request, id 34561, seq 16, length 64
13:56:33.519463 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP                                                                                                                                                                           echo request, id 58998, seq 16, length 64
13:56:34.520250  In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM                                                                                                                                                                          P echo request, id 34561, seq 17, length 64
13:56:34.520291 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP                                                                                                                                                                           echo request, id 58998, seq 17, length 64
13:56:35.521179  In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM                                                                                                                                                                          P echo request, id 34561, seq 18, length 64
13:56:35.521216 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP                                                                                                                                                                           echo request, id 58998, seq 18, length 64
13:56:36.522122  In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM                                                                                                                                                                          P echo request, id 34561, seq 19, length 64
13:56:36.522158 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP                                                                                                                                                                           echo request, id 58998, seq 19, length 64
13:56:37.522683  In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM                                                                                                                                                                          P echo request, id 34561, seq 20, length 64

相关内容