我最近在我的服务器上安装了 proxmox，现在我正在尝试使网络正常运行。
我遇到的问题是，对于某个 VLAN，来自 DHCP 服务器的 DHCP 响应似乎没有到达我的 VM。它似乎到达了 proxmox，但 VM 中没有任何内容。
我想要实现的目标如下：
从我的 ISP 那里，我有一个 /29 子网。我有一个运行 RouterOS 7 的 Mikrotik 路由器。我配置了一个 VLAN (id=200)，我想在其中添加从该子网获取 IP 的设备和虚拟机。除此之外，我还有一个标准的 192.168.some.thing 子网，我的家庭网络中的所有其他硬件都使用它。
因此，我想在 proxmox 中添加一个具有两个虚拟接口的 VM，一个从 192.168 子网获取 IP，另一个从我的 ISP 的 /29 子网获取 IP。
我的网络拓扑如下：
ISP <-> Mikrotik 路由器 <-> 运行 Proxmox 的服务器 <-> 交换机 <-> PC
我的服务器有两个物理接口，我将它们桥接在一起 (vmbr0)。我这样做是因为我的交换机不受管理，而且我读到它会从数据包中剥离 VLAN 标签。因此，交换机在更下游运行，后面还有几块硬件。我不认为这会对我的设置产生太大影响，但这样你就能了解整个情况。
那么，我在 proxmox 中做了什么来实现这一点：
我为虚拟机添加了一个额外的接口，并将 VLAN 标记设置为 200 我使网桥 (vmbr0) 具有 VLAN 感知能力 但是，接口似乎没有获得 IP。奇怪的是，我的路由器提供了 DHCP 租约。更奇怪的是：它似乎最终进入了 proxmox，但它停在那里（它没有到达虚拟机）。
为了分析问题，我运行了 tcpdump 来收集 DHCP 流量：
tcpdump -i vmbr0v200 -pvn port 67 and port 68
首先，我在 proxmox 上运行了这个。结果如下：
10:31:59.579453 IP (tos 0x0, ttl 16, id 0, offset 0, flags [none], proto UDP (17), length 328)
45.xxx.xxx.233.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 300, xid 0x6c990843, Flags [none]
Your-IP 45.xxx.xxx.234
Server-IP 45.xxx.xxx.233
Client-Ethernet-Address 9a:01:73:ba:ab:15
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53), length 1: Offer
Subnet-Mask (1), length 4: 255.255.255.248
Default-Gateway (3), length 4: 45.xxx.xxx.233
Domain-Name-Server (6), length 8: 8.8.8.8,8.8.4.4
Lease-Time (51), length 4: 600
Server-ID (54), length 4: 45.xxx.xxx.233
所以，这一切似乎都很好！给出了正确的 IP 地址，接口 MAC 地址是正确的（9a:01:73:ba:ab:15 是我放置 VLAN 标记的虚拟接口的 MAC 地址）。但是，虚拟机端什么都没有。
当我在虚拟机端运行同样的事情时：
tcpdump -i ens19 -pvn port 67 and port 68
我收到的请求如下：
14:34:54.988228 IP (tos 0xc0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 322)
0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 9a:01:73:ba:ab:15, length 294, xid 0xc62d7779, secs 1342, Flags [none]
Client-Ethernet-Address 9a:01:73:ba:ab:15
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53), length 1: Discover
Client-ID (61), length 19: hardware-type 255, 9e:37:69:8c:00:02:00:00:ab:11:f2:16:3b:8b:f0:03:d6:c8
Parameter-Request (55), length 11:
Subnet-Mask (1), Default-Gateway (3), Domain-Name-Server (6), Hostname (12)
Domain-Name (15), MTU (26), Static-Route (33), NTP (42)
Unknown (119), Unknown (120), Classless-Static-Route (121)
MSZ (57), length 2: 576
Hostname (12), length 10: "kubernetes"
有谁知道这个设置可能出现什么问题？
一些额外的发现：我知道我的路由器会隐式地用 VLAN 1 标记无 VLAN 的数据包。因此，在 192.168 子网所在的虚拟接口上，我将 VLAN 标记设置为 1。这有效，我确实从路由器获得了 IP。我在虚拟机中运行了 tcpdump，输出如下：
tcpdump: listening on ens18, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:42:37.582434 IP (tos 0xc0, ttl 64, id 11934, offset 0, flags [DF], proto UDP (17), length 322)
192.168.88.157.68 > 192.168.88.1.67: BOOTP/DHCP, Request from c6:13:ea:18:09:cc, length 294, xid 0x120faea6, secs 1805, Flags [none]
Client-IP 192.168.88.157
Client-Ethernet-Address c6:13:ea:18:09:cc
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53), length 1: Request
Client-ID (61), length 19: hardware-type 255, ca:53:09:5a:00:02:00:00:ab:11:f2:16:3b:8b:f0:03:d6:c8
Parameter-Request (55), length 11:
Subnet-Mask (1), Default-Gateway (3), Domain-Name-Server (6), Hostname (12)
Domain-Name (15), MTU (26), Static-Route (33), NTP (42)
Unknown (119), Unknown (120), Classless-Static-Route (121)
MSZ (57), length 2: 576
Hostname (12), length 10: "kubernetes"
14:42:37.593791 IP (tos 0x0, ttl 16, id 0, offset 0, flags [none], proto UDP (17), length 328)
192.168.88.1.67 > 192.168.88.157.68: BOOTP/DHCP, Reply, length 300, xid 0x120faea6, Flags [none]
Client-IP 192.168.88.157
Your-IP 192.168.88.157
Server-IP 192.168.88.1
Client-Ethernet-Address c6:13:ea:18:09:cc
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53), length 1: ACK
Subnet-Mask (1), length 4: 255.255.255.0
Default-Gateway (3), length 4: 192.168.88.1
Domain-Name-Server (6), length 8: 192.168.88.1,8.8.8.8
Lease-Time (51), length 4: 600
Server-ID (54), length 4: 192.168.88.1
编辑：
因此，我进行了更多的挖掘，似乎在 Proxmox 上一切都很好：
nmap --script broadcast-dhcp-discover -e vmbr0
结果是：
Starting Nmap 7.80 ( https://nmap.org ) at 2023-07-06 12:32 CEST
Pre-scan script results:
| broadcast-dhcp-discover:
| Response 1 of 1:
| IP Offered: 192.168.88.155
| DHCP Message Type: DHCPOFFER
| Subnet Mask: 255.255.255.0
| Router: 192.168.88.1
| Domain Name Server: 192.168.88.1, 8.8.8.8
| IP Address Lease Time: 10m00s
|_ Server Identifier: 192.168.88.1
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 0.85 seconds
和nmap --script broadcast-dhcp-discover -e vmbr0v200
结果是：
Starting Nmap 7.80 ( https://nmap.org ) at 2023-07-06 12:32 CEST
Pre-scan script results:
| broadcast-dhcp-discover:
| Response 1 of 1:
| IP Offered: 45.xxx.xxx.236
| DHCP Message Type: DHCPOFFER
| Subnet Mask: 255.255.255.248
| Router: 45.xxx.xxx.233
| Domain Name Server: 8.8.8.8, 8.8.4.4
| IP Address Lease Time: 10m00s
|_ Server Identifier: 45.xxx.xxx.233
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 0.72 seconds
编辑2：
关于我的设置的一些更多相关信息：
/etc/network/interfaces 的内容：
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!
auto lo
iface lo inet loopback
iface enp99s0f0 inet manual
iface enx3a0de3575b59 inet manual
iface enp99s0f1 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.88.200/24
gateway 192.168.88.1
bridge-ports enp99s0f0 enp99s0f1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
proxmox 本身的防火墙设置都是默认的：我还没有做任何配置。因此，proxmox 应该可以让所有内容通过，并且在 VM 端应该没有任何阻止任何内容。此外，对于其他子网，DHCP 租约正常通过。
以下是我的 mikrotik 配置的相关部分：请注意：ether5 是我的 proxmox 服务器连接到的物理接口。
/interface vlan
add interface=bridge name=vlan-public vlan-id=200
/ip pool
add name=dhcp ranges=192.168.88.10-192.168.88.254
add name=dhcp_pool_freedom ranges=45.xxx.xxx.234-45.xxx.xxx.238
/ip dhcp-server
add address-pool=dhcp interface=bridge lease-time=10m name=defconf
add address-pool=dhcp_pool_freedom interface=vlan-public lease-time=10m name=\
dhcp-freedom
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5
add bridge=bridge interface=vlan-public
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=\
192.168.88.0
add address=45.xxx.xxx.233/29 interface=vlan-public network=45.xxx.xxx.232
/ip dhcp-server network
add address=45.xxx.xxx.232/29 dns-server=8.8.8.8,8.8.4.4 gateway=45.xxx.xxx.233 \
netmask=29
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1 netmask=24
/ip dns
set allow-remote-requests=yes servers=8.8.8.8
/ip dns static
add address=192.168.88.1 disabled=yes name=router.lan
add address=8.8.8.8 name="Google DNS 1"
/ip firewall filter
add action=accept chain=forward connection-nat-state=dstnat disabled=yes
add action=accept chain=input comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related,untracked
add action=drop chain=input src-address=!192.168.88.0/24
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
"defconf: accept established,related, untracked" connection-state=\
established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
connection-state=invalid
add action=drop chain=forward comment=\
"defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
connection-state=new in-interface-list=WAN
/ip firewall mangle
add action=passthrough chain=prerouting
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
ipsec-policy=out,none out-interface-list=WAN src-address=\
!45.xxx.xxx.232/29
add action=masquerade chain=srcnat out-interface=ether1
/ip route
add disabled=no dst-address=0.0.0.0/0 gateway=pppoe-freedom routing-table=\
to-freedom
add disabled=no distance=10 dst-address=0.0.0.0/0 gateway=pppoe-freedom \
pref-src="" routing-table=main scope=30 suppress-hw-offload=no \
target-scope=10
add disabled=no distance=1 dst-address=45.xxx.xxx.232/29 gateway=45.xxx.xxx.232 \
pref-src="" routing-table=main suppress-hw-offload=no
请注意，上面的导出中省略了不相关的条目。