在 vagrant / hyper-v / libvirt 上的嵌套虚拟化集群中设置路由

在 vagrant / hyper-v / libvirt 上的嵌套虚拟化集群中设置路由

我正在尝试使用 vagrant 在本地笔记本上设置特定的虚拟机集群。我有一台 Win10 Pro,以 HyperV 作为主机。不幸的是,我无法更改它,因此我想在 HyperV 上创建一个 Linux VM(我称之为“嵌套”),并启用嵌套虚拟化。然后在“嵌套”Linux 上,我想使用 KVM/libvirt 创建多个 VM。为简单起见,示例代码中只有一个(“guest”)。

假设是在 Windows Host 上进行最小设置并通过 Vagrant 配置所有内容。但事实上这并不那么容易,因为 HyperV 提供程序不允许在公共和私有接口中设置静态 IP。

目标是创建一个私有网络(例如 10.75.10.0/24)并为每个虚拟机分配一个静态 IP 地址,并允许它们相互看到。我部分实现了这一点。我的虚拟机可以互相 ping 通,但嵌套的虚拟机除外(“guest1”<->“guest2”,“guest”<->Host,Host<->Nested)。guest<->nested 在两个方向上都看不到对方。

要求: 1. 启用 HyperV 的 Win10 2. 用户需要属于 HyperV 管理组

我使用 PowerShell 脚本(Adm)在主机中创建了一个内部交换机:

New-VMSwitch –SwitchName “NAT-Switch” –SwitchType Internal –Verbose
Get-NetAdapter
#put the proper index in the next line
New-NetIPAddress –IPAddress 10.75.0.1 -PrefixLength 24 -InterfaceIndex 16 –Verbose
New-NetNat –Name NATNetwork –InternalIPInterfaceAddressPrefix 10.75.0.0/24 –Verbose
Get-VM | Get-VMNetworkAdapter | Connect-VMNetworkAdapter –SwitchName “NAT-Switch”

ipconfig:

Ethernet adapter vEthernet (NAT-Switch):

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter #5
   Physical Address. . . . . . . . . : XXXX
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : XXXX
   IPv4 Address. . . . . . . . . . . : 10.75.10.1(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

这是我的项目结构:

./nested
./nested/Vagrantfile
./provision.sh
./Vagrantfile

猫Vagrantfile

$bridge_script = <<EOF
    ifconfig eth0 10.75.10.10 netmask 255.255.255.0 up
    route add default gw 10.75.10.1
EOF

Vagrant.configure("2") do |config|

    config.vm.box = "generic/ubuntu1804"
    config.vm.define "nested"
    config.vm.hostname = "nested"
    config.vm.provider "hyperv" do |hv|
        hv.cpus = "2"
        hv.memory = "2048"
        hv.maxmemory = "2048"
        hv.enable_virtualization_extensions = true
        hv.differencing_disk = true
    end

    config.vm.network "public_network", :bridge => "NAT-Switch", auto_config: false
    config.vm.provision :"shell", run: "always", inline: $bridge_script
    config.vm.provision :file, source: "nested", destination: "~/"
    config.vm.provision :shell, :path => "provision.sh"
    config.vm.provision :shell, inline: "sudo -i -u vagrant bash -c 'cd ~vagrant/nested; vagrant up --provider=libvirt'"

end

猫规定.sh

apt-get update
apt-get install -y bridge-utils qemu-kvm virtinst libvirt-bin qemu-utils qemu ebtables dnsmasq vagrant vagrant-libvirt
usermod -aG libvirt-qemu,libvirt vagrant
systemctl enable libvirt-bin
systemctl start libvirt-bin
vagrant plugin install vagrant-libvirt
vagrant plugin list

猫嵌套/Vagrantfile

Vagrant.configure("2") do |config|
        config.vm.box = "generic/ubuntu1604"
        config.vm.define "guest"
        config.vm.network "public_network", ip: "10.75.10.11", netmask: "24"
        config.vm.hostname = "guest"

        config.vm.provider :libvirt do |libvirt|
                libvirt.uri = 'qemu+unix:///system'
                libvirt.cpus = "1"
                libvirt.memory = "1024"
        end
end

运行后:

vagrant --provider=hyperv

我得到以下结果:

嵌套虚拟机:

vagrant@nested:~$ ip 地址

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:15:5d:38:01:5c brd ff:ff:ff:ff:ff:ff
    inet 10.75.10.10/24 brd 10.75.10.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::215:5dff:fe38:15c/64 scope link
       valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:6f:8e:84 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:6f:8e:84 brd ff:ff:ff:ff:ff:ff
5: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:6c:f9:2f brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.1/24 brd 192.168.121.255 scope global virbr1
       valid_lft forever preferred_lft forever
6: virbr1-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr1 state DOWN group default qlen 1000
    link/ether 52:54:00:6c:f9:2f brd ff:ff:ff:ff:ff:ff
7: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr1 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:bc:01:c9 brd ff:ff:ff:ff:ff:ff
8: macvtap0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 500
    link/ether 52:54:00:a7:eb:55 brd ff:ff:ff:ff:ff:ff

vagrant@nested:~$ route -n

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.75.10.1      0.0.0.0         UG    0      0        0 eth0
10.75.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.121.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

对于 VM1:

vagrant@guest:~$ ip 地址

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:bc:01:c9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.45/24 brd 192.168.121.255 scope global eth0
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:a7:eb:55 brd ff:ff:ff:ff:ff:ff
    inet 10.75.10.11/24 brd 10.75.10.255 scope global eth1
       valid_lft forever preferred_lft forever

vagrant@guest:~$ route -n

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.121.1   0.0.0.0         UG    0      0        0 eth0
10.75.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.121.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0

我希望有人能重现这个状态。我提供了 99% 自动化的脚本。

问题是:我该如何改进此设置以使嵌套虚拟机可见?操作接口寻址和路由会导致 vagrant 长时间中断。我也可以改进这一点吗?有没有更好的方法在嵌套虚拟机中设置网络?

相关内容