我在主机上有一个虚拟机，并且想在现有虚拟机内创建另一个虚拟机（嵌套）。
- 主机：opensuse Tumbleweed，内核 4.19.7，96 个 Intel-Xeon CPU，256GB RAM（在 kvm 中启用嵌套 virt）
- L1 VM：opensuse Leap，内核 4.12.14，4 个 vCPU，8 GB RAM
- L2 VM：opensuse Leap，1 vCPU，1GB RAM
L1 VM 上的虚拟化显然已启用：
$ virt-host-validate
QEMU: Checking for hardware virtualization : PASS
QEMU: Checking if device /dev/kvm exists : PASS
QEMU: Checking if device /dev/kvm is accessible : PASS
QEMU: Checking if device /dev/vhost-net exists : PASS
QEMU: Checking if device /dev/net/tun exists : PASS
QEMU: Checking for cgroup 'cpu' controller support : PASS
QEMU: Checking for cgroup 'cpuacct' controller support : PASS
QEMU: Checking for cgroup 'cpuset' controller support : PASS
QEMU: Checking for cgroup 'memory' controller support : PASS
QEMU: Checking for cgroup 'devices' controller support : PASS
QEMU: Checking for cgroup 'blkio' controller support : PASS
QEMU: Checking for device assignment IOMMU support : WARN (No ACPI DMAR table found, IOMMU either disabled in BIOS or not supported by this hardware platform)
LXC: Checking for Linux >= 2.6.26 : PASS
LXC: Checking for namespace ipc : PASS
LXC: Checking for namespace mnt : PASS
LXC: Checking for namespace pid : PASS
LXC: Checking for namespace uts : PASS
LXC: Checking for namespace net : PASS
LXC: Checking for namespace user : PASS
LXC: Checking for cgroup 'cpu' controller support : PASS
LXC: Checking for cgroup 'cpuacct' controller support : PASS
LXC: Checking for cgroup 'cpuset' controller support : PASS
LXC: Checking for cgroup 'memory' controller support : PASS
LXC: Checking for cgroup 'devices' controller support : PASS
LXC: Checking for cgroup 'freezer' controller support : PASS
LXC: Checking for cgroup 'blkio' controller support : PASS
LXC: Checking if device /sys/fs/fuse/connections exists : PASS
lscpu 显示 vmx 标志：
lscpu|grep vmx
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch topoext cpuid_fault invpcid_single pti tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt arat umip
创建虚拟机后，它将进入暂停模式并且无法恢复。
$ virsh list --all
Id Name State
---------------------
3 myvm paused
$ virsh resume myvm
error: Failed to resume domain myvm
error: internal error: unable to execute QEMU command 'cont': Resetting the Virtual Machine is required
这是 qemu 日志，显示 kvm 存在一些问题，硬件错误 0x7。
$ cat /var/log/libvirt/qemu/myvm.log
2018-12-19 15:34:27.814+0000: starting up libvirt version: 4.10.0, qemu version: 3.0.95openSUSE Leap 15.1, kernel: 4.12.14-lp151.19-default, hostname: susetest
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=spice /usr/bin/qemu-system-x86_64 -name guest=myvm,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-myvm/master-key.aes -machine pc-i440fx-3.1,accel=kvm,usb=off,dump-guest-core=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid ff9104ff-4992-4747-bbd4-33c8517a9570 -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=24,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/var/lib/libvirt/images/mydisk.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=27 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=be:a6:22:ed:1a:7b,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charchannel0 -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev pty,id=charconsole1 -device virtconsole,chardev=charconsole1,id=console1 -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -object rng-random,id=objrng0,filename=/dev/random -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x7 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
char device redirected to /dev/pts/1 (label charserial0)
char device redirected to /dev/pts/2 (label charchannel0)
char device redirected to /dev/pts/3 (label charconsole1)
KVM: entry failed, hardware error 0x7
答案1
您可能需要检查是否支持并启用了嵌套 KVM 虚拟化。要检查是否至少支持嵌套虚拟化，请尝试以下操作：
cat /sys/module/kvm_intel/parameters/nested
必须有Y
然后您需要激活（启用）嵌套 KVM 并配置 virt-manager。
尝试阅读文章，本文提供了如何执行此操作的分步说明。还有一些其他文章可供阅读。