KVM | QEMU(包括 virt-manager)在 Linux 主机单 GPU 直通 macOS 客户机时出现黑屏问题

KVM | QEMU(包括 virt-manager)在 Linux 主机单 GPU 直通 macOS 客户机时出现黑屏问题

好了,各位!需要提前知道的是,在扩展基线配置以进一步整合单 GPU 直通作为急需的性能改进手段之前,我的初始 VM 解决方案已在 Kali/Debian Linux 主机上安装了 macOS Ventura 客户机。

使用最新的 GNOME 和 Kali(基于 Debian)Linux,具有自定义编译的 6.2.2 内核。

硬件是配备 Haswell CPU 和 Intel i915 GPU(带 HDMI)的 MacBook Air 2013。

$ uname -a
Linux ... 6.2.2 #1 SMP PREEMPT Thu Mar  9 20:05:06 GMT 2023 x86_64 GNU/Linux

$ cat /proc/cpuinfo
...

model name  : Intel(R) Core(TM) i5-4250U CPU @ 1.30GHz

...

首先,以下是相关的 GRUB 2 配置:

$ grep LINUX_DEFAULT /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="nosplash loglevel=3 intel_iommu=on i915.enable_gvt=1 i915.enable_guc=0 vfio-pci.ids=8086:0a26,8086:0a0c video=efifb:off kvm.ignore_msrs iommu=pt”

此外,以下内容相应地验证了 IOMMU 配置:

$ dmesg | grep -i iommu
[    0.000000] Command line: BOOT_IMAGE=/BOOT/sunv2@/vmlinuz-6.2.2 root=ZFS=/ROOT/sunv2 ro root=ZFS=zroot/ROOT/sunv2 nosplash loglevel=3 intel_iommu=on i915.enable_gvt=1 i915.enable_guc=0 vfio-pci.ids=8086:0a26,8086:0a0c video=efifb:off kvm.ignore_msrs iommu=pt
[    0.038172] Kernel command line: BOOT_IMAGE=/BOOT/sunv2@/vmlinuz-6.2.2 root=ZFS=/ROOT/sunv2 ro root=ZFS=zroot/ROOT/sunv2 nosplash loglevel=3 intel_iommu=on i915.enable_gvt=1 i915.enable_guc=0 vfio-pci.ids=8086:0a26,8086:0a0c video=efifb:off kvm.ignore_msrs iommu=pt
[    0.038366] DMAR: IOMMU enabled
[    0.113621] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.255936] iommu: Default domain type: Passthrough (set via kernel command line)
[    0.293253] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.293257] DMAR: IOMMU feature sc_support inconsistent
[    0.293259] DMAR: IOMMU feature pass_through inconsistent
[    0.340122] pci 0000:00:02.0: Adding to iommu group 0
[    0.340249] pci 0000:00:00.0: Adding to iommu group 1
[    0.340278] pci 0000:00:03.0: Adding to iommu group 2
[    0.340305] pci 0000:00:14.0: Adding to iommu group 3
[    0.340349] pci 0000:00:15.0: Adding to iommu group 4
[    0.340372] pci 0000:00:15.4: Adding to iommu group 4
[    0.340404] pci 0000:00:16.0: Adding to iommu group 5
[    0.340430] pci 0000:00:1b.0: Adding to iommu group 6
[    0.340456] pci 0000:00:1c.0: Adding to iommu group 7
[    0.340480] pci 0000:00:1c.1: Adding to iommu group 8
[    0.340506] pci 0000:00:1c.2: Adding to iommu group 9
[    0.340532] pci 0000:00:1c.4: Adding to iommu group 10
[    0.340557] pci 0000:00:1c.5: Adding to iommu group 11
[    0.340599] pci 0000:00:1f.0: Adding to iommu group 12
[    0.340624] pci 0000:00:1f.3: Adding to iommu group 12
[    0.340648] pci 0000:02:00.0: Adding to iommu group 13
[    0.340678] pci 0000:03:00.0: Adding to iommu group 14
[    0.340703] pci 0000:05:00.0: Adding to iommu group 15
[    0.340731] pci 0000:06:00.0: Adding to iommu group 16
[    0.340757] pci 0000:06:03.0: Adding to iommu group 17
[    0.340784] pci 0000:06:04.0: Adding to iommu group 18
[    0.340809] pci 0000:06:05.0: Adding to iommu group 19
[    0.340834] pci 0000:06:06.0: Adding to iommu group 20
[    0.340845] pci 0000:07:00.0: Adding to iommu group 16
[    0.340871] pci 0000:04:00.0: Adding to iommu group 21

以下是从 virt-manager 启动 VM 时的相关日志输出(注意没有错误或失败):

$ sudo tail -f /var/log/libvirt/qemu/macOS.log     
2023-03-10 14:01:12.580+0000: starting up libvirt version: 9.0.0, package: 1 (Andrea Bolognani <[email protected]> Sat, 28 Jan 2023 17:03:53 +0100), qemu version: 7.2.0Debian 1:7.2+dfsg-4, kernel: 6.2.2, hostname: macbook-spellegrino.sunv2.com
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-1-macOS \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-macOS/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-macOS/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-macOS/.config \
/usr/bin/qemu-system-x86_64 \
-name guest=macOS,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-macOS/master-key.aes"}' \
-blockdev '{"driver":"file","filename":"/vm/macOS/git/OSX-KVM/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
-blockdev '{"driver":"file","filename":"/vm/macOS/git/OSX-KVM/OVMF_VARS-1024x768.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
-machine pc-q35-4.2,usb=off,dump-guest-core=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \
-accel kvm \
-cpu qemu64 \
-m 2048 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":2147483648}' \
-overcommit mem-lock=off \
-smp 2,sockets=2,cores=1,threads=1 \
-uuid 2aca0dd6-cec9-4717-9ab2-0b7b13d111c3 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
-device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
-device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
-device '{"driver":"pcie-root-port","port":11,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x3"}' \
-device '{"driver":"pcie-root-port","port":12,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x4"}' \
-device '{"driver":"pcie-root-port","port":13,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x1.0x5"}' \
-device '{"driver":"pcie-root-port","port":14,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x1.0x6"}' \
-device '{"driver":"pcie-root-port","port":15,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x1.0x7"}' \
-device '{"driver":"pcie-pci-bridge","id":"pci.9","bus":"pci.1","addr":"0x0"}' \
-device '{"driver":"ich9-usb-ehci1","id":"usb","bus":"pcie.0","addr":"0x7.0x7"}' \
-device '{"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pcie.0","multifunction":true,"addr":"0x7"}' \
-device '{"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pcie.0","addr":"0x7.0x1"}' \
-device '{"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pcie.0","addr":"0x7.0x2"}' \
-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.2","addr":"0x0"}' \
-blockdev '{"driver":"file","filename":"/vm/macOS/git/OSX-KVM/OpenCore/OpenCore.qcow2","aio":"threads","node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-2-storage","backing":null}' \
-device '{"driver":"ide-hd","bus":"ide.0","drive":"libvirt-2-format","id":"sata0-0-0","bootindex":2,"write-cache":"on"}' \
-blockdev '{"driver":"file","filename":"/vm/macOS/macOS.qcow2","aio":"threads","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":null}' \
-device '{"driver":"ide-hd","bus":"ide.1","drive":"libvirt-1-format","id":"sata0-0-1","bootindex":1,"write-cache":"on"}' \
-netdev '{"type":"tap","fd":"34","id":"hostnet0"}' \
-device '{"driver":"vmxnet3","netdev":"hostnet0","id":"net0","mac":"52:54:00:e6:85:40","bus":"pci.9","multifunction":true,"addr":"0x1"}' \
-chardev socket,id=charchannel0,fd=32,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-audiodev '{"id":"audio1","driver":"none"}' \
-device '{"driver":"vfio-pci","host":"0000:00:02.0","id":"hostdev0","bus":"pcie.0","multifunction":true,"addr":"0x3.0x1","rombar":1,"romfile":"/vm/macOS/i915/i915vbios.rom"}' \
-device '{"driver":"vfio-pci","host":"0000:00:03.0","id":"hostdev1","bus":"pcie.0","addr":"0x3.0x3","rombar":1}' \
-fw_cfg name=opt/igd-opregion,file=/vm/macOS/i915/opregion.bin \
-fw_cfg name=opt/igd-bdsm-size,file=/vm/macOS/i915/bdsmSize.bin \
-device 'isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc' \
-smbios type=2 \
-usb \
-device usb-tablet \
-device usb-kbd \
-cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check \
-sandbox off \
-msg timestamp=on

为了隔离问题,已应用以下权限放宽措施(直到找到完全可行的解决方案):

$ sudo grep -v \# /etc/libvirt/qemu.conf 
user = "+0"
group = "+0"
seccomp_sandbox = 0

$ id
...27(sudo),29(audio),30(dip),44(video),46(plugdev),102(input),104(kvm),106(netdev),120(libvirt)

$ sudo grep -v \# /etc/security/limits.conf 
@kvm            soft    memlock         unlimited
@kvm            hard    memlock         unlimited
@libvirt        soft    memlock         unlimited
@libvirt        hard    memlock         unlimited

$ sudo cat /etc/udev/rules.d/vfio-kvm.rules
SUBSYSTEM=="vfio", OWNER="root", GROUP=“kvm"
$ sudo udevadm control —reload
$ sudo udevadm trigger

VM 运行前/后的钩子如下:

$ tree /etc/libvirt/hooks
/etc/libvirt/hooks
├── qemu
└── qemu.d
    └── macOS
        ├── prepare
        │   └── begin
        │       └── start.sh
        └── release
            └── end
                └── stop.sh

7 directories, 3 files


$ cat /etc/libvirt/hooks/qemu
#!/bin/bash

GUEST_NAME="$1"
HOOK_NAME="$2"
STATE_NAME="$3"
MISC="${@:4}"

BASEDIR="$(dirname $0)"

HOOKPATH="$BASEDIR/qemu.d/$GUEST_NAME/$HOOK_NAME/$STATE_NAME"
set -e # If a script exits with an error, we should as well.

if [ -f "$HOOKPATH" ]; then
eval \""$HOOKPATH"\" "$@"
elif [ -d "$HOOKPATH" ]; then
while read file; do
  eval \""$file"\" "$@"
done <<< "$(find -L "$HOOKPATH" -maxdepth 1 -type f -executable -print;)"
fi


$ sudo cat /etc/libvirt/hooks/qemu.d/macOS/prepare/begin/start.sh 
#!/bin/bash
set -x

# Stop display manager
systemctl stop display-manager
    
# Unbind VTconsoles
echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind

# Unbind EFI Framebuffer
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

sleep 3

modprobe --remove-dependencies -f i915

modprobe --remove-dependencies -f snd_hda_intel

sleep 3

# Unload AMD kernel module
# modprobe -r amdgpu

# Detach GPU devices from host
# Use your GPU and HDMI Audio PCI host device
virsh nodedev-detach pci_0000_00_02_0
virsh nodedev-detach pci_0000_00_03_0

sleep 3

# Load vfio module
modprobe vfio-pci


$ cat /etc/libvirt/hooks/qemu.d/macOS/release/end/stop.sh 
#!/usr/bin/bash
set -x

# Attach GPU devices to host
# Use your GPU and HDMI Audio PCI host device
virsh nodedev-reattach pci_0000_00_02_0
virsh nodedev-reattach pci_0000_00_03_0

# Unload vfio module
modprobe -r vfio-pci

# Rebind framebuffer to host
echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind

modprobe i915
modprobe snd_hda_intel

# Bind VTconsoles
echo 1 > /sys/class/vtconsole/vtcon0/bind
echo 1 > /sys/class/vtconsole/vtcon1/bind

# Restart Display Manager
systemctl start display-manager

这些钩子脚本在执行时会产生以下输出(请注意,使用 解除 EFI 帧缓冲区绑定时似乎存在问题echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind):

$ sudo /etc/libvirt/hooks/qemu macOS prepare begin 
+ systemctl stop display-manager
+ echo 0
+ echo 0
+ echo efi-framebuffer.0
/etc/libvirt/hooks/qemu.d/macOS/prepare/begin/start.sh: line 12: echo: write error: No such device
+ sleep 3
+ modprobe --remove-dependencies -f i915
+ modprobe --remove-dependencies -f snd_hda_intel
+ sleep 3
+ virsh nodedev-detach pci_0000_00_02_0
Device pci_0000_00_02_0 detached

+ virsh nodedev-detach pci_0000_00_03_0
Device pci_0000_00_03_0 detached

+ sleep 3
+ modprobe vfio-pci
+ sleep 3

请注意,执行钩子脚本后,以下 VFIO 设备可用start.sh

$ ls -l /dev/vfio
total 0
crw-rw---- 1 root kvm  244,   0 Mar 10 14:32 0
crw-rw---- 1 root kvm  244,   1 Mar 10 14:32 2
crw-rw-rw- 1 root root  10, 196 Mar 10 14:08 vfio

钩子脚本似乎没有问题end.sh,它使系统恢复到主机操作系统已知的一致状态:

$ sudo /etc/libvirt/hooks/qemu macOS release end
+ virsh nodedev-reattach pci_0000_00_02_0
Device pci_0000_00_02_0 re-attached

+ virsh nodedev-reattach pci_0000_00_03_0
Device pci_0000_00_03_0 re-attached

+ modprobe -r vfio-pci
+ echo efi-framebuffer.0
/etc/libvirt/hooks/qemu.d/macOS/release/end/stop.sh: line 16: echo: write error: No such device
+ modprobe i915
+ modprobe snd_hda_intel
+ echo 1
+ echo 1
+ systemctl start display-manager

请注意,执行钩子end.sh脚本后,以前的 VFIO 设备现在不可用:

$ ls -l /dev/vfio
total 0
crw-rw-rw- 1 root root 10, 196 Mar 10 14:08 vfio

此外,请参考以下输出中的“正在使用的内核驱动程序”(在调用 VM 之前),在本例中是i915snd_hda_intel

$ lspci -v 
...

00:02.0 VGA compatible controller: Intel Corporation Haswell-ULT Integrated Graphics Controller (rev 09) (prog-if 00 [VGA controller])
    Subsystem: Apple Inc. Haswell-ULT Integrated Graphics Controller
    Flags: bus master, fast devsel, latency 0, IRQ 70, NUMA node 0, IOMMU group 0
    Memory at b0000000 (64-bit, non-prefetchable) [size=4M]
    Memory at a0000000 (64-bit, prefetchable) [size=256M]
    I/O ports at 2000 [size=64]
    Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
    Capabilities: <access denied>
    Kernel driver in use: i915
    Kernel modules: i915

00:03.0 Audio device: Intel Corporation Haswell-ULT HD Audio Controller (rev 09)
    Subsystem: Apple Inc. Haswell-ULT HD Audio Controller
    Flags: bus master, fast devsel, latency 0, IRQ 75, NUMA node 0, IOMMU group 2
    Memory at b0a10000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: <access denied>
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel

此外,请从以下输出(调用 VM 后)中再次引用“正在使用的内核驱动程序”,在这种情况下,它现在vfio-pci适用于前一个i915snd_hda_intel实例(因此我们绑定到正确的 GPU/PCI 直通驱动程序):

00:02.0 VGA compatible controller: Intel Corporation Haswell-ULT Integrated Graphics Controller (rev 09) (prog-if 00 [VGA controller])
    Subsystem: Apple Inc. Haswell-ULT Integrated Graphics Controller
    Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0, IOMMU group 0
    Memory at b0000000 (64-bit, non-prefetchable) [size=4M]
    Memory at a0000000 (64-bit, prefetchable) [size=256M]
    I/O ports at 2000 [size=64]
    Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
    Capabilities: <access denied>
    Kernel driver in use: vfio-pci
    Kernel modules: i915

00:03.0 Audio device: Intel Corporation Haswell-ULT HD Audio Controller (rev 09)
    Subsystem: Apple Inc. Haswell-ULT HD Audio Controller
    Flags: fast devsel, IRQ 16, NUMA node 0, IOMMU group 2
    Memory at b0a10000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: <access denied>
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel

为简洁起见,在解决此问题时还有更多输出可供参考:

$ cat /etc/modules-load.d/kvm.conf 
kvmgt
vfio-iommu-type1
mdev
vfio-pci


$ lsmod | grep -iE '(vfio|iommu|mdev|i915|snd_hda_intel)'
vfio_pci               24576  0
vfio_pci_core         131072  1 vfio_pci
snd_hda_intel          73728  1
snd_intel_dspcfg       40960  1 snd_hda_intel
snd_hda_codec         278528  3 snd_hda_codec_generic,snd_hda_intel,snd_hda_codec_cirrus
snd_hda_core          180224  4 snd_hda_codec_generic,snd_hda_intel,snd_hda_codec,snd_hda_codec_cirrus
snd_pcm               258048  3 snd_hda_intel,snd_hda_codec,snd_hda_core
snd                   200704  12 snd_hda_codec_generic,snd_seq,snd_seq_device,snd_hwdep,snd_hda_intel,snd_hda_codec,snd_timer,snd_pcm
mdev                   28672  1 kvmgt
vfio_iommu_type1       73728  0
vfio                   86016  4 vfio_pci_core,kvmgt,vfio_iommu_type1,vfio_pci
i915                 5607424  11 kvmgt
drm_buddy              28672  1 i915
i2c_algo_bit           24576  1 i915
drm_display_helper    274432  1 i915
cec                    98304  2 drm_display_helper,i915
ttm                   151552  1 i915
drm_kms_helper        323584  2 drm_display_helper,i915
drm                   974848  10 drm_kms_helper,kvmgt,drm_display_helper,drm_buddy,i915,ttm
video                  81920  1 i915

值得一提的是,从下面的输出来看,底层设备没有mdev_supported_types(如果我能获得有关这方面的更多见解和信息就好了):

$ ls -l /sys/devices/pci0000:00/0000:00:02.0/
total 0
-r--r--r-- 1 root root      4096 Mar 10 14:09 ari_enabled
drwxr-xr-x 3 root root         0 Mar 10 14:20 backlight
-r--r--r-- 1 root root      4096 Mar 10 13:59 boot_vga
-rw-r--r-- 1 root root      4096 Mar 10 14:09 broken_parity_status
-r--r--r-- 1 root root      4096 Mar 10 13:59 class
-rw-r--r-- 1 root root       256 Mar 10 13:59 config
-r--r--r-- 1 root root      4096 Mar 10 14:09 consistent_dma_mask_bits
lrwxrwxrwx 1 root root         0 Mar 10 14:20 consumer:pci:0000:00:03.0 -> ../../virtual/devlink/pci:0000:00:02.0--pci:0000:00:03.0
-rw-r--r-- 1 root root      4096 Mar 10 14:09 d3cold_allowed
-r--r--r-- 1 root root      4096 Mar 10 13:58 device
-r--r--r-- 1 root root      4096 Mar 10 14:09 dma_mask_bits
lrwxrwxrwx 1 root root         0 Mar 10 14:20 driver -> ../../../bus/pci/drivers/i915
-rw-r--r-- 1 root root      4096 Mar 10 14:20 driver_override
drwxr-xr-x 4 root root         0 Mar 10 14:20 drm
-rw-r--r-- 1 root root      4096 Mar 10 14:09 enable
lrwxrwxrwx 1 root root         0 Mar 10 14:09 firmware_node -> ../../LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00
drwxr-xr-x 3 root root         0 Mar 10 14:20 graphics
drwxr-xr-x 3 root root         0 Mar 10 14:20 i2c-1
drwxr-xr-x 3 root root         0 Mar 10 14:20 i2c-2
drwxr-xr-x 3 root root         0 Mar 10 14:20 i2c-3
drwxr-xr-x 3 root root         0 Mar 10 14:20 i2c-4
drwxr-xr-x 3 root root         0 Mar 10 14:20 i2c-5
drwxr-xr-x 3 root root         0 Mar 10 14:20 i2c-6
lrwxrwxrwx 1 root root         0 Mar 10 14:09 iommu -> ../../virtual/iommu/dmar0
lrwxrwxrwx 1 root root         0 Mar 10 13:59 iommu_group -> ../../../kernel/iommu_groups/0
-r--r--r-- 1 root root      4096 Mar 10 14:09 irq
drwxr-xr-x 2 root root         0 Mar 10 13:58 link
-r--r--r-- 1 root root      4096 Mar 10 14:09 local_cpulist
-r--r--r-- 1 root root      4096 Mar 10 14:09 local_cpus
-r--r--r-- 1 root root      4096 Mar 10 13:58 modalias
-rw-r--r-- 1 root root      4096 Mar 10 14:09 msi_bus
drwxr-xr-x 2 root root         0 Mar 10 14:20 msi_irqs
-rw-r--r-- 1 root root      4096 Mar 10 13:59 numa_node
drwxr-xr-x 2 root root         0 Mar 10 13:58 power
-r--r--r-- 1 root root      4096 Mar 10 14:09 power_state
--w--w---- 1 root root      4096 Mar 10 14:09 remove
--w------- 1 root root      4096 Mar 10 14:09 rescan
--w------- 1 root root      4096 Mar 10 14:09 reset
-rw-r--r-- 1 root root      4096 Mar 10 14:09 reset_method
-r--r--r-- 1 root root      4096 Mar 10 13:59 resource
-rw------- 1 root root   4194304 Mar 10 14:09 resource0
-rw------- 1 root root 268435456 Mar 10 14:09 resource2
-rw------- 1 root root 268435456 Mar 10 14:09 resource2_wc
-rw------- 1 root root        64 Mar 10 14:09 resource4
-r--r--r-- 1 root root      4096 Mar 10 13:59 revision
-rw------- 1 root root    131072 Mar 10 14:11 rom
lrwxrwxrwx 1 root root         0 Mar 10 13:58 subsystem -> ../../../bus/pci
-r--r--r-- 1 root root      4096 Mar 10 13:59 subsystem_device
-r--r--r-- 1 root root      4096 Mar 10 13:59 subsystem_vendor
-rw-r--r-- 1 root root      4096 Mar 10 13:58 uevent
-r--r--r-- 1 root root      4096 Mar 10 13:58 vendor

我所有最终问题的潜在原因可能都源于错误的 vBIOS ROM。首先,我尝试使用以下方法获取合适的 vBIOS ROM:

# echo 1 > /sys/bus/pci/devices/0000:00:02.0/rom 

# cat /sys/bus/pci/devices/0000:00:02.0/rom > rom
cat: '/sys/bus/pci/devices/0000:00:02.0/rom': Input/output error

# echo 0 > /sys/bus/pci/devices/0000:00:02.0/rom 

# cat /sys/bus/pci/devices/0000:00:02.0/rom > rom
cat: '/sys/bus/pci/devices/0000:00:02.0/rom': Invalid argument

但是,这种方法毫无用处(有人知道原因吗)?我确实尝试研究dd在内存区域使用 bit-nibbler 的方法,但没有找到任何特定于 i915 的方法。

当无法从逻辑设备树中获取合适的 vBIOS ROM 时,将克隆并编译以下内容:

https://github.com/patmagauran/i915ovmfPkg

相关文件包括以下内容(请注意,即使在构建目录中显示为 64 位,但大多数文件似乎都是 32 位,这令人困惑):

bdsmSize.bin:   data
i915ovmf.debug: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
i915ovmf.efi:   PE32+ executable (EFI boot service driver) x86-64, for MS Windows, 3 sections
i915ovmf.rom:   BIOS (ia32) ROM Ext. (111*512) instruction 0x00f10e00; at 0x1c PCI Intel device=0x1926 PRIOR, ProgIF=3, revision 3, code revision 0x3, code type 0x3, last ROM
i915vbios.rom:  BIOS (ia32) ROM Ext. (39*512) instruction 0x00f10e00; at 0x1c PCI UNKNOWN (0x1f96) device=0x0778 PRIOR, revision 3, code revision 0x3, code type 0x3, last ROM
opregion.bin:   data

所有这些 i915 相关文件(可互换使用)都在 QEMU/virt-manager XML 配置文件中引用,如下所示:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <rom bar='on' file='/vm/macOS/i915/i915vbios.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
      </source>
      <rom bar='on'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
    </hostdev>

...

  <qemu:commandline>
    <qemu:arg value='-fw_cfg'/>
    <qemu:arg value='name=opt/igd-opregion,file=/vm/macOS/i915/opregion.bin'/>
    <qemu:arg value='-fw_cfg'/>
    <qemu:arg value='name=opt/igd-bdsm-size,file=/vm/macOS/i915/bdsmSize.bin'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc'/>
    <qemu:arg value='-smbios'/>
    <qemu:arg value='type=2'/>
    <qemu:arg value='-usb'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-tablet'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-kbd'/>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check'/>
  </qemu:commandline>

有谁知道是什么原因导致我在一个经过充分测试的综合配置下出现黑屏问题?如果有人愿意解决这个问题并需要任何进一步的信息,请尽快告诉我,我将提供进一步排除故障和解决此问题所需的信息。非常感谢任何帮助解决问题的人。-STace

-- 对原始帖子进一步更新 -------

经过进一步调查,现在有了进一步的更新。

我重新编译了 i915 vBIOS ROM 源代码,但强制使用 64 位构建环境源配置,其中包含:

export CFLAGS=“-m64”
export CXXFLAGS=“-m64”
export LDFLAGS=“-m64”
export CPPFLAGS=“-m64”
export CC=“gcc -m64”
export CXX=“g++ -m64”

但是,这并没有改变构建输出二进制文件的文件类型!尽管如此,还是值得调查一下,以排除这是导致我的问题的原因。在浏览 i915 vBIOS ROM 的网站时,有一个小细节被忽略了,那就是根据……

https://github.com/patmagauran/i915ovmfPkg/wiki/Having-Problems%3F

第一点指出:

iGPU 必须位于 GUEST OS 中的地址 00:02.00。

因此,我修改了相关条目,具体如下:

<hostdev mode='subsystem' type='pci' managed=‘yes'>
  <driver name=“vfio”/>
  <source>
    <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
  </source>
  <rom bar='on' file='/vm/macOS/i915/i915vbios.rom'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x2' function='0x0' multifunction='on'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed=‘yes'>
  <driver name=“vfio”/>

  <source>
    <address domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
  </source>
  <rom bar='on'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</hostdev>

注意 XML 中添加了驱动程序标签。

这样就可以将地址从底层源硬件直接镜像到虚拟映射。但是,从 virt-manager 启动虚拟机时,在日志文件末尾发现了以下错误:

2023-03-11T01:18:04.390922Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:00:02.0","id":"hostdev0","bus":"pcie.0","multifunction":true,"addr":"0x2","rombar":1,"romfile":"/vm/macOS/i915/i915ovmf.efi"}: IGD device 0000:00:02.0 cannot support legacy mode due to existing devices at address 1f.0

最初,人们认为 1f.0 是虚拟地址映射,但事实并非如此。事实上,它在底层硬件上引用了以下内容:

00:1f.0 ISA bridge: Intel Corporation 8 Series LPC Controller (rev 04)

因此,下一个合乎逻辑的进展是进一步深入研究错误:

IGD device 0000:00:02.0 cannot support legacy mode due to existing devices at address 1f.0

这是产生该错误的 QEMU 源代码:

+ /*
+ * We need to create an LPC/ISA bridge at PCI bus address 00:1f.0 that we
+ * can stuff host values into, so if there's already one there and it's not
+ * one we can hack on, legacy mode is no-go. Sorry Q35.
+ */
+ lpc_bridge = pci_find_device(pci_device_root_bus(&vdev->pdev),
+ 0, PCI_DEVFN(0x1f, 0));
+ if (lpc_bridge && !object_dynamic_cast(OBJECT(lpc_bridge),
+ "vfio-pci-igd-lpc-bridge")) {
+ error_report("IGD device %s cannot support legacy mode due to existing "
+ "devices at address 1f.0", vdev->vbasedev.name);
+ return;
}

相关内容