Prevent binding an iGPU to the host OS to use it later as PCI passthrough to a VM
0
votes
0
answers
120
views
I have an AMD iGPU (integrated in an AMD Ryzen 9 9950x CPU) that I want to use as the video output for a Windows VM in proxmox (I have no other GPU installed). I discovered that it doesn't have the Function Level Reset capability, so from my understanding I can't unbind it from the host OS and rebind it to the VM. I tried using vendor-reset, but it is not compatible with my iGPU. Then, I tried to prevent the iGPU from ever being initialised by the host OS, by blacklisting my GPU drivers (amdgpu and radeon) by creating a config file in
/etc/modprobe.d/
that looks like this:
blacklist amdgpu
blacklist radeon
blacklist vgaarb
blacklist efifb
blacklist vesafb
install amdgpu /bin/false
install radeon /bin/false
install vgaarb /bin/false
I tried to specify every possible restriction on any frame buffer or GPU driver, as well as giving full control of the GPU to the vfio driver. I hoped this would prevent any part of the OS from ever touching the iGPU, but vgaarb still sets it as the boot VGA device, only to deactivate it afterwards, as displayed by dmesg:
[ 0.472763] pci 0000:72:00.0: vgaarb: setting as boot VGA device
[ 0.472763] pci 0000:72:00.0: vgaarb: bridge control possible
[ 0.472763] pci 0000:72:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[ 0.490408] pci 0000:72:00.1: D0 power state depends on 0000:72:00.0
[ 0.491001] pci 0000:72:00.0: Adding to iommu group 35
[ 2.052946] vfio-pci 0000:72:00.0: vgaarb: deactivate vga console
My full kernel command line (as displayed by dmesg) is:
BOOT_IMAGE=/vmlinuz-6.8.12-5-pve root=ZFS=/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet amd_iommu=on iommu=pt nofb pcie_aspm=off nomodeset video=vesafb:off video=efifb:off video=vesa:off video=simplefb:off disable_vga=1 vga=none vgaarb=off video=off vfio-pci.ids=1002:13c0,1002:1640 vfio-pci.disable_vga=1 initcall_blacklist=sysfb_init,amd_iommu_v2_init modprobe.blacklist=amdgpu,radeon,snd_hda_intel,r8169 module_blacklist=amdgpu,radeon vgaarb.multi_gpu_switch=0 video.ignore_edd=1 vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 rd.driver.blacklist=amdgpu rd.driver.blacklist=radeon rd.driver.pre=vfio-pci pcie_acs_override=downstream,multifunction radeon.modeset=0 amdgpu.modeset=0 amdgpu.aspm=0
I can confirm that in the end the GPU uses vfio as its driver, but it already has been initialised by the host OS and starting the VM returns the error is already attached
The output of -nnk
:
72:00.0 VGA compatible controller : Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:13c0] (rev c1)
Subsystem: Gigabyte Technology Co., Ltd Device [1458:d000]
Kernel driver in use: vfio-pci
Kernel modules: amdgpu
The VM's configuration file:
bios: ovmf
boot: order=ide2;net0
cores: 8
cpu: host
efidisk0: local-zfs:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:72:00,pcie=1,x-vga=1
ide2: local:iso/windows-11.iso,media=cdrom,size=5695402K
machine: pc-q35-9.0
memory: 16384
meta: creation-qemu=9.0.2,ctime=1733853637
name: windows11
net0: virtio=BC:24:11:2B:1A:A2,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsi0: local-zfs:vm-100-disk-1,aio=native,iothread=1,size=128G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=e73054d1-f03b-4b84-8cdc-9115d85676fa
sockets: 1
tpmstate0: local-zfs:vm-100-disk-2,size=4M,version=v2.0
vmgenid: e16f7ba2-2469-4495-ab4f-7480c7f89041
args: -device vfio-pci,host=72:00.0,multifunction=on,x-vga=on,rombar=0 -device vfio-pci,host=72:00.1
What else could I try? Thank you in advance!
Asked by CiprianC
(1 rep)
Dec 25, 2024, 12:14 PM