Cant get vfio-pci driver to load for nvidia GPU

Okay, I'm not getting any further so asking for help.. I've tried everything I can think of or find online. I'm trying to get the GPU passthrough working so I can use it in a VM with virt-manager/KVM. I followed this guide mainly (below) set all files, updated kernel and set grub lines. I cant get any output from dmesg | grep vfio following another question (below), so maybe that's a clue. One answer said vfio modules are integrated into the kernel, so lsmod wont show, and my kernel config file shows vfio entries. I've used pre: commands to try to load before the nvidia driver. I was able to use a blocklist.conf to block it, but my display card is nvidia also, and I couldnt get to a shell in recovery mode. https://github.com/NVIDIA/deepops/blob/master/virtual/README.md#bootloader-changes https://askubuntu.com/questions/1247058/how-do-i-confirm-that-vfio-is-working-in-20-04 --- lspci -nn | grep NVIDIA 03:00.0 VGA compatible controller : NVIDIA Corporation GF108GL [Quadro 600] [10de:0df8] (rev a1) 03:00.1 Audio device : NVIDIA Corporation GF108 High Definition Audio Controller [10de:0bea] (rev a1) 08:00.0 3D controller : NVIDIA Corporation GF110GL [Tesla M2090] [10de:1091] (rev a1) 08:00.1 Audio device : NVIDIA Corporation GF110 High Definition Audio Controller [10de:0e09] (rev a1) --- lspci -nnk -d 10de:1091 08:00.0 3D controller : NVIDIA Corporation GF110GL [Tesla M2090] [10de:1091] (rev a1) Subsystem: NVIDIA Corporation GF110GL [Tesla M2090] [10de:0887] Kernel driver in use: nvidia Kernel modules: nvidiafb, nouveau, nvidia --- "linux /boot/vmlinuz root=UUID=$uuid acpi=noirq intel_iommu=on iommu=pt vfio-pci ids=10de:1091,10de:0e09 vfio_iommu_type1 allow_unsafe_interrupts=1" --- I tried both vfio_iommu_type1 allow_unsafe_interrupts=1 and vfio_iommu_type1.allow_unsafe_interrupts=1. --- CONFIG_VFIO_IOMMU_TYPE1=y CONFIG_VFIO_VIRQFD=y CONFIG_VFIO=y CONFIG_VFIO_NOIOMMU=y CONFIG_VFIO_PCI=y CONFIG_VFIO_PCI_VGA=y CONFIG_VFIO_PCI_MMAP=y CONFIG_VFIO_PCI_INTX=y CONFIG_VFIO_PCI_IGD=y CONFIG_VFIO_MDEV=m CONFIG_VFIO_MDEV_DEVICE=m --- grep -oE 'svm|vmx' /proc/cpuinfo | uniq vmx --- cat /etc/modules # /etc/modules: kernel modules to load at boot time. # # This file contains the names of kernel modules that should be loaded # at boot time, one per line. Lines beginning with "#" are ignored. bonding pci_stub vfio vfio_iommu_type1 vfio_pci kvm kvm_intel --- cat /etc/modules-load.d/vfio-pci.conf vfio-pci --- cat /etc/modprobe.d/vfio.conf options vfio-pci ids=10de:1091,10de:0e09 options vfio_iommu_type1 allow_unsafe_interrupts=1 --- cat /etc/modprobe.d/nvidia.conf softdep nvidia_384 pre: vfio-pci #softdep radeon pre: vfio-pci #softdep amdgpu pre: vfio-pci softdep snd_hda_intel pre: vfio-pci softdep nouveau pre: vfio-pci softdep nvidia pre: vfio-pci softdep nvidia* pre: vfio-pci #softdep drm pre: vfio-pci #softdep xhci_hdc pre: vfio-pci #options kvm_amd avic=1 --- modprobe -c | grep vfio options vfio_pci ids=10de:1091,10de:0e09 options vfio_iommu_type1 allow_unsafe_interrupts=1 softdep mdev post: vfio_mdev softdep nvidia_384 pre: vfio-pci softdep snd_hda_intel pre: vfio-pci softdep nouveau pre: vfio-pci softdep nvidia pre: vfio-pci softdep nvidia* pre: vfio-pci --- cat /etc/initramfs-tools/modules # List of modules that you want to include in your initramfs. # They will be loaded at boot time in the order below. # # Syntax: module_name [args ...] # # You must run update-initramfs(8) to effect this change. # # Examples: # # raid1 # sd_mod vfio vfio_iommu_type1 vfio_pci vfio_virqfd vhost-net --- journalctl -b | grep vfio Dec 10 19:35:17 osboxes kernel: Command line: BOOT_IMAGE=/boot/vmlinuz root=UUID=ef2ecb3b-8e9a-4b20-bf15-47e0c7c98a1f acpi=noirq intel_iommu=on iommu=pt vfio-pci ids=10de:1091,10de:0e09 vfio_iommu_type1 allow_unsafe_interrupts=1 Dec 10 19:35:17 osboxes kernel: Kernel command line: BOOT_IMAGE=/boot/vmlinuz root=UUID=ef2ecb3b-8e9a-4b20-bf15-47e0c7c98a1f acpi=noirq intel_iommu=on iommu=pt vfio-pci ids=10de:1091,10de:0e09 vfio_iommu_type1 allow_unsafe_interrupts=1 Dec 10 19:35:17 osboxes systemd-modules-load: Module 'vfio' is built in Dec 10 19:35:17 osboxes systemd-modules-load: Module 'vfio_iommu_type1' is built in Dec 10 19:35:17 osboxes systemd-modules-load: Module 'vfio_pci' is built in Dec 10 19:35:17 osboxes systemd-modules-load: Module 'vfio_pci' is built in EDIT: yeah, after only blacklisting nouveau, which still caused no driver to be loaded, I removed the all the settings except blacklist nouveau, and even nvidia driver doesn't show.. take of that blacklist and everything is fine.