How to passthrough GPU in QEMU-KVM? Failing to start

I have tried countless guides and always ended up with the same result, so it's time I ask those who know what's going on. I have all virtualisation IOMMU etc enabled in my BIOS. I am running the VM without issues when not passing through the GPU, no errors, great performance etc. When I do try to passthrough the GPU (both GPU and same IOMMU group devices) and press start the VM, nothing happens. Literally nothing, as if I did not press start the VM. When I try to remove the PCI passthrough, virt-manager crashes. If I forcibly close it then start it again, I can't connect to the server (qemu://system) until I fully restart my PC (logout doesn't help). What am I missing? I have a second GPU of course, which is what I've connected my monitor to. I have Nvidia drivers installed. Both GPUs are recognised and functional (tested by switching the monitor). The GPUs are 3060 and 560. Using Debian 10 and everything is updated to the latest version.