Libvirt Yubikey passthrough

So I've been trying to get a yubikey passthrough to work for a few days now with no luck. Does anyone know what I'm doing wrong or what's wrong with my setup? **Setup:**\ I'm using ubuntu 18.04 as both host and guest.\ Libvirt for virtualisation **Procedure:**\ I'm passing through the usb using the following hostdev section:

I boot the VM and verify that the usb was passed through properly with lsusb. (And it is)\ Then I try yubico-piv-tool -a status to see if the key is working. **Results:**\ yubico-piv-tool -a status returns "Failed to read device". **Observations:**\ The yubikey functions on the host, yubico-piv-tool -a status prints as expected.\ The yubikey still shows up on the host after the passthrough when I run lsusb on the host.\ The yubikey does not function on the host while the passthrough is active.\ I've tried to passthrough the yubikey on a laptop with a windows host on virtualbox and was successful with no extra settings.\ I tried virtualbox on the linux host and was not successful, same results as with libvirt. While the USB device is not passed through, lsusb -t shows that it's not assigned to any driver; when passed through, it's owned by the usbfs driver, and the libvirt-qemu user's qemu-syst process has the device open. **My guess:**\ There's missing parameters on the passthrough and the key isn't getting passed entirely (Something CCID/FIDO/OTP related?).\ The virtualization software was not able to disconnect the device from the host.