No such device with Yubikey GnuPG

I have a yubikey 5 NFC with GPG keys configured that I use for encryption and signing. Usually, this works fine, but sometimes gpg will tell me that no card is connected; if I unplug and replug the key, it sometimes works again (although sometimes only after several tries). When it's not working, gpg --card-status will say gpg: selecting card failed: No such device, and the light on the key often (but not always) lights up permanently. This seems to happen at random, but usually after it's been plugged in for a while. One consistent way to reproduce this I've found is immediately after I log in with pam_u2f with the key, but only for a new X session (if I log in via su, for example, the key still works after). Restarting gpg does not fix the problem, only replugging (and that also only sometimes). This occurs both via usb and via NFC (HID Omnikey reader). Other functionality on the key (eg. u2f, WebAuthn) still works fine. Versions: - OS: OpenSuSE Leap 15.5 - GPG: 2.2.27 (from repo)