Bypass 2FA if the SSH key type is "-sk"

I have a server with Google Authentication + PAM set up so it requires a TOTP when connecting to it through SSH. But I would like to make an exception for SSH keys that are stored in a security key, is this possible? [This question](https://unix.stackexchange.com/questions/612443/help-with-bypassing-ssh-2-factor-authentication-when-on-same-local-network-macos) made me think that it is possible to have a separate authorized_keys for these exceptions and control this via PAM, but I don't know if it's possible to switch between them.