Loading OpenSSH certificate into ssh-agent without the private key

I'm storing my ssh keys on a yubikey and hence I don't have any private key file on disk. This gives me a problem when I'm also using OpenSSH Certificates to authenticate. If I would like to bring the certificate with me using the ssh-agent I need to add it to the agent some how. This is done automatically if I have a private key called priv and a cert called priv-cert.pub. But since I don't have a file I cant find a way to add the certificate file to the agent. Does anyone have a clue how to do this? It seems there is no support for this, I found this feature request: https://bugzilla.mindrot.org/show_bug.cgi?id=2472