How to enable gpg password prompt for OAuth2 token in mutt

I am using mutt with outlook365 which has OAuth2 support. I generated an access token which is checked by the following lines in my muttrc:

set imap_authenticators = "xoauth2"
set imap_oauth_refresh_command = "~/.config/neomutt/mutt_oauth2.py ~/.config/neomutt/mytoken.token"
set smtp_authenticators = ${imap_authenticators}
set smtp_oauth_refresh_command = ${imap_oauth_refresh_command}

Where mutt_oauth2.py is the script provided by mutt , and mytoken.token is a GPG encrypted token generated accordingly. My issue is that I have settings in my gpg-agent.conf for limited time caching of my gpg password for security reasons. If this times out and I try to send an email I am confronted with Authenticating (XOAUTH2)... followed by No authenticators available, and the email will not send. If I input my gpg password elsewhere on the system for other purposes (via pinentry-curses) and then try to send again the authentication works fine as the password is now (temporarily) in the cache. So my question is why does mutt not prompt me for my gpg password when trying to send, and how can I make it so that it does? Thanks in advance.