What is the best way to make LDAP based TOTP only required in some cases?

I have a 2FA setup on my network for certain kinds of authentication, but not others. The way it currently works is a combination of pam_ldap for passwords, and pam_oath for TOTP. PAM is configured in a way that it requires only pam_ldap for local login/unlock, only pam_oath for SSH (because that already requires publickey), and both for sudo and su. The problem with this setup is that pam_oath reads from /etc/users.oath. I need to periodically regenerate this file based on LDAP contents. Even worse, it also writes to this file, which I currently just ignore. There is a TOTP module for OpenLDAP I want to try, called [slapo-otp](https://man7.org/linux/man-pages/man5/slapo-otp.5.html) . This would solve the problem of having to keep the file in sync, but I would lose the option to skip it in some cases. I don't want to use it for unlocking the screen every time, people will just turn off the screen locker, honestly even I would do that. Please give me some ideas.