Use USB OTG mic audio device for audio input ONLY (use mic with headphones)

I want to record some audio using a USB mic and my OTG-supporting Android phone. The problem is, because the phone recognizes the mic as an audio input/output device even though the mic has no physical way to make sound, I can’t use the headphone jack while the mic is plugged in. Is there an app or a setting I can use to only use the input portion of the USB audio device, and instead use the 3.5mm headphone jack for output?