How to connect to a bluetooth device already connected to a smartphone

I have a Bluetooth headset (Soundcore Q30), and when it is connected to my computer, I can still connect to it via my Android smartphone and play sound on it. However, if I want to do the opposite and connect my computer to the headset when it is already connect to my phone (even if not playing sound), bluez fails with the error: SET_CONFIGURATION request rejected: Stream End Point in Use (19) The error makes sense, but I am suprised my computer cannot do what my phone can. Is there a way to bypass that? I am on Ubuntu 20.04 with Linux 5.15 and Bluez 5.53.