2nd display repeatedly loses connection when adb bridge is runnning

I am currently working on a project which includes smartphone instrumentation via FRIDA, which requires me to attach to the device via adb. Somehow, once the adb bridge is online my 2nd display keeps losing the connection, which results in the screen to turn of for 2-3 seconds every 10-15 seconds which is quite annoying. I am working on a laptop with Ubuntu 20.04 and have two external displays attached to it. Any help is appreciated.