USB audio devices not outputting audio unless leaving pulseaudio control applet open

I am having intermittent issues with the output going silent on a USB sound card. Note that this affects both an Apple audio dongle and a Ugreen sound card, two very different chipsets, in the same way, ruling out a hardware issue. I noticed that unplugging the USB cable and plugging it into another USB socket often fixes the problem, but not always. Rebooting also fixes it temporarily. Sometimes, I can be listening to audio, and I momentarily pause my video or audio app or even skip/scrub backwards and then the problem will come back and all audio will be silent. Weirdly, I've noticed that if the Pulseaudio volume control applet is open, the problem for now isn't occurring, and sometimes, if the problem happens and I start that applet, sometimes it fixes the issue (but not always, and sometimes nothing I do in the pulseaudio applet can fix it). Things I've checked: - Pulseaudio is showing correct output device, and the VU meter is showing the audio is playing on it, and it's not muted, but nothing is heard. - Alsamixer shows nothing is muted on the relevant audio device - It affects various software that plays audio, and when it happens, it appears to happen to all software that is playing to that device. - lsusb and similar show nothing odd about the USB device, which is still recognised, and there is nothing in dmesg to indicate a problem with it. - Even when the problem of no audio output is occurring, pulseaudio is correctly detecting the device being plugged and unplugged and the type of device it is. Using Debian stable with Gnome desktop, pulseaudio config is pretty vanilla