Earphones stuck in call answer mode

So I've got this problem with my earphones where they're stuck in call answer mode. They have a button which when pressed should switch the music to the background when in call answer mode and back to normal when pressed again. Well now I have the problem where the only way to have them back to normal mode is to press and hold the button. I have no idea myself how to fix this.