Play/Pause button on headphones works for a time, then stops

I have a Galaxy Note 3, and the play/pause button on the wired headset only works for a short time. Once I start playing music or whatever, with Ginko/Music/Play Music, the button works fine. After listening to a song or two however, pushing the button has no effect and I have to pull my phone out to play or pause. Has anyone else had this problem, or is it something I should be submitting for repair? Thanks.