How to stop S Voice from starting every time the headphones is inserted to Samsung Galaxy S5?

The moment I insert my headphones to my Samsung Galaxy S5 to play music, the S Voice will auto appear and my music stops. I have to cancel it and after a while, the S Voice repeats itself. I've already deactivated the S Voice by double-tapping on the "home" key and unchecking “Open via the home key.” How do I permanently stop the S Voice from auto appearing? Is there another way to stop S Voice from appearing whilst I play music over the headphone.