How to transfer conversation to a new number?

I have a contact who has a new phone number, so I update the old number with the new. But now when I check my sms conversations, I have 2 conversations: the "old" number conversation, and the "new" number conversation. Is there a way to fix this ? I already tried to merge them and then delete the old one but the old one is still linked to that old phone number. Thanks for your time