Shared Google Calendar, but contacts are now also shared

I shared my Google Calendar with my family member, and he also shared his with me. Now, however, all of our contacts are also shared with each other. I don't want our contacts shared. He has a lot (400+) and I don't want to see all of his contacts mixed in with mine. Other solutions I have checked into don't seem to address this situation specifically. There has to be a setting somewhere that says 'don't share my contacts', but I can't find it. I have a Samsung Galaxy A42 and my family member has a Samsung Galaxy A25.