Using WhatsApp iOS app without granting it access to contacts

I have avoided WhatsApp so far because I don't want to grant it/facebook permission to offload my contacts from my device into their cloud. However, this becomes a bit hard to sustain, so I was wondering whether I could perhaps be "passively" contacted on WhatsApp (instead of "actively" sending sending own messages) without such requirement. So I have now installed the app, without granting it permission to access my contacts. I was hoping that friends who already use the service and who have my phone number in their list of iOS contacts could now send me messages. However, this does not seem to be the case. Apparently, I do not show up among their "WhatsApp contacts". So my question is this: do both parties participating in a WhatsApp chat have to grant their apps access to their (full list of) contacts, or is there a way to participate in a more "private" way like I intended?