Prevent WhatsApp from restoring media from Google Drive Backup

I recently uninstalled and installed WhatsApp on my Samsung Galaxy S2, and it started to restore my chats. I'm fine with my chats but after being done with the chats, it automatically started to restore the old media too, which would cause my phone to download data of 2 GB. I want to stop it, but I can't find how to stop it. It used to restore only messages before having the Google Drive backup.