Problem with ADB on my Nexus 4 (unauthorized phone)

When running adb devices in cmd, it says that my phone is unauthorized. It appears that it has something to do with the RSA fingerprint prompt. I don't remember ever getting such a prompt. Yes, I am running in PTP mode and debugging is enabled. I tried, after looking around on internet for multiple hours: * Enabling/disabling debugging mode repeatedly * Uninstalling and reinstalling my drivers for my phone or ADB * adb-killserver and adb-startserver * Restarting my phone and unplugging/replugging in multiple ways I did notice that in my (user)/.android/ folder, I don't have a file called adbkey.pub (which is apparently the file that's supposed to have my key). Anyway, I don't think I've ever received the prompt about RSA fingerprint and I can't make it appear in any way. Do you guys have any idea what I could do at this point. I tried about everything on these forums and around on the Internet but maybe there is something else I could do. Should I reset my phone to factory settings at this point to make the prompt appear again?