Cannot remove module nvidia nvidia-uvm in order to install drivers

I was trying to update the drivers of a system with a couple of Nvidia GTX 980 cards but somehow I messed up and now I encounter this error when I run the installer with Nvidia: ERROR: An NVIDIA kernel module 'nvidia-uvm' appears to already be loaded in your kernel. This may be because it is in use (for example, by the X server), but may also happen if your kernel was configured without support for module unloading. Please be sure you have exited X before attempting to upgrade your driver. If you have exited X, know that your kernel supports module unloading, and still receive this message, then an error may have occured that has corrupted the NVIDIA kernel module's usage count; the simplest remedy is to reboot your computer. lsmod | grep -i nvidia gives: nvidia_uvm 77824 0 nvidia 8540160 77 nvidia_uvm drm 344064 4 nvidia So the suggestion that an error may have occured that corrupted the kernel module usage count makes sense, however, the remedy does not help and rebooting does not do a thing. I have tried blacklisting both modules in different ways and no matter what I do they are always back. Doing rmmod or modprobe -r does not help either. In fact with the later I get: modprobe: FATAL: Module nvidia-uvm not found. I have tried everything I found on the net, nothing changed that 77 there. Any ideas? Thanks!