Disabling vim visual mode in /etc/vim/vimrc does not work

I found plenty of questions regarding disabling visual mode in vim but none that tackles my particular problem: I added set mouse-=a to my /etc/vim/vimrc file to disable visual mode for good. Thing is: That seems to do nothing. However when I put the exact same directive into my user's .vimrc file it works. Is this expected behaviour? Did I miss something? Has anyone a solution which doesn't involve managing a .vimrc file for each and every user? Thanks in advance! I am on Debian 10, fully updated by the way.