Why is my ISP DNS still in resolv.conf after a VPN connection and how can this be fixed?

Ubuntu 15.10 and dns=dnsmasq is commented out in /etc/NetworkManager/NetworkManager.conf Before I connect to a VPN /etc/resolv.conf contains nameserver 2xx.xx.xx.xx <-- ISP DNS 1 nameserver 2xx.xx.xx.xx <-- ISP DNS 2 after a VPN connection /etc/resolv.conf contains nameserver 1xx.xx.xx.xx <-- VPN DNS 1 nameserver 1xx.xx.xx.xx <-- VPN DNS 2 nameserver 2xx.xx.xx.xx <-- ISP DNS 1 The regular wired connection and the VPN have DNS servers set in network manager with automatic (only addresses). The ISP server shouldn't be there at all. What else can I change? (removing dns=dnsmasq was one change to stop split DNS).