VNC input extremely laggy on Linux Mint install

I'm running a headless Linux Mint (19.3) server for Plex and Homebridge. I only manage it via SSH or VNC. Currently running x11vnc (0.9.13) with the following config -noxrecord -ncachecr -auth guess -forever -noxdamage -repeat -rfbauth /etc/x11vnc/vncpwd -rfbport -shared Everything works fine except input lag on VNC is nearly unbearable. Typing is about a character every 2 seconds and mouse movement/clicks are also very slow. When the screen changes e.g. showing some sort of animation, it seems to be at a near normal pace that I'd expect for VNC. Also when I SSH to the machine, everything is perfectly speedy.