Gateway failover using netplan

I'm having trouble setting up 2 IP address so that if one fails it will use the secondary IP address. I've setup netplan as shown in the picture. I can connect via ssh using both IP address, I test which IP address is the main one by using speedtest-cli, using the server control panel I disable the IP address that was used by speedtest-cli. However, when I try to use speedtest-cli again it doesn't work, I expected it to automatically switch to the IP that is still available. I tried pinging known IP address but it doesn't work. Does anyone know why this isn't working as I expect it to and how I can get it working?