Testing for dir via ssh script - nothing happens

I usually run server scripts like this, with a here-document. But when I test if a directory exists, the script continues even though the test fails -- why?

ssh $user@$ip /bin/bash  ~/error.txt
  exit 1
fi

SCRIPT

I run this from my local computer and the server in question is a VM.