Named Instance to use port 1433 but ssms needs to include port 1433 for remote connection

I have a named instance installed and have it set to use TCP port 1433 instead of Dynamic ports. The SQL browser is running and TCP IP is enabled. I have to put the port number 1433 on ssms and without including the port number it wont connect. Please advice what am I missing here. Is this a firewall issue or am I missing something.