Mac firewall blocking incoming connections to app even though it's allowed

I have 2 executables, a server and client. The server just listens on port 44888 for UDP packets and displays them. The client sends UDP packets to an address and port. I determined that the Mac firewall was causing packets to get dropped before they reach the server, and so I disabled it and now they reach the server. My problem is that when I re enable the firewall and specify to allow incoming connections to the server executable, they are dropped again, so it's only working when the firewall is completely disabled. Does anyone know why this would be? They are just rudimentary C++ executables so they aren't signed applications or anything also.