Unable to deny remote access to Postgresql even if there is no entry in pg_hba.conf

I have 5 Postgres Servers, with IP addresses let's say as A,B,C,D and E. A is the master server and remaining are the child servers which need to access A. In the postgresql.conf file of A, the listen address has been set to '*'. Now, let's say I want B to access A remotely. For that, the pg_hba.conf file of A should contain an entry with the IP address of B (along with other necessary data), like this for example: host all all IP_Address_of_B 32 md5 But, the problem which I am facing is that even if I am entering the details of any one of the child servers in the pg_hba.conf file of A, the access to A is enabled to all of the child servers. And if I remove that entry, then the access is disabled for all. As far as I know, if the entry of B is present in the pg_hba.conf file of A, the only B must be allowed to access A, not the remaining. But, in my case, either all have the access or none. Is their a way to enable only B (or only a selected servers, not all) to access A ?