Merge Replication Subscriber does not Expire or cleaned up

I have a merge repliaction with retention period set to 1 day. Allow anonymous Subscriptions is set to False. I Add 2 subscribers. One of the subscriber machine name is wrong, so it is inaccessible. On Repliaction monitor, this inaccessible subscription keeps trying to connect and fails with could not connect to subscriber error. It has been days, but this subscriber still exists. It does not expire. Running Expired subscription clean up job succeed, but the subscription does not got removed from subscribers. What is the possible reason for this not to work?