Control source IP of local DNS queries to local server?

I have a local DNS server running on my Linux router. I have it configured to only allow requests from my LAN (192.168.1.0/24). I also want the server to be able to query itself for DNS. To that end I did allow 127.0.0.1 as a source as well. The problem is that every query the box makes to itself has its source IP set to my external IP. I confirmed this with tcpdump; when the server queries itself at 127.0.0.1, a packet arrives on interface lo with the destination IP 127.0.0.1 but the source IP is that of my ISP. Using dig -b does not help. The same exact effect occurs. This means that unless I explicitly add the IP of my ISP to the allowed IPs, DNS lookup will not work locally. Since my IP can be dynamic, this actually means adding an entire range of IPs to the DNS server. This is obviously not a problem on machines on my LAN as they are setting their source IPs properly. The problem is specific to local queries on the server to itself. I want to be able to tell the server to use an explicit source IP address (not just a source interface necessarily) to make queries to itself. Can this be done?