Android pings wrong network

My phone (Google Pixel 6 and Android 15) is connected to both 4G and a Wifi network with no Internet. The wifi network gives me IP address 192.168.4.2 and I know there's a WIFI Access point on 192.168.4.1. The 4G network gives me IP address 10.96.106.X. When I am connected to both networks I go to Termux and I try:

ping 192.168.4.1

but ping is unsuccessful - all the packets time out. Then I disconnect the 4G and I try again:

ping 192.168.4.1

and then the ping is successful. It seems like Android is thinking "The only way to perform the ping is go through the 4G network because your Wifi doesn't have internet", so it does the ping over the 4G network and hence it is unssuccesful. But that's wrong - I'm trying to ping an internal IP address on the wifi network so it should do the ping succesfully. Any suggestions? P.S. I can manually specify the interface like this:

ping -I wlan0 192.168.4.1

and then ping works fine, but unfortunately I'm building a flutter app and the APIs I am using does not allow specifying the interface name. To me it looks like a bug in Android.