Packets are unexpectedly routed to local lo interface

I have a question about routing between different network devices within the same Linux host: I created a Geneve network device (and this example is not related to Geneve tunnel; it's just that I happened to create the network link of Geneve type) and assigned an IP address to that device using the following command:

ip link add dev geneve0 type geneve dstport 6081 external
ip link set dev geneve0 up
ip addr add dev geneve0 10.0.0.1/30

When I checked the routing table and here are the outputs:

[root@9097d4d76e67 /]# ip route show table main
default via 172.19.0.1 dev eth0 
10.0.0.0/30 dev geneve0 proto kernel scope link src 10.0.0.1 
172.19.0.0/16 dev eth0 proto kernel scope link src 172.19.0.3

However, when tested out

[root@9097d4d76e67 /]# ip route get 10.0.0.1   
local 10.0.0.1 dev lo src 10.0.0.1 uid 0 
    cache

And it looks like the packet with daddr of 10.0.0.1 should be routed to the lo interface? Why is that? I did some research, and it seem relating to the local routing table

[root@9097d4d76e67 /]# ip route show table local
local 10.0.0.1 dev geneve0 proto kernel scope host src 10.0.0.1 
broadcast 10.0.0.3 dev geneve0 proto kernel scope link src 10.0.0.1 
...

So I guess kernel decide to use local routing table to route the packets, and I understand that local routing table has a higher priority. My question is: what I should do to force the packets being routed to geneve0 device? I tried deleting that rule from local routing table (I am not sure if I am supposed to do that):

[root@9097d4d76e67 /]# ip route del table local 10.0.0.1 dev geneve0
[root@9097d4d76e67 /]# ip route get 10.0.0.1                        
10.0.0.1 dev geneve0 src 10.0.0.1 uid 0 
    cache

Now ip route seems returns correct device However, when I tested out with real traffic, when the packets are reaching eth0, and then are DNAT to 10.0.0.1 ip ip table rule in PREROUTING stage, and the packets that are DNATed are not routed to geneve0 device as I would expect, anyone knows why? Is it related to ARP?