Shrunk virtual disk file, now reported file size doesn't match

I originally had a single virtual disk file (.qcow2) of roughly 3.5 TiB and I decided to "group" all similar data on its own virtual disk file, for various reasons. Obviously that meant I could shrink the original disk file (to 900 GiB) but ls, stat and friends still list the original 3.5 TiB:

$ ls -lhks mydisk.qcow2 
722G -rw-r--r-- 1 root root 3.5T Aug  6 18:06 mydisk.qcow2

Of course I shrunk the filesystem itself (ext4) and sparsified the file afterwards, which is why only 722 GiB is currently allocated. But my backup tools also still detect 3.5 TiB and so will scan it all while the disk itself is only capable of holding 900 GiB, meaning it takes almost 4 times longer to finish than it has to. How can I "refresh" the reported sizes? This is a machine with HDDs so I'm thinking maybe the file is too fragmented and there's some stuff around the 3.5 TiB mark? But wouldn't copying the file fix that automatically (I tried and it didn't, at least)? Also, the disk file itself resides on ZFS, if that matters. If at all possible any solution would preferably work __in-place__. Shutting down the VM using the disk for a little while is not a problem.