vim pastes onto completely wrong location

This has not just once, but happened many times. I'm not sure what I'm doing wrong.. it tends to occur more frequently during remote control. I would set the mouse cursor at a certain position where I would paste, then it either goes to somewhere really far down or somewhere up.. until after meaningless trial/error(of which pattern I do not understand) it will at one point find where I'm looking for. Does anyone know why this is so? As I tried to recreate for capturing of screenshots, it now works fine, again with no logic -_-