Reading GPG passphrase from the physical keyboard in OpenBSD

I'm using Synergy to control an OpenBSD desktop from a Linux computer (think of it like a VNC client/server). That is to say what I type on the keyboard goes through a network connection before it reaches the remote terminal. How do I make sure that gpg-agent reads a passphrase only when it's typed from the **physical** keyboard connected to the OpenBSD workstation? In other words, how do I make sure that stdin comes from the keyboard and **not** from synergy? I'm working on a worst-case scenario where the linux box I'm using to control the remote OpenBSD machine is compromised with a keylogger. Note that synergy works in a reversed client-server configuration, i.e. the OpenBSD machine is the synergy *client*, and the Linux box is the synergy *server*. I've tried exporting GPG_TTY to /dev/wskbd0 but to no avail. Is that even possible?