SSH using keys in external storage - permissions?

I have a set of SSH keys to login to accounts and servers. However, ever since I moved my main work profile to a USB drive (for sync and ease of use among other reasons) I have been unable to *directly* use the SSH keys in the profile for authentication, having to create local copies instead. Normal SSH workflow usually goes ssh-add $HOME/.ssh/id_server_key ssh username@domain Which works without problems. However, to prevent each machine from having a copy of my keys I moved them to an xexternal drive, USB formatted as vfat. Then workflow *should* become like this: ssh-add /run/media/myself/USB/.ssh/id_server_key ssh username@domain ...Which doesn't work. SSH complains about doing something to protect me:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for 'id_server_key' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: id_server_key
Permission denied (publickey).

Which, while I can understand, it's not *really* protecting me in this case. The USB drive automounts in my distro, with a fmask of (I think) 022. I could unmount and remount the USB drive with a fmask of 077 but that would (1) require superuser privilege each time and (2) affect every single file in the device, not only the SSH keys. I've tried creating a symlink to the key, even with previous use of umask as I've seen in some tips&tricks: cd .ssh umask 077 ln -s /run/media/myself/USB/.ssh/id_server_key But the permissions can not be used this way for a symbolic link, so I get nothing. So, what **options** do I have here available if the goal is to **not** have a physical copy of the key on the machine? One of the machines is a family laptop that goes to various places, so I have an interest in avoiding some of the keys being discovered. So far I've considered the following: * Is there for example an option to force SSH (ssh-add primarily) to accept the keys from the external device? (I presume there should be - most programs follow a "I-know-what-I'm-doing" flag, but so far I can't find it in the manpage. * Setting a bindfs over the .ssh/ directory, but the problem is that the directory is non-empty (as it contains known_hosts and other data). * fuse-zip-ing the keys over /dev/shm seems to be a possible path as well. **EDIT**: As per @Gilles 's comment, yes, bindfs solves the issue. After a checking of the manpage, initial testing succeeded using the following invocation: bindfs -n -r -p 0700 -o nonempty /run/media/myself/USB/.ssh/ ~/.ssh Which binds disallowing access from other users (apparently even from root!), read-only, with reflected 0700 permissions. That reflection was the part that I lacked when working with bindfs originally. The nonempty flag was needed because of preexisting files such as known_hosts. This preserves the original workflow exactly except for an extra warning messae about not being able to add information to known_hosts (which in the end I might not mind). Of course, I'm bindfs-ing to another directory now (in /dev/shm) as originally suggested to make things slightly easier. I'm not as conspiranoid so as to encfs my USB drive (yet) so privacy-wise this is good enough for me.