How to allow rsync via ssh to a specific directory only

I want to allow moving files to a specific directory on my server using rsync + ssh. However, I don't want to fully trust the users using that SSH user. One solution I found is to set the shell of the user to rssh which can be configured to only allow sftp, rsync etc. However, in this case, the user would still be able to pull any readable files from the server, such as configurations in /etc which I don't want. I'm currently hesitating to go over my full directory structure and revoking the access for "others". Is there a way to allow a user to use rsync via ssh but only from / to a specific directory? I've seen that it seems possible to jail the SFTP access of openssh:

Match Group sftponly
  ChrootDirectory %h
  ForceCommand internal-sftp
  AllowTcpForwarding no
  X11Forwarding no
  PasswordAuthentication no

However, I would prefer rsync, as this account is used to upload bigger data and the internet connections are somewhat unstable (rural area with bad internet). rsync has proven very effective with all its features of continuing cancelled uploads.