Mounting mergerfs directory with bind mount under lxc shows nothing unless root or 555 permissions

So I'm using proxmox and have a few hard drives mounted to /mnt/hdd1, /mnt/hdd2 etc I use mergerfs so that they all show up as one drive. I then use a bind mount so that I can access the dir when in various lxc containers. I have the folder permissions set to 550 however, when I try ls -la on the directory, all it returns is "total 0". I can view the directory with sudo ls -la though so it does mount. Changing the permissions to 555 lets me view the directory properly however I checked using id username and I am a member of the group the directory is owned by. Also, if I mount /mnt/hdd1 for example using the same method, I can access the directory with permissions 550. I tried recreating the group. Any ideas what is causing this? I can access this fine from the host as the same user. (Again just a member of the group) My mergerfs settings in /etc/fstab are defaults,allow_other,use_ino,func.getattr=newest. Using default_permissions instead of allow_other results in d?????????, but still works for root.