How to remotely connect to a Mac with FileVault where the OS does not start until a user account logs in?

I have a Mac Mini, M1, which I use remotely via SSH / VNC. From time to time I need to issues a reboot to the machine. After rebooting however, I am unable to establish an SSH connection as all I get are 'Connection Refused' packets. It seems that machine does not actually finish the boot process until first login; only after entering my password on the local machine does it start accepting SSH connections. How can I make the Mac actually finish booting without someone physically at the machine?