Using and debugging `overlayroot` in Ubuntu 24.04 EC2

I'm trying to use the overlayroot package in ubuntu 24.04 to boot into a read only root filesystem but have a writable overlay. Everything I see online says to update the /etc/overlayroot.conf file and then reboot. Here's what I have configured:

overlayroot=tmpfs:recurse=1,debug=1
overlayroot_cfgdisk="disabled"

But when I reboot and log in via SSH my / isn't overlayfs and there seems to be no logs indicating what the issue might be. The only thing I can think of is if the fact I'm doing this in EC2 making a difference.