How do I enable unprivileged_userns_clone selectively for one executable or user?

How do I enable CLONE_NEWUSER in a more fine-grained fashion compared to just kernel.unprivileged_userns_clone? I want to keep kernel API attack surface manageable by keeping new and complicated things like non-root CAP_SYS_ADMIN or BPF disabled, but also selectively allow it for some specific programs. For example, chrome-sandbox wants either CLOSE_NEWUSER or suid-root for proper operation, but I don't want all the programs to be able to use such complicated tricks, only a handful of approved ones.