CPU affinity not following cpuset

When I run taskset -p of a process I am getting something like this back:

# taskset -p 1078
pid 1078's current affinity mask: 3f

And it keeps changing what it reports, sometimes it's 5f, other times df and so on. For the same process I can see that its allowed on all cores:

# cat /proc/1078/status | grep Cpus
Cpus_allowed:   ff
Cpus_allowed_list:      0-7

And its cpuset in cgroups also allows 0-7:

# cat /dev/cpuset/cpus 
0-7

If I try to set it to ff using taskset I still do not get ff back:

# taskset -p ff 1078
pid 1078's current affinity mask: 5f
pid 1078's new affinity mask: 5f

What mechanism is overriding the cpuset and taskset affinity? Any way I can force it to actually run on all cores? This is on Android 13 and kernel 5.15.