why ulimit restriction is affect by other processes?

I have some processes creating large amount of threads (using python ray module), like 32 (proc) * 120 (thread per proc). Then I found other processes would fail on creating threads while their nTH is actually small (like creating a new ssh localhost connection, or using top in other shell), far less than the typical restriction 4096 NPROC from ulimit -a. I wonder if the ulimit is a user-scope restriction or process-scope restriction? If the latter, why would it be affected by other processes?