Mac OS Big Sur - mouse stutter (not a Bluetooth problem)

I'm experiencing really annoying mouse stutter on my MacBook Pro 16 inch. When I move the cursor it's smooth for a few seconds and then it suddenly stops for a split-second before moving again. This stutter/lag is there all the time. It started about a week ago and it doesn't stop. I restarted the computer, reset everything that is to reset on a Mac (except reinstalling the os) and it still persists. I googled that problem and almost all posts I found relate this to the Bluetooth connection. That's not the case for me because the stutter is also there when I use the built-in trackpad. Any ideas?