High RAM and swap usage out of seemingly nowhere

(Manjaro 20, Linux 5.8.3, KDE, [this laptop](https://geizhals.de/schenker-xmg-a507-vsy-10504411-a1686447.html)) When reasonably much is going on in my system, the RAM and swap usage often goes much higher than it should. For example currently I have a VM with a VM in it running and two instances of Minecraft, also some smaller stuff like music. That might sound like much, but my CPU is totally fine and even the sum of all the RAM usage shown in the task manager seems to be less than 2GB. Despite this, almost all of my 16GB RAM and 16GB swap are in use by… something. This is the output of free:

total        used        free      shared  buff/cache   available
Mem:          15898       15218         151         305         527          92
Swap:         17490       16442        1047

This much calmer picture is seen in the task manager:  I've read [here](https://unix.stackexchange.com/a/261329/216015) that sometimes virtualisation causes weird RAM issues, but my outer VM is restricted to 8GB. Even if it somehow used all of that without showing it in the task manager, it would still not explain about a quarter of my RAM usage and none of the swap usage. free shows that I should not blame caching (for once). I've heard that disk I/O that can't be done in time gets queued up in RAM, but iotop shows nothing overly active. This is also not just a measurement error, I do indeed get lag spikes in pretty much everything due to this. So quite a portion of memory of the programs I'm actively using is in swap. What uses this additional RAM and why so much of it?