Memory runs out and PC freezes

I have been having issues playing video games on my PC, as I keep running out of RAM (8GB). When I run out of RAM, my displays and everything else freezes, except playing audio keeps looping. I have to force a shutdown and I often get banned from the game before I can rejoin. I want to solve this issue in some way, I'll write down my options in the order I prefer them in: 1. Prevent my PC from running out of RAM by reducing RAM usage safely somehow. Windows apparently already does this, scaling its RAM usage when running out. 2. Another option would be to only crash the game, not the whole OS when running out of memory. Some kind of script that kills the process using the most memory would do. This would at least prevent me from getting banned every time I run out of memory. 3. How viable is it to run a swapfile on a 130MBps max speed with 26ms delay HDD? Would I be able to keep playing or at least kill the game from htop running on my second display while running on swap memory?