How to prioritize SSD page cache eviction over HDD with slower speed?
2
votes
1
answer
56
views
I have a large slow HDD and a small fast SSD. This is about reads not [RAID](https://unix.stackexchange.com/q/471551/524752) . My desktop grinds to a near-halt when switching back to Firefox or man pages after (re/un)-loading 12+ GiB of Linux kernel build trees and 39 GiB total of different LLMs on the SSD while I only have 31 GiB of RAM:
$ free -h
total used free shared buff/cache available
Mem: 31Gi 10Gi 2.4Gi 1.0Gi 19Gi 20Gi
Swap: 0B 0B 0B
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 1.8T 0 disk
├─sda1 8:1 0 512M 0 part /boot/efi
├─sda2 8:2 0 1.7G 0 part /boot
└─sda3 8:3 0 1.8T 0 part
└─sda3_crypt 254:0 0 1.8T 0 crypt
├─vgubuntu-root 254:1 0 1.8T 0 lvm /
└─vgubuntu-swap_1 254:2 0 1.9G 0 lvm
nvme0n1 259:0 0 953.9G 0 disk
└─nvme0n1p1 259:1 0 100G 0 part
└─luks-... 254:3 0 100G 0 crypt /media/...
$ sysctl vm.swappiness
vm.swappiness = 60
The SSD is fast, so I'd rather Linux evict the SSD's page-cached files first. Its uncached read time is seconds anyway. What should stop is eviction of any file under /usr
or /home
. My man bash
and dpkg -S bin/bash
return instantly from the page cache, but uncached they take half a minute after exiting the LLMs. More severely, Firefox needs my ~/.mozilla
folder for history and cache; with it uncached, waiting for the address bar to work takes minutes.
I am looking for an configuration option. systemd-run
could set MemoryMax for ktorrent
, but I frequently restart llama-server
to switch between the ~6 GiB LLMs, and I don't want a separate daemon to keep the cgroup alive. The man
and dpkg
problems will be fixed when my /
moves to the SSD once I sort out fscrypt
fears; in the meantime, /usr
on tmpfs
would leave insufficient available RAM and overlayfs
is too much complexity. The LLM workload could, but shouldn't, remount the SSD as a workaround. That leaves the nice
d kernel build workload still evicting my web browsing one's cache.
I looked in /sys/block
but couldn't find the right config. [Cgroups v2](https://docs.kernel.org/admin-guide/cgroup-v2.html) has per-device options but only for parallel write workloads (io.max
) not for controlling how sequential workloads affect the cache. A [2011 patch](https://lore.kernel.org/lkml/4DFE987E.1070900@jp.fujitsu.com/T/) and a [2023 question](https://unix.stackexchange.com/q/755527/524752) don't see any userspace interface. Which setting can be used to force the SSD's page cache to be evicted before that of the HDD's?
Asked by Daniel T
(195 rep)
Apr 13, 2025, 10:44 PM
Last activity: Apr 14, 2025, 02:07 PM
Last activity: Apr 14, 2025, 02:07 PM