Sample Header Ad - 728x90

How to allocate huge pages forcibly and synchronously?

2 votes
0 answers
305 views
On Linux, (non-transparent) huge pages may be allocated by writing into the vm.nr_hugepages sysctl, or, equivalently, into the /sys/kernel/mm/hugepages/hugepages-kB/nr_hugepages sysfs file:
# to allocate 2 GiB worth of huge pages, assuming a huge page size of 2 MiB (default on x86)
$ sysctl vm.nr_hugepages=1024

# likewise, but explicitly
$ echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages; grep . /sys/kernel/mm/hugepages/hugepages-2048kB/*
/sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages:1024
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages:1024
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages_mempolicy:1024
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_overcommit_hugepages:0
/sys/kernel/mm/hugepages/hugepages-2048kB/resv_hugepages:0
/sys/kernel/mm/hugepages/hugepages-2048kB/surplus_hugepages:0
However, if Linux fails to allocate the desired amount of hugepages, the rest will be skipped silently:
$ sysctl vm.nr_hugepages=1024
vm.nr_hugepages = 1024

$ sysctl vm.nr_hugepages
vm.nr_hugepages = 640
Chances of allocating the requested amount of huge pages on a long-running system can be raised somewhat by allocating in a loop, mixed with page cache reset requests and memory compaction requests:
while :; do
    sysctl vm.drop_caches=3
    sysctl vm.compact_memory=1
    sysctl vm.nr_hugepages=1024

    nr=$(sysctl -n vm.nr_hugepages)
    echo "vm.nr_hugepages = $nr"
    if (( nr >= 1024 )); then
        break
    fi
done
However, this is a dirty hack at best, and incurs unnecessary invalidation and thrashing of the entire page cache. --- Is there a less hacky method to request N huge pages to be allocated, automatically triggering any and all work necessary to fulfill the allocation (e. g. memory compaction, page cache reclaim, ...) and synchronously waiting for it to complete? In other words, how do I ask the kernel "do whatever you must to get me N huge pages, and do not return unless N huge pages are allocated or unless it has been firmly established that it is absolutely impossible to fulfill such an allocation"?
Asked by intelfx (5699 rep)
Feb 16, 2024, 09:21 PM