How to reduce latency with fast short queries in Postgresql 15.3 on a NUMA server

We're running two different versions of Postgresql on two separate servers: - Server A: Postgresql 9.3 - Server B: Postgresql 15.3 Server B is way more powerful than server A: - Server A: 1x Intel Xeon E3-1270 3.5GHz, 2x 8GB DDR3, RAID0 - Server B: 2x Intel Xeon Platinum 8260 2.4GHz, 4x 16GB DDR4, RAID1 When we run fast short SELECT queries ( Index Only Scan using foobar_pkey on public.foobar (cost=0.42..8.44 rows=1 width=0) (actual time=0.008..0.008 rows=1 loops=1) Output: 1 Index Cond: (foobar.id = 1) Heap Fetches: 1 Buffers: shared hit=5 Total runtime: 0.017 ms (9 rows) Time: 0.281 ms

Server B:

# EXPLAIN (ANALYZE, BUFFERS, VERBOSE) SELECT 1 AS "a" FROM "foobar" WHERE ("foobar"."id" = 1) LIMIT 1; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------------------ Limit (cost=0.00..1.11 rows=1 width=4) (actual time=0.019..0.021 rows=1 loops=1) Output: 1 Buffers: shared hit=4 -> Index Only Scan using foobar_pkey on public.foobar (cost=0.00..1.11 rows=1 width=4) (actual time=0.018..0.018 rows=1 loops=1) Output: 1 Index Cond: (foobar.id = 1) Heap Fetches: 0 Buffers: shared hit=4 Planning Time: 0.110 ms Execution Time: 0.045 ms (10 rows) Time: 0.635 ms

RAID1 could add some latency on server B if we were reading from disk, but we've confirmed these queries are hitting the buffer/cache and therefore reading data from memory and not from disk. We've checked the hit rate with the following query:

sql SELECT 'cache hit rate' AS name, sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) AS ratio FROM pg_statio_user_tables; ``` The latency is quite low on both servers, but when you're running a bunch of fast short queries concurrently, on aggregate you see the difference, with server A being 0.1-1.0 seconds faster on average than server B. Server B has 2 CPUs and is using NUMA on Linux. Maybe that's adding some overhead/latency? Transparent Huge Tables is set to 'madvise' on Linux and huge_pages is set to try on Postgresql. Any ideas?