Postgresql: Why is a join or subquery so much slower than a literal in a where clause?
4
votes
1
answer
155
views
I'm using PostgreSQL 14.17.
My database schema has two tables:
Table "public.log_records"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------------+--------------------------+-----------+----------+------------------------------+----------+-------------+--------------+-------------
id | bigint | | not null | generated always as identity | plain | | |
log_id | bigint | | | | plain | | |
payload | jsonb | | not null | | extended | | |
created_at | timestamp with time zone | | | CURRENT_TIMESTAMP | plain | | |
Indexes:
"log_records_pkey" PRIMARY KEY, btree (id)
"idx_log_records_log_created" btree (log_id, created_at)
"idx_log_records_log_id" btree (log_id)
Foreign-key constraints:
"log_records_log_id_fkey" FOREIGN KEY (log_id) REFERENCES logs(id)
Access method: heap
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
------------------+--------------------------+-----------+----------+------------------------------+----------+-------------+--------------+-------------
id | bigint | | not null | generated always as identity | plain | | |
name | character varying(255) | | not null | | extended | | |
retention_period | integer | | not null | | plain | | |
created_at | timestamp with time zone | | | CURRENT_TIMESTAMP | plain | | |
Indexes:
"logs_pkey" PRIMARY KEY, btree (id)
"idx_logs_id_name" btree (id, name)
"logs_name_key" UNIQUE CONSTRAINT, btree (name)
Referenced by:
TABLE "log_records" CONSTRAINT "log_records_log_id_fkey" FOREIGN KEY (log_id) REFERENCES logs(id)
Access method: heap
There are a small number of records in logs and not all of the log_ids are used in the log_records table:
=# select count(*) from logs;
count
-------
13
(1 row)
=# select count(*) from log_records;
count
----------
14552534
(1 row)
=# select count(distinct log_id) from log_records;
count
-------
8
(1 row)
I want to query the log_records table for log records belonging to a named log. However, if I pass the name of a log that has not got any records in the log_records table the query is very slow:
# explain analyze
SELECT e.id, e.payload, e.created_at, e.headers
FROM log_records e JOIN logs l ON l.id = e.log_id
WHERE l.name = 'Log1'
ORDER BY e.id
LIMIT 100;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.43..244.61 rows=100 width=395) (actual time=7371.696..7371.698 rows=0 loops=1)
-> Nested Loop (cost=0.43..2729140.04 rows=1117680 width=395) (actual time=7371.696..7371.697 rows=0 loops=1)
Join Filter: (e.log_id = l.id)
Rows Removed by Join Filter: 14552570
-> Index Scan using log_records_pkey on log_records e (cost=0.43..2511191.29 rows=14529839 width=403) (actual time=0.012..4955.006 rows=14552570 loops=1)
-> Materialize (cost=0.00..1.17 rows=1 width=8) (actual time=0.000..0.000 rows=1 loops=14552570)
-> Seq Scan on logs l (cost=0.00..1.16 rows=1 width=8) (actual time=0.005..0.005 rows=1 loops=1)
Filter: ((name)::text = 'Log1'::text)
Rows Removed by Filter: 12
Planning Time: 0.233 ms
Execution Time: 7371.717 ms
(11 rows)
I get a different plan and better but still poor performance for a scalar subquery:
# explain analyze
SELECT e.id, e.payload, e.created_at, e.headers
FROM log_records e
WHERE e.log_id =
(SELECT id FROM logs WHERE name = 'Log1')
ORDER BY e.id
LIMIT 100;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=1.60..141.86 rows=100 width=395) (actual time=3686.420..3686.421 rows=0 loops=1)
InitPlan 1 (returns $0)
-> Seq Scan on logs (cost=0.00..1.16 rows=1 width=8) (actual time=0.007..0.008 rows=1 loops=1)
Filter: ((name)::text = 'Log1'::text)
Rows Removed by Filter: 12
-> Index Scan using log_records_pkey on log_records e (cost=0.43..2547511.66 rows=1816230 width=395) (actual time=3686.419..3686.419 rows=0 loops=1)
Filter: (log_id = $0)
Rows Removed by Filter: 14552594
Planning Time: 0.318 ms
Execution Time: 3686.487 ms
(10 rows)
However if I hardcode the log_id as a literal I get good performance:
# explain analyze
SELECT e.id, e.payload, e.created_at, e.headers
FROM log_records e
WHERE e.log_id = 13
ORDER BY e.id
LIMIT 100;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=4.08..4.09 rows=1 width=395) (actual time=0.015..0.015 rows=0 loops=1)
-> Sort (cost=4.08..4.09 rows=1 width=395) (actual time=0.014..0.015 rows=0 loops=1)
Sort Key: id
Sort Method: quicksort Memory: 25kB
-> Index Scan using idx_log_records_log_id on log_records e (cost=0.43..4.07 rows=1 width=395) (actual time=0.011..0.011 rows=0 loops=1)
Index Cond: (log_id = 13)
Planning Time: 0.098 ms
Execution Time: 0.028 ms
(8 rows)
Why can't I get this type of performance from the joins or subquery? Why is the subquery faster than the join when most advice would say don't use subqueries?
Asked by willn-cpx
(43 rep)
Aug 1, 2025, 10:24 AM
Last activity: Aug 6, 2025, 12:50 AM
Last activity: Aug 6, 2025, 12:50 AM