How can I speed up my query that involves `max(timestamp)` in a huge table? I already added an index to every field
0
votes
1
answer
970
views
I have a huge table that has fields
ip
, mac
, and timestamp
.
Neither of the three fields is unique, but the combination of all three is.
Table is automatically populated, with newer records added all the time. The field timestamp
refers to when a row was added. Records are never *UPDATE*d.
Here's the table description:
Column | Type | Nullable | Default
-----------+-----------------------------+----------+--------
event | text | not null |
ip | inet | not null |
mac | macaddr8 | not null |
timestamp | timestamp without time zone | not null | now()
Indexes:
"ip_idx" btree (ip)
"mac_idx" btree (mac)
"time_idx" btree ("timestamp")
"timestamp_ip_event_key" UNIQUE CONSTRAINT, btree ("timestamp", ip, event)
I have this very slow query, causing the website to take very long time to load
How can I speed it up?
Is it possible to take advantage of the fact that the table is basically ordered by timestamp?
I do not have access to the script that adds records.
Executed SQL
select ip,
max(timestamp)
from my_table
WHERE ip Gather Merge (cost=291919.94..292169.16 rows=2136 width=15) (actual time=696.220..704.558 rows=429 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Sort (cost=290919.92..290922.59 rows=1068 width=15) (actual time=679.313..679.325 rows=143 loops=3)
Sort Key: ip
Sort Method: quicksort Memory: 31kB
Worker 0: Sort Method: quicksort Memory: 31kB
Worker 1: Sort Method: quicksort Memory: 31kB
-> Partial HashAggregate (cost=290855.52..290866.20 rows=1068 width=15) (actual time=679.192..679.233 rows=143 loops=3)
Group Key: ip
Batches: 1 Memory Usage: 81kB
Worker 0: Batches: 1 Memory Usage: 81kB
Worker 1: Batches: 1 Memory Usage: 81kB
-> Parallel Bitmap Heap Scan on my_table (cost=12023.68..289019.89 rows=367126 width=15) (actual time=67.898..580.432 rows=312819 loops=3)
Filter: (ip Bitmap Index Scan on my_table_ip_idx (cost=0.00..11803.41 rows=881097 width=0) (actual time=62.721..62.721 rows=938457 loops=1)
Index Cond: ((ip > '10.38.69.0/24'::inet) AND (ip <= '10.38.69.255'::inet))
Planning Time: 1.049 ms
JIT:
Functions: 30
Options: Inlining false, Optimization false, Expressions true, Deforming true
Timing: Generation 2.180 ms, Inlining 0.000 ms, Optimization 1.470 ms, Emission 29.303 ms, Total 32.952 ms
Execution Time: 726.126 ms
Asked by Granny Aching
(393 rep)
Aug 11, 2023, 01:56 AM
Last activity: Feb 13, 2025, 03:04 AM
Last activity: Feb 13, 2025, 03:04 AM