Problem with indexing null values
-1
votes
1
answer
1121
views
There is a table 'test' with ~ 10M records with field 'finish'.
select count(*) from test where finish is null;
count
---------
2485009
(1 row)
I'd like to create index to speed up queries like
select * from test where finish is null;
Just for testing purpose the following indexes were created:
create index idx_t_0 on test(finish);
create index idx_t_1 on test((finish is null));
create index idx_t_2 on test(id) where finish is null;
gist=# set track_io_timing=on; [16/1920]
SET
gist=# EXPLAIN (ANALYZE, BUFFERS) select * from test where finish is null;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------
Seq Scan on test (cost=0.00..2829257.75 rows=2493440 width=1249) (actual time=0.031..23673.804 rows=2485009 loops=1)
Filter: (finish IS NULL)
Rows Removed by Filter: 6686627
Buffers: shared hit=129944 read=2606747
I/O Timings: read=15210.163
Planning time: 2.294 ms
Execution time: 23847.759 ms
(7 rows)
gist=# set enable_seqscan=off;
SET
gist=# EXPLAIN (ANALYZE, BUFFERS) select * from test where finish is null;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------
--------
Bitmap Heap Scan on test (cost=40312.83..2851483.71 rows=2493440 width=1249) (actual time=362.433..43436.427 rows=2485009 l
oops=1)
Recheck Cond: (finish IS NULL)
Rows Removed by Index Recheck: 3622568
Heap Blocks: exact=14958 lossy=891618
Buffers: shared read=913368
I/O Timings: read=23970.878
-> Bitmap Index Scan on idx_t_2 (cost=0.00..39689.47 rows=2493440 width=0) (actual time=357.046..357.046 rows=2485009 loops=1
)
Buffers: shared read=6792
I/O Timings: read=50.203
Planning time: 0.256 ms
Execution time: 43688.891 ms
(11 rows)
gist=# set enable_bitmapscan = off;
SET
gist=# EXPLAIN (ANALYZE, BUFFERS) select * from test where finish is null;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------
----------
Index Scan using idx_t_1 on test (cost=0.43..5670318.77 rows=2493440 width=1249) (actual time=2.049..14702.455 rows=2485009
loops=1)
Index Cond: ((finish IS NULL) = true)
Filter: (finish IS NULL)
Buffers: shared read=913369
I/O Timings: read=9994.256
Planning time: 0.227 ms
Execution time: 14875.190 ms
(7 rows)
**Config**
listen_addresses ='*'
port=5432
max_connections = 100
work_mem = 13107kB
shared_buffers = 2GB
effective_cache_size = 6GB
maintenance_work_mem = 1GB
# checkpoint_timeout = 300
# checkpoint_warning = 300
checkpoint_completion_target = 0.9
synchronous_commit = off
default_statistics_target = 500
random_page_cost = 4
effective_io_concurrency = 2
wal_buffers = 16MB
min_wal_size = 4GB
max_wal_size = 8GB
max_worker_processes = 4
max_parallel_workers_per_gather = 2
max_parallel_workers = 4
max_files_per_process=500
Asked by sim
(149 rep)
Nov 28, 2019, 03:33 PM
Last activity: Dec 4, 2019, 08:28 PM
Last activity: Dec 4, 2019, 08:28 PM