postgres_fdw slower then copy + scp + copy (~12 times)
5
votes
2
answers
2593
views
- foreign server 9.2
- local server 9.5
- table is 10GB
- data transfer performed on same network interface as foreign server works
- no indexes set on data
- old way:
1. copy to - 2:36
2. scp - 08:17
3. copy from - 10:11
- postgres_fdw:
1. by the time old way finished it has done 800MB of
insert into .. select * from foreign_table
Did I miss something in config (meaning I can improve it), or postgres_fdw
is just not meant for bulk load (meaning I can't improve it)?
(I use it for small data amount reconcile and it works fine. The idea of insert select from fdw
instead of running bash commands looked so sweet.)*
I tried psql to remote server from local server and \copy table
- six minutes - faster then over ssh.
The fetch_size
option, not available prior to 9.6, can be mocked up with dblink_fetch(CURSOR, fetch_size)
- see my answer below.
Asked by Vao Tsun
(1263 rep)
Feb 23, 2017, 04:11 PM
Last activity: May 30, 2022, 02:50 PM
Last activity: May 30, 2022, 02:50 PM