Database Administrators
Q&A for database professionals who wish to improve their database skills
Latest Questions
0
votes
1
answers
353
views
How to speed up Oracle to PostgreSQL data transfer
I have an Oracle schema that I am transferring to PostgreSQL with ogr2ogr. This particular table has 5 columns with number(22) each. There are some 300M rows. In under 10 minutes I can transfer this table between two oracle servers over the network. `ogr2ogr` took a day to transfer 10M records and w...
I have an Oracle schema that I am transferring to PostgreSQL with ogr2ogr.
This particular table has 5 columns with number(22) each. There are some 300M rows. In under 10 minutes I can transfer this table between two oracle servers over the network.
ogr2ogr
took a day to transfer 10M records and will take a month to transfer the whole table.
I suspect the problem is on the receiving end. Data is transferred to AWS PgSQL with 4 Gb ram. How to debug the bottleneck and speed up the transfer?
Stepan
(213 rep)
Jul 26, 2019, 02:14 PM
• Last activity: Apr 11, 2022, 11:01 AM
0
votes
0
answers
553
views
Using Ubuntu, `ogrinfo` fails to open an mdb file
Whenever I try to open a sample-mdb file, I'm getting ogrinfo Campaign_Template.mdb ERROR 1: PGeo: MDB Tools driver not found! Warning 1: Unable to install MDB driver for ODBC, MDB access may not supported. ERROR 1: Unable to initialize ODBC connection to DSN for DRIVER=Microsoft Access Driver (*.md...
Whenever I try to open a sample-mdb file, I'm getting
ogrinfo Campaign_Template.mdb
ERROR 1: PGeo: MDB Tools driver not found!
Warning 1: Unable to install MDB driver for ODBC, MDB access may not supported.
ERROR 1: Unable to initialize ODBC connection to DSN for DRIVER=Microsoft Access Driver (*.mdb, *.accdb);DBQ=Campaign_Template.mdb,
[unixODBC][Driver Manager]Can't open lib 'Microsoft Access Driver (*.mdb, *.accdb)' : file not found
ERROR 1: PGeo: MDB Tools driver not found!
Warning 1: Unable to install MDB driver for ODBC, MDB access may not supported.
ERROR 1: Unable to initialize ODBC connection to DSN for DRIVER=Microsoft Access Driver (*.mdb, *.accdb);DBQ=Campaign_Template.mdb,
[unixODBC][Driver Manager]Can't open lib 'Microsoft Access Driver (*.mdb, *.accdb)' : file not found
FAILURE:
Unable to open datasource `Campaign_Template.mdb' with the following drivers.
I'm seeing the driver only mentioned in the docs,
apt-file search drv_pgeo
libgdal-doc: /usr/share/doc/libgdal-doc/gdal/drv_pgeo.html
However, the current package of
libgdal20
does not have drv_pgeo
what happened to it, and where can I find it?
Evan Carroll
(65502 rep)
May 17, 2017, 01:49 AM
• Last activity: May 7, 2018, 04:19 AM
5
votes
4
answers
1240
views
Best way to model a database structure to store multidimensional variables?
I am working with some [NetCDF][1] data which has multidimensional variables. - As far as I'm aware, the fields are not mandatory. - Each variable will have the same fields/attributes. - Each variable represents a measurement of the atmosphere, in time & space. `Level` represents a pressure level (l...
I am working with some NetCDF data which has multidimensional variables.
- As far as I'm aware, the fields are not mandatory.
- Each variable will have the same fields/attributes.
- Each variable represents a measurement of the atmosphere, in time & space.
Level
represents a pressure level (like a layer of an onion -- a thick layer).
For example, a variable
called *SO2* (Sulphur Dioxide) may be *4-D*: Time
, Level (pressure)
, latitude
, longitude
.
I have about 14 variables that I need to store.
**Data retrieval tendencies**
Now, I'm going to need to query this data and ask questions like:
- Give me all the rows for variable
*SO2* that fall between latitude
*X* and longitude
*Y* at time
*T* and level
*L*.
- What is the average temperature
for time
*T* at level
*L*?
- etc...
**Current considerations**
I could let each row in my DB be an observation
and have the columns: type
, lat
, long
, time
, level
, value
.
Or, perhaps create a table for each kind of variable
and have the columns: lat
, long
, time
, level
, value
.
I'm not sure which will be more appropriate, or whether there is a better option, so I'd appreciate some advice.
pookie
(181 rep)
Feb 8, 2017, 08:43 PM
• Last activity: Aug 6, 2017, 04:59 PM
Showing page 1 of 3 total questions