Google Datastream errors on larger MySQL tables
0
votes
1
answer
38
views
I have set up a Datastream service, in order to replicate data from Cloud SQL (MySQL) to BigQuery.
Everything is set up correctly, connection works. But the weird thing is that only tables < 10mb size are replicated without issues.
The larger tables (100+ MB) all fail. When checking the error status, it only says "Timed-out while waiting for the query to complete."
I have not found anything useful regarding this error. What approaches can I try? Backfilling a specific table give the same error.
Source (MySQL) database connection flags are set to the recommended values:
net_read_timeout: 3600 seconds (1 hour)
net_write_timeout: 3600 seconds (1 hour)
wait_timeout: 86400 seconds (24 hours)
Asked by Spock
(101 rep)
Mar 30, 2024, 02:57 PM
Last activity: Apr 3, 2024, 06:40 AM
Last activity: Apr 3, 2024, 06:40 AM