Sample Header Ad - 728x90

Google Datastream errors on larger MySQL tables

0 votes
1 answer
38 views
I have set up a Datastream service, in order to replicate data from Cloud SQL (MySQL) to BigQuery. Everything is set up correctly, connection works. But the weird thing is that only tables < 10mb size are replicated without issues. The larger tables (100+ MB) all fail. When checking the error status, it only says "Timed-out while waiting for the query to complete." I have not found anything useful regarding this error. What approaches can I try? Backfilling a specific table give the same error. Source (MySQL) database connection flags are set to the recommended values: net_read_timeout: 3600 seconds (1 hour) net_write_timeout: 3600 seconds (1 hour) wait_timeout: 86400 seconds (24 hours)
Asked by Spock (101 rep)
Mar 30, 2024, 02:57 PM
Last activity: Apr 3, 2024, 06:40 AM