Sample Header Ad - 728x90

Importing a 16 GB MySQL file silently fails

3 votes
2 answers
1963 views
I've looked at a lot answers here https://dba.stackexchange.com/questions/83125/mysql-any-way-to-import-a-huge-32-gb-sql-dump-faster and on Server Fault but haven't found a solution to not being able to import a 16 GB MySQL file completely; it silently fails at different parts of the import. Sometimes 2 GB will import before the import stops, sometimes 10 GB. There are no errors in the logs. Sometimes the console returns to #, sometimes not. top shows MySQL running at 100% when importing, but then drops down to normal loads when the import stops. The database is 16 GBs, and has 90% InnoDB tables with a few MyISAM tables. To export the database on the first machine, I'm using mysqldump --single-transaction --lock-tables -u mydatabaseuser -p mydatabase > archive.sql The importing machine has 8 dedicated cores and 150 GB dedicated RAM (server is at Linode) and is running Alma Linux and MySQL 8.0.25. To import, I'm using mysql -u root -p mydatabase ` doesn't help. One table is 12 GB, and I tried importing that table by itself with no luck. Using these in my.cnf throws an error on mysql restart: autocommit=0 unique_checks=0 foreign_key_checks=0 Should I be exporting differently? Is MySQL on the importing machine timing out? What do I need to edit in my.cnf? **my.cnf on the importing machine:** [mysqld] disable-log-bin=1 default-authentication-plugin=mysql_native_password performance-schema=0 port = 3306 socket = /tmp/mysql.sock skip-external-locking skip-name-resolve # uncomment for import bulk_insert_buffer_size = 40G read_buffer_size = 40G # If I uncomment these, MySQL throws the error on restart # Job for mysqld.service failed because the control process exited with error code" # autocommit=0 # unique_checks=0 # foreign_key_checks=0 innodb_buffer_pool_size=50G innodb_buffer_pool_instances=56 innodb_log_file_size = 8G innodb_log_buffer_size=64M innodb_read_io_threads=8 innodb_write_io_threads=8 innodb_io_capacity=300 innodb_doublewrite = 0 max_allowed_packet=268435456 open_files_limit=40000 innodb_file_per_table=1 join_buffer_size=128M sort_buffer_size=2M read_rnd_buffer_size=2M key_buffer_size = 20M datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid **Edit 7/28/21** > mysql> SHOW GLOBAL VARIABLES LIKE '%timeout%'; > > connect_timeout 10 > delayed_insert_timeout 300 > have_statement_timeout YES > innodb_flush_log_at_timeout 1 > innodb_lock_wait_timeout 50 > innodb_rollback_on_timeout OFF > interactive_timeout 28800 > lock_wait_timeout 31536000 > mysqlx_connect_timeout 30 > mysqlx_idle_worker_thread_timeout 60 > mysqlx_interactive_timeout 28800 > mysqlx_port_open_timeout 0 > mysqlx_read_timeout 30 > mysqlx_wait_timeout 28800 > mysqlx_write_timeout 60 > net_read_timeout 30 > net_write_timeout 60 > replica_net_timeout 60 > rpl_stop_replica_timeout 31536000 > rpl_stop_slave_timeout 31536000 > slave_net_timeout 60 > wait_timeout 6000 > > mysql> SHOW SESSION VARIABLES LIKE '%timeout%'; > > connect_timeout 10 > delayed_insert_timeout 300 > have_statement_timeout YES > innodb_flush_log_at_timeout 1 > innodb_lock_wait_timeout 50 > innodb_rollback_on_timeout OFF > interactive_timeout 28800 > lock_wait_timeout 31536000 > mysqlx_connect_timeout 30 > mysqlx_idle_worker_thread_timeout 60 > mysqlx_interactive_timeout 28800 > mysqlx_port_open_timeout 0 > mysqlx_read_timeout 30 > mysqlx_wait_timeout 28800 > mysqlx_write_timeout 60 > net_read_timeout 30 > net_write_timeout 60 > replica_net_timeout 60 > rpl_stop_replica_timeout 31536000 > rpl_stop_slave_timeout 31536000 > slave_net_timeout 60 > wait_timeout 28800
Asked by BlueDogRanch (83 rep)
Jul 19, 2021, 04:30 PM
Last activity: Jul 28, 2021, 05:20 PM