Unexpected occasional DEADLOCK when re-recreating database from split MySQL dump
1
vote
2
answers
45
views
To replace the content of a test database from a production database
mysqldump
output the following command was used:
cat mysqldump-db-thisdate.sql | mysql -p ... mydb
There has never been any issue with this command.
However the DB grew a lot and this command takes several minutes.
In order to reduce this time, a Perl script was written that
- takes the mysqldump output as input
- creates a single file having all DROP TABLE
... CREATE TABLE
for each table
- run this drop-creation file on a single thread, before doing the tables feeding below
- creates as many files (see below) as there are tables (about 100 tables)
- makes a fork()
for each table file that is injected into the DB (all tables are dropped and created + fed in "parallel". table1..100
The DROP-CREATION file is something like
DROP TABLE IF EXISTS mytable1
;
CREATE TABLE mytable1
(
someid1
int NOT NULL,
...
PRIMARY KEY (someid1
)
) ENGINE=InnoDB AUTO_INCREMENT=111 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
DROP TABLE IF EXISTS mytable2
;
CREATE TABLE mytable2
(
someid2
int NOT NULL,
...
PRIMARY KEY (someid2
)
) ENGINE=InnoDB AUTO_INCREMENT=222 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
...
Each table file mytableI.sql
is like this, for instance for mytable1.sql
/*!40103 SET TIME_ZONE='+00:00' */;
SET foreign_key_checks = 0;
SET unique_checks = 0;
SET autocommit = 0;
START TRANSACTION;
LOCK TABLES mytable1
WRITE;
INSERT INTO mytable1
VALUES (...),(...),...;
UNLOCK TABLES;
COMMIT;
It's like doing, in parallel (pseudo code)
for each table 1 to 100 do
cat mytableI.sql | mysql -p ... mydb /* I is 1 ... 100 */
end for
This method works very well, and saves from 50% to 75% of the time compared to the simple cat whole-dump | mysql
usual method.
However, from time to time (maybe 1 / 10), doing this parallel method, mysql
throws an error
> Deadlock found when trying to get lock; try restarting transaction
It happens rarely, so just restarting the command is not a big deal.
But why? Each table is processed at once, foreign keys are not checked... Doesn't MySQL, thanks to "LOCK TABLES" (and other mechanisms) protect itself against deadlocks in this case?
*Addendum*: The mydb test database is not being accesses otherwise.
***edit testing other methods***
Trying to perform the DROP / CREATE operations in parallel, (each DROP / CREATE in the same thread, for each table), not even filling the tables with data, plenty of Deadlocks occur...
Could it be that MySQL does not handle very well DROP/CREATE operations performed simultaneously? (should be done by a single DB admin?)
Note:
"*simultaneously*" and "*in parallel*" meaning each thread has its own MySQL connection.
Asked by Déjà vu
(555 rep)
May 29, 2025, 07:29 AM
Last activity: May 31, 2025, 12:15 PM
Last activity: May 31, 2025, 12:15 PM