MySQL is pretty slow after 50-100k rows whats happen?
1
vote
2
answers
1969
views
I have been trying now many times, and I tried to find a way to make sense of and find out my problem. But the problem is still performance issues.
I tested it on a developer server ( 2 cores, 4gb ram ) but I still got issues and problems, here is the stored procedure I have typed.
DROP PROCEDURE IF EXISTS insertDistributorProduct;
DELIMITER $$
CREATE PROCEDURE insertDistributorProduct(
distributorUUID VARCHAR(36),
distributorTitle VARCHAR(256),
distributorDescription TEXT,
distributorSKU VARCHAR(128),
manufacturer VARCHAR(64),
manufacturerSKU VARCHAR(128),
productEan BIGINT(20),
productCostPrice INT(11),
productStock INT(11),
productStockExpected DATE
)
BEGIN
DECLARE _manufacturerUUID VARCHAR(36) DEFAULT NULL;
DECLARE _distributorProductUUID VARCHAR(36) DEFAULT NULL;
SET _manufacturerUUID = ( SELECT m.manufacturerUUID FROM manufacturer m
WHERE LOWER(m.manufacturerTitle) = LOWER(manufacturer) LIMIT 1 );
-- If manufacturer not exists insert it
IF ( _manufacturerUUID IS NULL ) THEN
SET _manufacturerUUID = ( SELECT ma.manufacturerUUID FROM manufacturerAlias ma
WHERE LOWER(ma.aliasTitle) = LOWER(manufacturer) LIMIT 1 );
-- _manufacturerUUID not found in manufacturer and _manufacturerAlias so insert
IF ( _manufacturerUUID IS NULL ) THEN
SET _manufacturerUUID = UUID();
INSERT INTO manufacturer(manufacturerUUID,manufacturerTitle,added)
VALUES(_manufacturerUUID,manufacturer,NOW());
END IF;
END IF;
SET _distributorProductUUID = ( SELECT dp.distributorProductUUID FROM distributorProduct dp
WHERE dp.distributorUUID = distributorUUID
AND LOWER(dp.distributorSKU) = LOWER(distributorSKU) );
-- Update distributorProduct if its all ready found
IF ( _distributorProductUUID IS NOT NULL ) THEN
UPDATE
distributorProduct dp
SET
dp.distributorTitle = distributorTitle,
dp.distributorDescription = distributorDescription,
dp.manufacturerUUID = _manufacturerUUID,
dp.manufacturerSKU = manufacturerSKU,
dp.productEan = productEan,
dp.productCostPrice = productCostPrice,
dp.productStock = productStock,
dp.productStockExpected = productStockExpected,
dp.updated = NOW(),
dp.activated = 1
WHERE
dp.distributorProductUUID = _distributorProductUUID
LIMIT
1;
-- Insert new product to distributorProduct if not exists.
ELSE
INSERT INTO distributorProduct (distributorProductUUID,distributorUUID,manufacturerUUID,distributorTitle,distributorDescription,distributorSKU,manufacturerSKU,productEan,productCostPrice,productStock,productStockExpected,added)
VALUES(UUID(),distributorUUID,_manufacturerUUID,distributorTitle,distributorDescription,distributorSKU,manufacturerSKU,productEan,productCostPrice,productStock,productStockExpected,NOW());
END IF;
END
$$
My problem is that every time I have over 50-100k rows, this script takes longer and longer over time, and I have to try to look up my index but still nothing happens.
How can I debug and improve the performance of my database? Every day I need to run this producer like 1-1,4milion times between 4-5 hours max, it updates all our products but right now it's useless.
The first time 60k rows take 15min, and after that 60k more ( total 120k ) it takes approx 30-45min to run, and I still don't know why it happened. What did I do wrong?
Everything I tested here is running in a virtual box software, I don't have to test this on real hardware.
Asked by ParisNakitaKejser
(249 rep)
Mar 26, 2015, 08:27 PM
Last activity: Apr 13, 2025, 06:07 AM
Last activity: Apr 13, 2025, 06:07 AM