Sample Header Ad - 728x90

Persona Xtradb cluster CONFLICT DETECTED

1 vote
0 answers
228 views
I am working on a high-traffic platform with MySQL clustering using Percona XtraDB Cluster. I have a cluster consisting of 4 nodes. The database is receiving more than 500 requests per second for inserting, selecting, and updating data on the same tables. Below is my MySQL configuration: [mysqld] ######## START General settings ############### # General settings user = mysql pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock mysqlx_socket=/tmp/xplugin.sock port = 3306 basedir = /usr datadir = /var/lib/mysql tmpdir = /tmp lc-messages-dir = /usr/share/mysql bind-address = 192.168.100.20 log-error-verbosity=3 innodb_redo_log_capacity=2147483648 character-set-server = utf8mb4 collation-server = utf8mb4_general_ci auto_increment_increment = 5 auto_increment_offset = 3 ############################## END General settings ############################## ############################## START wsrep ############################## #- server-id=2 # Path to Galera library #- wsrep_provider=/usr/lib/galera4/libgalera_smm.so # Cluster connection URL contains IPs of nodes #If no IP is found, this implies that a new cluster needs to be created, #in order to do that you need to bootstrap this node #- wsrep_cluster_address=gcomm://192.168.100.2,192.168.100.1,192.168.100.20,192.168.100.21 # In order for Galera to work correctly binlog format should be ROW #- binlog_format=ROW # Slave thread to use #- wsrep_slave_threads=16 #- wsrep_log_conflicts # This changes how InnoDB autoincrement locks are managed and is a requirement for Galera #- innodb_autoinc_lock_mode=2 # Node IP address #- wsrep_node_address=192.168.100.20 # Cluster name #- wsrep_cluster_name=pxc-cluster #If wsrep_node_name is not specified, then system hostname will be used #- wsrep_node_name=pxc-cluster-node-3 #pxc_strict_mode allowed values: DISABLED,PERMISSIVE,ENFORCING,MASTER #- pxc_strict_mode=ENFORCING # SST method #- wsrep_sst_method=xtrabackup-v2 #- pxc_encrypt_cluster_traffic = OFF ############################## END wsrep ############################## # Number of CPU cores innodb_read_io_threads = 16 innodb_write_io_threads = 16 innodb_thread_concurrency = 16 innodb_purge_threads = 4 innodb_page_cleaners = 4 innodb_log_write_ahead_size=4 thread_cache_size = 64 #thread_handling = pool-of-threads thread_handling = one-thread-per-connection # Buffer pool size (assuming 128GB RAM) innodb_buffer_pool_size = 100G # Storage device type innodb_io_capacity = 1500 # Adjust for NVMe SSD innodb_io_capacity_max = 3000 # Adjust for NVMe SSD ############################## START Performance settings ############################## default_storage_engine = InnoDB key_buffer_size = 256M innodb_buffer_pool_instances = 16 innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 16M innodb_flush_method = O_DIRECT innodb_max_dirty_pages_pct = 90 innodb_flush_neighbors = 0 innodb_lru_scan_depth = 4096 innodb_change_buffer_max_size = 50 innodb_adaptive_hash_index = 0 innodb_doublewrite = 0 innodb_flush_sync = 0 innodb_max_undo_log_size = 2G innodb_use_native_aio = 0 innodb_undo_log_truncate = ON ############################## END Performance settings ############################## ############################## Start Connection setting ############################## # Connection settings max_connections = 3000 max_allowed_packet = 256M wait_timeout = 600 interactive_timeout = 600 ############################## END Connection setting ############################## ############################## START Logging settings ############################## log_error = /var/log/mysql/error.log slow_query_log = 1 slow_query_log_file = /var/log/mysql/slow.log long_query_time = 5 log_queries_not_using_indexes = 1 log_slow_admin_statements = 1 log_slow_replica_statements = 1 ############################## END Logging settings ############################## However, I am facing an issue with "CONFLICT DETECTED" And LOCK issue on my nodes, and the following errors are being reported: 2024-01-12T22:52:49.382682Z 16 [Note] [MY-011825] [InnoDB] *** WAITING FOR THIS LOCK TO BE GRANTED: 2024-01-12T22:52:49.382697Z 16 [Note] [MY-011825] [InnoDB] SQL1: 2024-01-12T22:52:49.382711Z 16 [Note] [MY-011825] [InnoDB] SQL2: INSERT INTO requests_log ..... 2024-01-12T22:52:49.382739Z 16 [Note] [MY-000000] [WSREP] --------- CONFLICT DETECTED -------- 2024-01-12T22:52:49.382756Z 16 [Note] [MY-000000] [WSREP] cluster conflict due to high priority abort for threads: 2024-01-12T22:52:49.382768Z 16 [Note] [MY-000000] [WSREP] Winning thread: THD: 16, mode: high priority, state: exec, conflict: executing, seqno: 177984003 SQL: (null) 2024-01-12T22:52:49.382781Z 16 [Note] [MY-000000] [WSREP] Victim thread: THD: 2338759, mode: local, state: exec, conflict: certifying, seqno: -1 SQL: INSERT INTO requests_log ...... 2024-01-12T22:52:49.407148Z 2338784 [Note] [MY-011825] [InnoDB] *** Lock requesting TRANSACTION: TRANSACTION 255572069, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 , undo log entries 1 MySQL thread id 2338784, OS thread handle 139937898251840, query id 37507384 wsrep: writing row for write-set (177984004) 2024-01-12T22:52:49.407423Z 2338784 [Note] [MY-011825] [InnoDB] *** Lock holder TRANSACTION: TRANSACTION 255572067, ACTIVE (PREPARED) 0 sec , undo log entries 1 MySQL thread id 16, OS thread handle 140003591190080, query id 37507363 wsrep: preparing to commit write set(177984003) 2024-01-12T22:52:50.780305Z 21 [Note] [MY-011825] [InnoDB] *** Priority TRANSACTION: TRANSACTION 255572175, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT , undo log entries 1 MySQL thread id 21, OS thread handle 139994867009088, query id 37507865 wsrep: writing row for write-set (177984088) 2024-01-12T22:52:50.780456Z 21 [Note] [MY-011825] [InnoDB] *** Victim TRANSACTION: TRANSACTION 255572174, ACTIVE 0 sec, thread declared inside InnoDB 4999 mysql tables in use 1, locked 1 , undo log entries 1 MySQL thread id 2338797, OS thread handle 139937889797696, query id 37507862 192.168.100.20 mixsimverify wsrep: replicating and certifying write set(-1) update users set balance= .... 2024-01-12T22:52:50.780515Z 21 [Note] [MY-011825] [InnoDB] *** WAITING FOR THIS LOCK TO BE GRANTED: 2024-01-12T22:52:50.780530Z 21 [Note] [MY-011825] [InnoDB] SQL1: 2024-01-12T22:52:50.780545Z 21 [Note] [MY-011825] [InnoDB] SQL2: update users set balance= ..... 2024-01-12T22:52:50.780564Z 21 [Note] [MY-000000] [WSREP] --------- CONFLICT DETECTED -------- 2024-01-12T22:52:50.780588Z 21 [Note] [MY-000000] [WSREP] cluster conflict due to high priority abort for threads: 2024-01-12T22:52:50.780606Z 21 [Note] [MY-000000] [WSREP] Winning thread: THD: 21, mode: high priority, state: exec, conflict: executing, seqno: 177984088 SQL: (null) 2024-01-12T22:52:50.780624Z 21 [Note] [MY-000000] [WSREP] Victim thread: THD: 2338797, mode: local, state: exec, conflict: certifying, seqno: -1 SQL: update users set balance= ...... 2024-01-12T22:52:50.802889Z 2338810 [Note] [MY-011825] [InnoDB] *** Lock requesting TRANSACTION: TRANSACTION 255572178, ACTIVE 0 sec starting index read mysql tables in use 1, locked 1 MySQL thread id 2338810, OS thread handle 139937889797696, query id 37507868 Applying batch of row changes (update) 2024-01-12T22:52:50.803030Z 2338810 [Note] [MY-011825] [InnoDB] *** Lock holder TRANSACTION: TRANSACTION 255572175, ACTIVE (PREPARED) 0 sec committing , undo log entries 1 MySQL thread id 21, OS thread handle 139994867009088, query id 37507865 innobase_commit_low (177984088) 2024-01-12T22:52:51.291489Z 12 [Note] [MY-011825] [InnoDB] *** Priority TRANSACTION: TRANSACTION 255572217, ACTIVE 0 sec starting index read mysql tables in use 1, locked 1 LOCK WAIT MySQL thread id 12, OS thread handle 140007278396992, query id 37508017 Applying batch of row changes (update) 2024-01-12T22:52:51.291632Z 12 [Note] [MY-011825] [InnoDB] *** Victim TRANSACTION: TRANSACTION 255572216, ACTIVE 0 sec, thread declared inside InnoDB 4999 mysql tables in use 1, locked 1 , undo log entries 1 MySQL thread id 2338798, OS thread handle 139954061203008, query id 37508015 192.168.100.20 mixsimverify wsrep: replicating and certifying write set(-1) update users set balance= .... 2024-01-12T22:52:51.291691Z 12 [Note] [MY-011825] [InnoDB] *** WAITING FOR THIS LOCK TO BE GRANTED: 2024-01-12T22:52:51.291707Z 12 [Note] [MY-011825] [InnoDB] SQL1: 2024-01-12T22:52:51.291751Z 12 [Note] [MY-011825] [InnoDB] SQL2: update users set balance= .... 2024-01-12T22:52:51.291779Z 12 [Note] [MY-000000] [WSREP] --------- CONFLICT DETECTED -------- 2024-01-12T22:52:51.291798Z 12 [Note] [MY-000000] [WSREP] cluster conflict due to high priority abort for threads: How can I address and resolve this conflict and the LOCK issue?
Asked by Sam Wiki (21 rep)
Jan 12, 2024, 11:04 PM