Sample Header Ad - 728x90

Database Administrators

Q&A for database professionals who wish to improve their database skills

Latest Questions

2 votes
1 answers
987 views
MySQL Semi-synchronous replication with Multi-Master
Is it possible to use semi-synchronous replication with a Multi-Master setup? I've tried to follow this guide to setup a semi-synchronous replication for a master-slave setup: https://avdeo.com/2015/02/02/semi-synchronous-replication-in-mysql/ But I'm not sure how to implement this on a Multi-Master...
Is it possible to use semi-synchronous replication with a Multi-Master setup? I've tried to follow this guide to setup a semi-synchronous replication for a master-slave setup: https://avdeo.com/2015/02/02/semi-synchronous-replication-in-mysql/ But I'm not sure how to implement this on a Multi-Master setup. There are two plugins: one for the master and one for the slave. Since a Multi-Master act as a Master and Slave, does that mean I have to install both plugins on all servers? I'm using MySQL 5.7
TheOddGuy (183 rep)
Jun 17, 2016, 08:42 AM • Last activity: Feb 20, 2025, 06:02 AM
1 votes
1 answers
235 views
Difference between flushing and synchronisation and writing to disk (MySQL)
I get confused with the terminology such as flushing and writing to disk and synchronising to the disk. Could someone please explain me what is the difference between flushing and writing and synchronising to disk especially in context of MySQL. I used to think that they are both the same operations...
I get confused with the terminology such as flushing and writing to disk and synchronising to the disk. Could someone please explain me what is the difference between flushing and writing and synchronising to disk especially in context of MySQL. I used to think that they are both the same operations, but while reading various concepts it confuses me. For example. The documentation says for **rpl_semi_sync_source_wait_point** variable that if I set it to **AFTER_SYNC** then the source writes transactions to bin log and replica first and then syncs the binlog to disk. https://dev.mysql.com/doc/refman/8.0/en/replication-options-source.html#sysvar_rpl_semi_sync_source_wait_point AFTER_SYNC (the default): The source writes each transaction to its binary log and the replica, and syncs the binary log to disk. This statement is itself confusing. First it says source writes each transaction to its binary log and the replica and then it says syncs the binary log to disk. So what exactly is happening in this case. What exactly is flushing, syncing and writing to the disk in the context of MySQL? The question is not about sync_binlog or any other parameter in MySQL. The question is about flushing, syncing and writing and about when do these activities happen? Again when I tried to read more, in the following link, it explains that, there's a binlog cache, a page cache and an fsync operation. Thus, when a transaction happens, if I understand correctly, the buffer is made dirty, the contents of the transaction are written to binlog cache (at MySQL level) which is separate cache for each session, then binlog cache is written to page cache (at Linux level) then depending upon **sync_binlog** value, it is flushed/written/synced to the files on disk. https://betterprogramming.pub/understanding-mysqls-binlog-4ac8de4d20ee
Avinash Pawar (216 rep)
Jul 27, 2024, 06:59 AM • Last activity: Jul 30, 2024, 03:49 PM
2 votes
1 answers
54 views
Can PostgresSQL cluster have synchronus and asyncronus standbys at the same time for the same dbs?
In this question, https://dba.stackexchange.com/questions/267400/write-concern-in-postgresql/267407#267407 [Laurenz Albe][1] explained how to control the set of standby servers when data is committed to PostgresSQL, and that gave birth to another question. What if I want to have N standbys, and want...
In this question, https://dba.stackexchange.com/questions/267400/write-concern-in-postgresql/267407#267407 Laurenz Albe explained how to control the set of standby servers when data is committed to PostgresSQL, and that gave birth to another question. What if I want to have N standbys, and want to have synchronous replication enabled for some M subset of N, is that possible? A use case could be a cold backup. I don't care of data inconsistency for N-M subset, I just need to have that on M subset of N standbys.
Edik Mkoyan (229 rep)
May 20, 2020, 12:38 PM • Last activity: May 20, 2020, 02:02 PM
0 votes
0 answers
602 views
mysql unable to start with semi sync parameter
Im using mysql 5.7 and it has 2 slaves with GTID. I tried to enable semi-sync and I have done with `set global variable` in mysql prompt. It was running fine. Then I did a restart, after that mysql unable to start and it says `unknown variable 'rpl-semi-sync-master-enabled=1'` ## mysql log 2018-11-0...
Im using mysql 5.7 and it has 2 slaves with GTID. I tried to enable semi-sync and I have done with set global variable in mysql prompt. It was running fine. Then I did a restart, after that mysql unable to start and it says unknown variable 'rpl-semi-sync-master-enabled=1' ## mysql log 2018-11-03T08:28:35.453089Z 5 [Note] Start binlog_dump to master_thread_id(5) slave_server(2), pos(, 4) 2018-11-03T08:28:35.453119Z 5 [Note] Start asynchronous binlog_dump to slave (server_id: 2), pos(, 4) 2018-11-03T08:28:35.793103Z 6 [Note] Start binlog_dump to master_thread_id(6) slave_server(3), pos(, 4) 2018-11-03T08:28:35.793141Z 6 [Note] Start semi-sync binlog_dump to slave (server_id: 3), pos(, 4) 2018-11-03T08:29:25.656749Z 0 [Note] Giving 5 client threads a chance to die gracefully 2018-11-03T08:29:25.656793Z 0 [Note] Shutting down slave threads 2018-11-03T08:29:33.658167Z 5 [Note] Stop asynchronous binlog_dump to slave (server_id: 2) 2018-11-03T08:29:33.658360Z 6 [Note] Stop semi-sync binlog_dump to slave (server_id: 3) 2018-11-03T08:29:35.658275Z 0 [Note] Forcefully disconnecting 0 remaining clients 2018-11-03T08:29:35.658368Z 0 [Note] Event Scheduler: Purging the queue. 0 events 2018-11-03T08:29:35.658532Z 0 [Note] Binlog end 2018-11-03T08:29:35.662977Z 0 [Note] Shutting down plugin 'rpl_semi_sync_slave' 2018-11-03T08:29:35.663025Z 0 [Note] Shutting down plugin 'rpl_semi_sync_master' 2018-11-03T08:29:35.663059Z 0 [Note] unregister_replicator OK 2018-11-03T08:29:35.663062Z 0 [Note] Shutting down plugin 'auth_socket' ## syslog: Nov 3 04:11:17 mysql-systemd-start: ERROR: Unable to start MySQL server: Nov 3 04:11:17 mysql-systemd-start: 2018-11-03T04:11:17.961175Z 0 [ERROR] unknown variable 'rpl-semi-sync-master-enabled=1' Nov 3 04:11:17 mysql-systemd-start: 2018-11-03T04:11:17.964899Z 0 [ERROR] Aborting ## Update: I used the lines then it worked. But what is the use of these parameters? loose-rpl_semi_sync_master_enabled = 1 loose-rpl_semi_sync_slave_enabled = 1
TheDataGuy (1986 rep)
Nov 3, 2018, 08:34 AM • Last activity: Nov 3, 2018, 08:47 AM
3 votes
3 answers
1342 views
High disk IO with MySQL master-master replication
We've been evaluating an active-passive master-master replication setup for our MySQL database, running on version 8.0. While diagnosing some general slowness issues, we were surprised to see very high disk IO ranging from 500MB/s - 1500MB/s on both servers. I've found that stopping the slave on eit...
We've been evaluating an active-passive master-master replication setup for our MySQL database, running on version 8.0. While diagnosing some general slowness issues, we were surprised to see very high disk IO ranging from 500MB/s - 1500MB/s on both servers. I've found that stopping the slave on either server brings the disk IO back to expected levels, 15MB/s - 30 MB/s, and I'm concerned that we have a misconfiguration somewhere. Here is what I've checked: * The server ID's are unique. * We are doing row-based replication. * We are using the semi-synchronous replication plugin. * The writes are happening in an LVM volume that exclusively houses the data directory. * vmstat shows no swapping. * From the network perspective, we see a few megabits per second going between the servers at most. Here is the redacted configuration: [mysqld] ... # General server_id = event-scheduler = ON sync_binlog = 1 log_bin = /var/lib/mysql/mysql-bin binlog_format = ROW innodb_file_per_table innodb_flush_log_at_trx_commit = 1 skip-external-locking # Replication relay_log = /var/lib/mysql/mysql-relay-bin relay_log_recovery = ON log_slave_updates = 1 sync_master_info = 1 sync_relay_log = 1 sync_relay_log_info = 1 rpl_semi_sync_master_enabled = 1 rpl_semi_sync_master_timeout = 10000 # 10 seconds rpl_semi_sync_slave_enabled = 1 # Fine Tuning key_buffer_size = 16M max_allowed_packet = 16M thread_stack = 192K thread_cache_size = 8 myisam-recover-options = BACKUP max_connections = 800 max_connect_errors = 1000 innodb_buffer_pool_size = 192G # Server has 256GB RAM innodb_log_buffer_size = 256M innodb_log_file_size = 2G innodb_flush_method = O_DIRECT innodb_lock_wait_timeout = 60 ... My problem sounds suspiciously similar to this person's dead thread from 2013 (Who were you, DenverCoder9? WHAT DID YOU SEE!? ) **EDIT:** In an effort to zero in on the problem, we simplified the setup for testing. We used one master and one slave, both running 8.0.11, same configuration as above, and we saw the following write patterns according to iostat (master is blue, slave is red): Master-Slave IO (The "Time" on the X-axis is the minutes and seconds of the current time.) Those per-minute 1+ GB spikes on the master were tracked down to a script updating two small tables (1000-2000 rows each), around 10-15 MB in size on disk. These tables had keys, but the keys were not primary or unique. The script would DELETE all rows in the tables and then repopulate them by performing one INSERT statement per row, all within a transaction. Despite the glaring problems, I still don't understand how operations on tables this small can result in this much disk IO. Without fixing the problems, we then returned to the master-master active-passive setup and saw the following (active master is blue, passive master is red): Active-Passive Master-Master IO So some additional questions I have are this. Why is the slave consistently writing much more than the master in the first graph? Also, why in master-master replication is there the ping-pong effect seen above where writes to the master result in heavy IO on the slave, which results in heavy IO again on the master?
billyw (151 rep)
May 21, 2018, 05:35 PM • Last activity: Jun 18, 2018, 06:37 PM
1 votes
1 answers
121 views
Accessing Oracle DB from SQL Server DB
I am looking for the best and simplest way for accessing an Oracle db from an SQL Server db. Both are on separate physical servers. I am looking for a one-way access in read-only mode. SQL Server Database would be used by one system for now but would scale to be used by multiple applications in the...
I am looking for the best and simplest way for accessing an Oracle db from an SQL Server db. Both are on separate physical servers. I am looking for a one-way access in read-only mode. SQL Server Database would be used by one system for now but would scale to be used by multiple applications in the future. I currently have three options in mind: ---------- Option 1: Directly accessing Materialised views - Every hour or so, data would be stored into Materialised views in Oracle in order to prevent impacting original DB performance. - SQL Server DB would access the views on-demand, using Linked servers ---------- Option 2: Copy data to SQL Server DB - Every hour or so, replicated tables into SQL Server DB using CDC mechanisms. (Not so sure how that'll work). ---------- Option 3: Replicate Oracle tables using SQL Server Publishing - Setup the Oracle DB as the distributor. - Setup the SQL Server as the publisher (snapshot mode). Thank you for your advices.
Francis P (111 rep)
Mar 13, 2018, 08:19 PM • Last activity: Mar 13, 2018, 08:26 PM
1 votes
1 answers
946 views
MySQL Database replication for high availability
**configuration:** - 1 balancer - 2 Azure machines(4 gb ram, 2 cores), with MySQL master-master replication enabled. X users connecting to the balancer to sync data. **test:** i've tested the MySQL replication with loader.io (for a connection spike), for 6000 calls for 1 min to the balancer (100 con...
**configuration:** - 1 balancer - 2 Azure machines(4 gb ram, 2 cores), with MySQL master-master replication enabled. X users connecting to the balancer to sync data. **test:** i've tested the MySQL replication with loader.io (for a connection spike), for 6000 calls for 1 min to the balancer (100 conn per 1 minute test), and with the current setup both dbs sync in about 6-7 mins (all 6000 syncs) **scenario:** A client should sync it's data from last connection no matter what machine the balancer distributed his connection to, and that db for example might not have finished the previous sync period yet (let's say the 6 minute, as in the above example). I guess a sticky balancer solution might not help(Nginx, HAProxy) and Azure balancer is not persistent\sticky anyway (for example user is connected to 3G, 4G and IP changes frequent).. **problem:** 1. how do i get the current replication sync the fastest way possible between both machines? 2. is there a way to transform my current setup to synchronous replication? 2. is there any other solution (preferable windows OS) that does synchronous sync (like: [semisync replication](https://dev.mysql.com/doc/refman/5.5/en/replication-semisync.html) )?
bob (51 rep)
Jun 3, 2014, 03:59 PM • Last activity: Jun 4, 2014, 02:23 AM
2 votes
1 answers
443 views
Syncing updates to master in master/slave setup [semi-sync replication]
Background info --------------- I have two MySQL 5.5 servers set up in a Master/Slave configuration with **Semi-synchronous** replication. The database is being used for a high-profile WordPress site. Problem ------- It seems to work great except for one thing: When the master is down and changes ar...
Background info --------------- I have two MySQL 5.5 servers set up in a Master/Slave configuration with **Semi-synchronous** replication. The database is being used for a high-profile WordPress site. Problem ------- It seems to work great except for one thing: When the master is down and changes are made to the slave, they are not written back to the master when it comes back up. What I've done so far ------------------ I tried the answer [[here]][1] , but it broke my replication completely. Questions --------- - Is there a more standard way of setting this up or is it generally not supported? - Should I just disallow writes on the slave? - Is there a simple way to notify the master of changes, and sync them upstream?
Vasili Syrakis (125 rep)
May 21, 2014, 12:10 AM • Last activity: May 21, 2014, 02:50 AM
1 votes
2 answers
383 views
Breaking Semisynchronous Replication in MySQL 5.5
I've set up Semisynchronous Replication between two MySQL 5.5 servers running on Windows 7. My application is running and updating the database of the master server and same is being updated in the slave database server. But due to some unknown reasons sometimes, Replication breaks. On running the c...
I've set up Semisynchronous Replication between two MySQL 5.5 servers running on Windows 7. My application is running and updating the database of the master server and same is being updated in the slave database server. But due to some unknown reasons sometimes, Replication breaks. On running the command: SHOW STATUS LIKE 'Rpl_semi_sync%'; It gives this status: 'Rpl_semi_sync_master_no_times', '0' 'Rpl_semi_sync_master_no_tx', '0' 'Rpl_semi_sync_master_status', 'ON' <<------------- 'Rpl_semi_sync_master_timefunc_failures', '0' 'Rpl_semi_sync_master_tx_avg_wait_time', '338846' 'Rpl_semi_sync_master_tx_wait_time', '29479685' 'Rpl_semi_sync_master_tx_waits', '87' 'Rpl_semi_sync_master_wait_pos_backtraverse', '0' 'Rpl_semi_sync_master_wait_sessions', '0' 'Rpl_semi_sync_master_yes_tx', '3106' Ideally, in semi synchronization, when the sync breaks the status should come as OFF since master is not able to receive any acknowledgement from the slave. Please help us in this regard.
Neeru Sharma (173 rep)
Mar 12, 2013, 10:32 AM • Last activity: Aug 30, 2013, 01:41 PM
5 votes
1 answers
1580 views
Installing and enabling both master and slave semisynchronous plugins on same server
I want to run a semisynchronous replication setup, and for simplicity I'd like to not have to INSTALL and UNINSTALL plugins when a slave is promoted to a master. So I'm wondering if I'll experience any problems having *both* master and slave plugins installed and enabled (see [here](http://dev.mysql...
I want to run a semisynchronous replication setup, and for simplicity I'd like to not have to INSTALL and UNINSTALL plugins when a slave is promoted to a master. So I'm wondering if I'll experience any problems having *both* master and slave plugins installed and enabled (see [here](http://dev.mysql.com/doc/refman/5.5/en/replication-semisync-installation.html)) It looks like after installing the second plugin, both variables rpl_semi_sync_master_enabled and rpl_semi_sync_slave_enabled are reset to off (the non-default). When I include these lines in my.cnf: loose-rpl_semi_sync_master_enabled = 1 loose-rpl_semi_sync_slave_enabled = 1 ...and restart MySQL, both plugins are loaded and enabled. Is this going to cause any problems for me?
jberryman (481 rep)
Oct 29, 2012, 08:28 PM • Last activity: May 12, 2013, 10:42 AM
4 votes
1 answers
239 views
Question about MySQL Semisynchronous Replication
I was researching about [MySQL Semisynchronous Replication][1], my understanding about the workflow is the follow: 1. Master Receive a successful transaction from a client session. 2. Master write it to his binaty log 3. Master wait for `rpl_semi_sync_master_timeout` milliseconds to receive an ackno...
I was researching about MySQL Semisynchronous Replication , my understanding about the workflow is the follow: 1. Master Receive a successful transaction from a client session. 2. Master write it to his binaty log 3. Master wait for rpl_semi_sync_master_timeout milliseconds to receive an acknowledgment from at least one slave 4. Slave pick this successful transaction, write it to his relay log, flush it to disk 5. Slave return an acknowledgment to master 6. Master return successful to his client session. My question is, at any point, this type of replication ensures that the transaction is executed on slave(SQL_TREAD applies it). Having in mind that it will add a delay on each transaction and in a case of master failure, is not 100% guaranteed that the slave will be consistent with master, what is the benefit(s) of using this type of replication?
altmannmarcelo (1217 rep)
May 5, 2013, 01:08 PM • Last activity: May 5, 2013, 03:18 PM
Showing page 1 of 11 total questions