Sample Header Ad - 728x90

MySQL InnoDB Cluster config - configure node address

0 votes
1 answer
295 views
I first asked this question on Stack Overflow but I woke up this morning to the realization that it belongs over here. I'm setting up an InnoDB Cluster using mysqlsh. This is in Kubernetes, but I think this question applies more generally. When I use cluster.configureInstance() I see messages that includes: >This instance reports its own address as node-2:3306 However, the nodes can only find *each other* through DNS at an address like node-2.cluster:3306. The problem comes when adding instances to the cluster; they try to find the other nodes without the qualified name. Errors are of the form: [GCS] Error on opening a connection to peer node node-0:33061 when joining a group. My local port is: 33061. It is using node-n:33061 rather than node-n.cluster:33061. If it matters, the "DNS" is set up as a headless service in Kubernetes that provides consistent addresses as pods come and go.It's very simple, and I named it "cluster" to created addresses of the form node-n.cluster. The cluster itself is a statefulset that numbers nodes sequentially. I don't want to cloud this question with detail I don't think matters, however, as surely other configurations require the instances in the cluster to use DNS as well. I thought that setting localAddress when creating the cluster and adding the nodes would solve the problem. Indeed, after I added that to the createCluster options, I can look in the database and see | group_replication_local_address | node-0.cluster:33061 | Yet when I look at the topology, it seems that the localAddress option has no effect whatsoever (at this point I don't see what it does at all):
{
    "clusterName": "mycluster", 
    "defaultReplicaSet": {
        "name": "default", 
        "primary": "node-0:3306", 
        "ssl": "REQUIRED", 
        "status": "OK_NO_TOLERANCE", 
        "statusText": "Cluster is NOT tolerant to any failures.", 
        "topology": {
            "node-0:3306": {
                "address": "node-0:3306", 
                "memberRole": "PRIMARY", 
                "mode": "R/W", 
                "readReplicas": {}, 
                "replicationLag": null, 
                "role": "HA", 
                "status": "ONLINE", 
                "version": "8.0.29"
            }
        }, 
        "topologyMode": "Single-Primary"
    }, 
    "groupInformationSourceMember": "node-0:3306"
}
And adding more instances continues to fail with the same communication errors. Notable perhaps that I cannot set localAddress to node-0.cluster:3306; I get a message that that port is already in use. I tried 3307 but as before it had no effect. How do I convince each instance that the address it needs to advertise is different? I will try other permutations of the localAddress setting, but it doesn't look like it's intended to fix the problem I'm having. How do I reconcile the address the instance reports for itself with the address that's actually useful for other instances to find it? Edit to add: Maybe it is a Kubernetes thing? Kubernetes sets the hostname of the pod to match the name of the pod. If so, how do I override it? There has to be a way to tell the cluster to use something other than the machines' hostnames for discovering each other.
Asked by Jerry (101 rep)
Jun 30, 2022, 04:43 PM
Last activity: May 16, 2025, 08:09 AM