When I start corosync all servers panics with core dumps
1
vote
1
answer
306
views
I upgraded my servers. Then I started corosync service one by one on my servers. I started first on 3 server and I wait 5 min. Then I started next 4 corosync on other servers and 7 server crashed in same time.
I'm using corosync since 5 years. I was using;
Kernel: 4.14.32-1-lts
Corosync 2.4.2-1
Pacemaker 1.1.18-1
and I never saw this before.
I guess something is broken in new corosync version really really bad!
Kernel: 4.14.70-1-lts
Corosync 2.4.4-3
Pacemaker 2.0.0-1
-
**This is my corosync.conf: https://paste.ubuntu.com/p/7KCq8pHKn3/**
**Can you tell me how can I find the reason of the problem?**
Sep 25 08:56:03 SRV-2 corosync: [TOTEM ] A new membership (10.10.112.10:56) was formed. Members joined: 7
Sep 25 08:56:03 SRV-2 corosync: [VOTEQ ] Waiting for all cluster members. Current votes: 7 expected_votes: 28
Sep 25 08:56:03 SRV-2 corosync: [VOTEQ ] Waiting for all cluster members. Current votes: 7 expected_votes: 28
Sep 25 08:56:03 SRV-2 corosync: [VOTEQ ] Waiting for all cluster members. Current votes: 7 expected_votes: 28
Sep 25 08:56:03 SRV-2 corosync: [VOTEQ ] Waiting for all cluster members. Current votes: 7 expected_votes: 28
Sep 25 08:56:03 SRV-2 corosync: [QUORUM] Members: 1 2 3 4 5 6 7
Sep 25 08:56:03 SRV-2 corosync: [MAIN ] Completed service synchronization, ready to provide service.
Sep 25 08:56:03 SRV-2 corosync: [VOTEQ ] Waiting for all cluster members. Current votes: 7 expected_votes: 28
Sep 25 08:56:03 SRV-2 systemd: Created slice system-systemd\x2dcoredump.slice.
Sep 25 08:56:03 SRV-2 systemd: Started Process Core Dump (PID 43798/UID 0).
Sep 25 08:56:03 SRV-2 systemd: corosync.service: Main process exited, code=dumped, status=11/SEGV
Sep 25 08:56:03 SRV-2 systemd: corosync.service: Failed with result 'core-dump'.
Sep 25 08:56:03 SRV-2 kernel: watchdog: watchdog0: watchdog did not stop!
Sep 25 08:56:03 SRV-2 systemd-coredump: Process 29089 (corosync) of user 0 dumped core.
Stack trace of thread 29089:
#0 0x0000000000000000 n/a (n/a)
Write failed: Broken pipe
coredumpctl info
PID: 23658 (corosync)
UID: 0 (root)
GID: 0 (root)
Signal: 11 (SEGV)
Timestamp: Mon 2018-09-24 09:50:58 +03 (1 day 3h ago)
Command Line: corosync
Executable: /usr/bin/corosync
Control Group: /system.slice/corosync.service
Unit: corosync.service
Slice: system.slice
Boot ID: 79d67a83f83c4804be6ded8e6bd5f54d
Machine ID: 9b1ca27d3f4746c6bcfcdb93b83f3d45
Hostname: SRV-1
Storage: /var/lib/systemd/coredump/core.corosync.0.79d67a83f83c4804be6ded8e6bd5f54d.23658.153777185>
Message: Process 23658 (corosync) of user 0 dumped core.
Stack trace of thread 23658:
#0 0x0000000000000000 n/a (n/a)
PID: 5164 (corosync)
UID: 0 (root)
GID: 0 (root)
Signal: 11 (SEGV)
Timestamp: Tue 2018-09-25 08:56:03 +03 (4h 9min ago)
Command Line: corosync
Executable: /usr/bin/corosync
Control Group: /system.slice/corosync.service
Unit: corosync.service
Slice: system.slice
Boot ID: 2f49ec6cdcc144f0a8eb712bbfbd7203
Machine ID: 9b1ca27d3f4746c6bcfcdb93b83f3d45
Hostname: SRV-1
Storage: /var/lib/systemd/coredump/core.corosync.0.2f49ec6cdcc144f0a8eb712bbfbd7203.5164.1537854963>
Message: Process 5164 (corosync) of user 0 dumped core.
Stack trace of thread 5164:
#0 0x0000000000000000 n/a (n/a)
I cant find more log so I can't dig the problem.
Asked by Ozbit
(439 rep)
Sep 25, 2018, 10:03 AM
Last activity: Oct 11, 2018, 12:29 PM
Last activity: Oct 11, 2018, 12:29 PM