在做Corosync+DRBD的高可用MySQL集群實驗中,意外發(fā)現(xiàn)各個節(jié)點無法識別對方,連接為StandAlone則主從節(jié)點無法通信,顯示如下:
[root@node1 ~]# cat /proc/drbd
version: 8.3.15 (api:88/proto:86-97)
GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by mockbuild@builder17.CentOS.org, 2013-03-27 16:04:08
0: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown r-----
ns:0 nr:0 dw:36 dr:285 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:24
[root@node2 ~]# cat /proc/drbd
version: 8.3.15 (api:88/proto:86-97)
GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by mockbuild@builder17.centos.org, 2013-03-27 16:04:08
0: cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:8
查看Node1節(jié)點的日志發(fā)現(xiàn)腦裂:
[root@node1 ~]# tail -n 12 /var/log/messages
Sep 21 11:19:53 lab4 kernel: block drbd0: Split-Brain detected but unresolved, dropping connection!
Sep 21 11:19:53 lab4 kernel: block drbd0: helper command: /sbin/drbdadm split-brain minor-0
Sep 21 11:19:53 lab4 kernel: block drbd0: helper command: /sbin/drbdadm split-brain minor-0 exit code 0 (0x0)
Sep 21 11:19:53 lab4 kernel: block drbd0: conn( WFReportParams -> Disconnecting )
Sep 21 11:19:53 lab4 kernel: block drbd0: error receiving ReportState, l: 4!
Sep 21 11:19:53 lab4 kernel: block drbd0: meta connection shut down by peer.
Sep 21 11:19:53 lab4 kernel: block drbd0: asender terminated
Sep 21 11:19:53 lab4 kernel: block drbd0: Terminating drbd0_asender
Sep 21 11:19:53 lab4 kernel: block drbd0: Connection closed
Sep 21 11:19:53 lab4 kernel: block drbd0: conn( Disconnecting -> StandAlone )
Sep 21 11:19:53 lab4 kernel: block drbd0: receiver terminated
Sep 21 11:19:53 lab4 kernel: block drbd0: Terminating drbd0_receiver
以下為DRBD腦裂手動恢復(fù)過程:
將Node1設(shè)置為主節(jié)點并掛載測試
[root@node1 ~]# drbdadm primary mydrbd
[root@node1 ~]# mount /dev/drbd0 /mydata
[root@node1 ~]# ll /mydata/
total 20
-rw-r--r-- 1 root root 1666 Sep 20 18:18 inittab
drwx------ 2 root root 16384 Sep 20 18:15 lost+found
將Node2設(shè)置為從節(jié)點并丟棄資源數(shù)據(jù)
[root@node2 ~]# drbdadm secondary mydrbd
[root@node2 ~]# drbdadm -- --discard-my-data connect mydrbd
在Node1主節(jié)點上手動連接資源
1 [root@node1 ~]# drbdadm connect mydrbd
最后查看各個節(jié)點狀態(tài),連接已恢復(fù)正常
[root@node1 ~]# cat /proc/drbd
version: 8.3.15 (api:88/proto:86-97)
GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by mockbuild@builder17.centos.org, 2013-03-27 16:04:08
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:36 nr:0 dw:24 dr:185 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
[root@node2 ~]# cat /proc/drbd
version: 8.3.15 (api:88/proto:86-97)
GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by mockbuild@builder17.centos.org, 2013-03-27 16:04:08
0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
ns:0 nr:36 dw:36 dr:0 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
在線幫助文檔參考:(Manual split brain recovery)
http://www.drbd.org/users-guide/s-resolve-split-brain.html
相關(guān)閱讀:
Linux 高可用(HA)集群之DRBD詳解 http://www.linuxidc.com/Linux/2013-08/89035.htm
DRBD中文應(yīng)用指南 PDF http://www.linuxidc.com/Linux/2013-08/89034.htm
CentOS 6.3下DRBD安裝配置筆記 http://www.linuxidc.com/Linux/2013-06/85600.htm
基于DRBD+Corosync實現(xiàn)高可用MySQL http://www.linuxidc.com/Linux/2013-05/84471.htm
CentOS 6.4下DRBD 安裝配置 http://www.linuxidc.com/Linux/2013-09/90536.htm
聯(lián)系客服