Heartbeat是一個基於Linux開源的，被廣泛使用的高可用集群系統。我們可以基於Heartbeat構建web高可用服務環境。本文在CentOS 6.5下做了一個簡單示例，並對其日志進行了初步分析，供大家參考。
有關Heartbeat的相關知識，可以參考： Heartbeat 集群組件概述 Heartbeat 安裝及配置
###主機名配置，與/etc/hosts中的解析兩者配置保持一致
[root@orasrv1 ~]# more /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=orasrv1.xlk.com
[root@orasrv1 ~]# more /etc/hosts
127.0.0.1 localhost.localdomain localhost
#public ip
192.168.21.10 orasrv1.xlk.com orasrv1 ###node 1
192.168.21.13 orasrv2.xlk.com orasrv2 ###node 2
#private ip
192.168.144.128 orasrv1-priv.xlk.com orasrv1-priv
192.168.144.129 orasrv2-priv.xlk.com orasrv2-priv
[root@orasrv1 ~]# scp /etc/hosts orasrv2:/etc/hosts
[root@orasrv1 ~]# ifconfig |grep eth -A1
eth0 Link encap:Ethernet HWaddr 00:0C:29:35:31:E5 #eth0對外提供服務
inet addr:192.168.21.10 Bcast:192.168.21.255 Mask:255.255.255.0
--
eth1 Link encap:Ethernet HWaddr 00:0C:29:35:31:EF #eth1用作心跳網絡
inet addr:192.168.144.128 Bcast:192.168.144.255 Mask:255.255.255.0
二、配置等效性
###節點1
# ssh-keygen -t rsa -f ~/.ssh/id_rsa -P ''
# ssh-copy-id -i ./.ssh/id_rsa.pub [email protected]
# ssh orasrv2 date;ssh 192.168.21.13 date;
###節點2
# ssh-keygen -t rsa -f ~/.ssh/id_rsa -P ''
# ssh-copy-id -i ./.ssh/id_rsa.pub [email protected]
# ssh orasrv1 date;ssh 192.168.21.10 date;
三、配置時間同步此處略。可以參考：配置Linux 11G R2 RAC NTP服務
四、安裝httpd
節點1：orasrv1
# yum install httpd
# service httpd start ###啟動服務
# echo "orasrv1.xlk.com" >>/var/www/html/index.html
###打開浏覽器看看能否正常訪問，http://192.168.21.10/index.html
###如能正常訪問，關閉服務，關閉自啟動
# service httpd stop
# chkconfig httpd off
節點2：orasrv2
# yum install httpd
# service httpd start ###啟動服務
# echo "orasrv2.xlk.com" >>/var/www/html/index.html
###打開浏覽器看看能否正常訪問，http://192.168.21.13/index.html
###如能正常訪問，關閉服務，關閉自啟動
# service httpd stop
# chkconfig httpd off
五、配置heartbeat
1、配置認證方式
[root@orasrv1 ~]# vi /etc/ha.d/authkeys |grep -v ^#
auth 1
1 md5 74a6a503b5bec9373b22bb630608df41 ###使用md5驗證
[root@orasrv1 ~]# chmod 600 /etc/ha.d/authkeys
2、配置ha.cf
[root@orasrv1 ~]# vi /etc/ha.d/ha.cf
logfile /var/log/ha-log
keepalive 2
deadtime 15
warntime 10
initdead 60
udpport 694
bcast eth1
auto_failback off
node orasrv1.xlk.com
node orasrv2.xlk.com
ping 192.168.21.254
#respawn hacluster /usr/lib64/heartbeat/ipfail
3、配置haresources
[root@orasrv1 ~]# vi /etc/ha.d/haresources
orasrv1.xlk.com IPaddr::192.168.21.1/24/eth0 httpd
4、同步到第二節點
# scp -p /etc/ha.d/{ha.cf,authkeys,haresources} 192.168.21.13:/etc/ha.d/
六、驗證測試
1、節點1啟動heartbeat
###備注：本樣例並未使用到共享文件系統，如配置drbd，或者nfs等。
[root@orasrv1 ~]# service heartbeat start
Starting High-Availability services:
2015/11/06_11:46:51 INFO: Resource is stopped ###此處提示資源處於停止狀態
Done.
###以下為基於節點1上的日志分析
[root@orasrv1 ~]# tail -fn 50 /var/log/ha-log
heartbeat[4373]: 2015/11/06_11:46:51 info: Version 2 support: false
heartbeat[4373]: 2015/11/06_11:46:51 WARN: Logging daemon is disabled --enabling logging daemon is recommended
heartbeat[4373]: 2015/11/06_11:46:51 info: **************************
heartbeat[4373]: 2015/11/06_11:46:51 info: Configuration validated. Starting heartbeat 2.1.4
heartbeat[4374]: 2015/11/06_11:46:51 info: heartbeat: version 2.1.4
heartbeat[4374]: 2015/11/06_11:46:52 info: Heartbeat generation: 1446714158
heartbeat[4374]: 2015/11/06_11:46:52 info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth0
heartbeat[4374]: 2015/11/06_11:46:52 info: glib: UDP Broadcast heartbeat closed on port 694 interface eth0 - Status: 1
heartbeat[4374]: 2015/11/06_11:46:52 info: glib: ping heartbeat started.
heartbeat[4374]: 2015/11/06_11:46:52 info: G_main_add_TriggerHandler: Added signal manual handler
heartbeat[4374]: 2015/11/06_11:46:52 info: G_main_add_TriggerHandler: Added signal manual handler
heartbeat[4374]: 2015/11/06_11:46:52 info: G_main_add_SignalHandler: Added signal handler for signal 17
heartbeat[4374]: 2015/11/06_11:46:52 info: Local status now set to: 'up'
heartbeat[4374]: 2015/11/06_11:46:52 info: Link 192.168.21.254:192.168.21.254 up.
heartbeat[4374]: 2015/11/06_11:46:52 info: Status update for node 192.168.21.254: status ping
heartbeat[4374]: 2015/11/06_11:46:52 info: Link orasrv1.xlk.com:eth0 up.
heartbeat[4374]: 2015/11/06_11:47:52 WARN: node orasrv2.xlk.com: is dead
###上面的日志可以看出節點1上的heartbeat已經成功啟動，此時尚未開始分配資源
###緊接著下面的提示節點2處於死亡狀態，其時間間隔為60s(46:52s~47:52s)，即我們配置文件中的initdead
###也就是說等待1分鐘還沒有探測到對方心跳，此時認為節點2死亡，注意，這裡的死亡不等同於deadtime
###下面開始資源分配
heartbeat[4374]: 2015/11/06_11:47:52 info: Comm_now_up(): updating status to active
heartbeat[4374]: 2015/11/06_11:47:52 info: Local status now set to: 'active'
heartbeat[4374]: 2015/11/06_11:47:52 WARN: No STONITH device configured.
heartbeat[4374]: 2015/11/06_11:47:52 WARN: Shared disks are not protected.
heartbeat[4374]: 2015/11/06_11:47:52 info: Resources being acquired from orasrv2.xlk.com.
harc[4385]: 2015/11/06_11:47:52 info: Running /etc/ha.d/rc.d/status status
mach_down[4419]: 2015/11/06_11:47:52 info: /usr/share/heartbeat/mach_down: nice_failback: foreign resources acquired
mach_down[4419]: 2015/11/06_11:47:52 info: mach_down takeover complete for node orasrv2.xlk.com.
heartbeat[4374]: 2015/11/06_11:47:52 info: Initial resource acquisition complete (T_RESOURCES(us))
heartbeat[4374]: 2015/11/06_11:47:52 info: mach_down takeover complete.
IPaddr[4457]: 2015/11/06_11:47:52 INFO: Resource is stopped
heartbeat[4386]: 2015/11/06_11:47:52 info: Local Resource acquisition completed.
harc[4518]: 2015/11/06_11:47:52 info: Running /etc/ha.d/rc.d/ip-request-resp ip-request-resp
ip-request-resp[4518]: 2015/11/06_11:47:52 received ip-request-resp IPaddr::192.168.21.1/24/eth0 OK yes
ResourceManager[4537]: 2015/11/06_11:47:52 info: Acquiring resource group: orasrv1.xlk.com IPaddr::192.168.21.1/24/eth0 httpd
IPaddr[4563]: 2015/11/06_11:47:52 INFO: Resource is stopped ###上一條日志獲取資源組，下一條日志啟動資源
ResourceManager[4537]: 2015/11/06_11:47:52 info: Running /etc/ha.d/resource.d/IPaddr 192.168.21.1/24/eth0 start
IPaddr[4658]: 2015/11/06_11:47:53 INFO: Using calculated netmask for 192.168.21.1: 255.255.255.0
IPaddr[4658]: 2015/11/06_11:47:53 INFO: eval ifconfig eth0:0 192.168.21.1 netmask 255.255.255.0 broadcast 192.168.21.255
IPaddr[4629]: 2015/11/06_11:47:53 INFO: Success ###vip 分配成功
ResourceManager[4537]: 2015/11/06_11:47:53 info: Running /etc/init.d/httpd start ###開啟httpd服務
heartbeat[4374]: 2015/11/06_11:48:03 info: Local Resource acquisition completed. (none)
heartbeat[4374]: 2015/11/06_11:48:03 info: local resource transition completed.
###驗證VIP分配，如下，eth0:0上已獲得VIP
[root@orasrv1 ~]# ifconfig |grep eth -A1
eth0 Link encap:Ethernet HWaddr 00:0C:29:35:31:E5
inet addr:192.168.21.10 Bcast:192.168.21.255 Mask:255.255.255.0
--
eth0:0 Link encap:Ethernet HWaddr 00:0C:29:35:31:E5
inet addr:192.168.21.1 Bcast:192.168.21.255 Mask:255.255.255.0
--
eth1 Link encap:Ethernet HWaddr 00:0C:29:35:31:EF
inet addr:192.168.144.128 Bcast:192.168.144.255 Mask:255.255.255.0
###驗證httpd服務，如下，此時透過VIP訪問到主頁，從orasrv1，即節點1返回數據
http://192.168.21.1/index.html
orasrv1.xlk.com
2、節點2啟動heartbeat
[root@orasrv2 ~]# service heartbeat start
Starting High-Availability services:
2015/11/06_11:50:22 INFO: Resource is stopped
Done.
###下面的日志為節點1，繼續跟蹤節點1的日志
###從下面的日志可以得知orasrv2.xlk.com: status up以及remote resource transition completed
heartbeat[4374]: 2015/11/06_11:50:23 info: Link orasrv2.xlk.com:eth0 up.
heartbeat[4374]: 2015/11/06_11:50:23 info: Status update for node orasrv2.xlk.com: status init
heartbeat[4374]: 2015/11/06_11:50:23 info: Status update for node orasrv2.xlk.com: status up
harc[4809]: 2015/11/06_11:50:23 info: Running /etc/ha.d/rc.d/status status
harc[4825]: 2015/11/06_11:50:23 info: Running /etc/ha.d/rc.d/status status
heartbeat[4374]: 2015/11/06_11:50:25 info: Status update for node orasrv2.xlk.com: status active
harc[4840]: 2015/11/06_11:50:25 info: Running /etc/ha.d/rc.d/status status
heartbeat[4374]: 2015/11/06_11:50:25 info: remote resource transition completed.
3、節點1停止heartbeat
###通過下面的命令來停止節點1上的heartbeat
[root@orasrv1 ~]# service heartbeat stop
Stopping High-Availability services:
Done.
###此時繼續觀察節點1的日志變化情況
heartbeat[4374]: 2015/11/06_11:53:13 info: Heartbeat shutdown in progress. (4374)
heartbeat[4873]: 2015/11/06_11:53:13 info: Giving up all HA resources. ###節點1開始放棄資源
ResourceManager[4886]: 2015/11/06_11:53:14 info: Releasing resource group: orasrv1.xlk.com IPaddr::192.168.21.1/24/eth0 httpd
ResourceManager[4886]: 2015/11/06_11:53:14 info: Running /etc/init.d/httpd stop ###停止httpd服務
ResourceManager[4886]: 2015/11/06_11:53:14 info: Running /etc/ha.d/resource.d/IPaddr 192.168.21.1/24/eth0 stop
IPaddr[4975]: 2015/11/06_11:53:14 INFO: ifconfig eth0:0 down
IPaddr[4946]: 2015/11/06_11:53:14 INFO: Success
heartbeat[4873]: 2015/11/06_11:53:14 info: All HA resources relinquished.
###所有的HA資源在節點1放棄完畢，下面開始kill heartbeat相關進程
heartbeat[4374]: 2015/11/06_11:53:15 WARN: 1 lost packet(s) for [orasrv2.xlk.com] [95:97]
heartbeat[4374]: 2015/11/06_11:53:15 info: No pkts missing from orasrv2.xlk.com!
heartbeat[4374]: 2015/11/06_11:53:16 info: killing HBREAD process 4381 with signal 15
heartbeat[4374]: 2015/11/06_11:53:16 info: killing HBFIFO process 4377 with signal 15
heartbeat[4374]: 2015/11/06_11:53:16 info: killing HBWRITE process 4378 with signal 15
heartbeat[4374]: 2015/11/06_11:53:16 info: killing HBREAD process 4379 with signal 15
heartbeat[4374]: 2015/11/06_11:53:16 info: killing HBWRITE process 4380 with signal 15
heartbeat[4374]: 2015/11/06_11:53:16 info: Core process 4378 exited. 5 remaining
heartbeat[4374]: 2015/11/06_11:53:16 info: Core process 4379 exited. 4 remaining
heartbeat[4374]: 2015/11/06_11:53:16 info: Core process 4381 exited. 3 remaining
heartbeat[4374]: 2015/11/06_11:53:16 info: Core process 4377 exited. 2 remaining
heartbeat[4374]: 2015/11/06_11:53:16 info: Core process 4380 exited. 1 remaining
heartbeat[4374]: 2015/11/06_11:53:16 info: orasrv1.xlk.com Heartbeat shutdown complete.
###觀察節點2，此時也獲取VIP
[root@orasrv2 ~]# ifconfig |grep eth -A1
eth0 Link encap:Ethernet HWaddr 00:0C:29:F1:2E:E4 #Author : Leshami
inet addr:192.168.21.13 Bcast:192.168.21.255 Mask:255.255.255.0
--
eth0:0 Link encap:Ethernet HWaddr 00:0C:29:F1:2E:E4 #Blog : http://blog.csdn.net/leshami
inet addr:192.168.21.1 Bcast:192.168.21.255 Mask:255.255.255.0
--
eth1 Link encap:Ethernet HWaddr 00:0C:29:F1:2E:EE
inet addr:192.168.144.129 Bcast:192.168.144.255 Mask:255.255.255.0
###驗證httpd服務，如下，此時透過VIP訪問到主頁，從orasrv2，即節點2返回數據
http://192.168.21.1/index.html
orasrv2.xlk.com
4、使用hb_standby切換
###下面通過hb_standby將服務切換回節點1，切換前啟動節點1上的heartbeat
[root@orasrv2 ~]# /usr/share/heartbeat/hb_standby
2015/11/06_11:56:05 Going standby [all].
###繼續跟蹤節點1的日志
heartbeat[5114]: 2015/11/06_11:55:23 info: Version 2 support: false
heartbeat[5114]: 2015/11/06_11:55:23 WARN: Logging daemon is disabled --enabling logging daemon is recommended
heartbeat[5114]: 2015/11/06_11:55:23 info: **************************
heartbeat[5114]: 2015/11/06_11:55:23 info: Configuration validated. Starting heartbeat 2.1.4
heartbeat[5115]: 2015/11/06_11:55:23 info: heartbeat: version 2.1.4
heartbeat[5115]: 2015/11/06_11:55:23 info: Heartbeat generation: 1446714159
heartbeat[5115]: 2015/11/06_11:55:23 info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth0
heartbeat[5115]: 2015/11/06_11:55:23 info: glib: UDP Broadcast heartbeat closed on port 694 interface eth0 - Status: 1
heartbeat[5115]: 2015/11/06_11:55:23 info: glib: ping heartbeat started.
heartbeat[5115]: 2015/11/06_11:55:23 info: G_main_add_TriggerHandler: Added signal manual handler
heartbeat[5115]: 2015/11/06_11:55:23 info: G_main_add_TriggerHandler: Added signal manual handler
heartbeat[5115]: 2015/11/06_11:55:23 info: G_main_add_SignalHandler: Added signal handler for signal 17
heartbeat[5115]: 2015/11/06_11:55:23 info: Local status now set to: 'up'
heartbeat[5115]: 2015/11/06_11:55:23 info: Link 192.168.21.254:192.168.21.254 up.
heartbeat[5115]: 2015/11/06_11:55:23 info: Status update for node 192.168.21.254: status ping
heartbeat[5115]: 2015/11/06_11:55:23 info: Link orasrv1.xlk.com:eth0 up.
heartbeat[5115]: 2015/11/06_11:55:25 info: Link orasrv2.xlk.com:eth0 up.
heartbeat[5115]: 2015/11/06_11:55:25 info: Status update for node orasrv2.xlk.com: status active
harc[5124]: 2015/11/06_11:55:25 info: Running /etc/ha.d/rc.d/status status
heartbeat[5115]: 2015/11/06_11:55:25 info: Comm_now_up(): updating status to active
heartbeat[5115]: 2015/11/06_11:55:25 info: Local status now set to: 'active'
heartbeat[5115]: 2015/11/06_11:55:26 info: remote resource transition completed.
heartbeat[5115]: 2015/11/06_11:55:26 info: remote resource transition completed.
heartbeat[5115]: 2015/11/06_11:55:26 info: Local Resource acquisition completed. (none)
heartbeat[5115]: 2015/11/06_11:55:26 info: Initial resource acquisition complete (T_RESOURCES(them))
###上面的日志表面，此時節點1已經啟動完畢，並且2節點均處於活動狀態
###下面是開啟hb_standby之後輸出的日志信息，所有的資源被節點1接管
heartbeat[5115]: 2015/11/06_11:56:06 info: orasrv2.xlk.com wants to go standby [all]
heartbeat[5115]: 2015/11/06_11:56:06 info: standby: acquire [all] resources from orasrv2.xlk.com
heartbeat[5143]: 2015/11/06_11:56:06 info: acquire all HA resources (standby).
ResourceManager[5156]: 2015/11/06_11:56:06 info: Acquiring resource group: orasrv1.xlk.com IPaddr::192.168.21.1/24/eth0 httpd
IPaddr[5182]: 2015/11/06_11:56:06 INFO: Resource is stopped
ResourceManager[5156]: 2015/11/06_11:56:06 info: Running /etc/ha.d/resource.d/IPaddr 192.168.21.1/24/eth0 start
IPaddr[5277]: 2015/11/06_11:56:06 INFO: Using calculated netmask for 192.168.21.1: 255.255.255.0
IPaddr[5277]: 2015/11/06_11:56:06 INFO: eval ifconfig eth0:0 192.168.21.1 netmask 255.255.255.0 broadcast 192.168.21.255
IPaddr[5248]: 2015/11/06_11:56:06 INFO: Success
ResourceManager[5156]: 2015/11/06_11:56:06 info: Running /etc/init.d/httpd start
heartbeat[5143]: 2015/11/06_11:56:07 info: all HA resource acquisition completed (standby).
heartbeat[5115]: 2015/11/06_11:56:07 info: Standby resource acquisition done [all].
heartbeat[5115]: 2015/11/06_11:56:07 info: remote resource transition completed.
七、小結a、對於集群上節點之間的切換實質上是資源的接管，通常包括VIP分配回收，磁盤卸載掛載，以及服務的啟停等。
b、對於備用節點，其相應的共享磁盤(非嚴格意義上)不可讀寫，服務處於停止狀態。
