oracle RAC一个节点频繁重启解决
故障现象：
2011年的一次问题，oracle 11gr2 rac + redhat linux ，2节点rac中的其中一个节点频繁重启；
原因分析：
主机日志
VIP发生了漂移，重启后又归位
node1
Nov 23 18:22:27 dtydb2 avahi-daemon[13096]: Withdrawing address record for 10.4.124.242 on bond2.
Nov 23 18:22:31 dtydb2 avahi-daemon[13096]: Withdrawing address record for 169.254.188.250 on bond1.
Nov 23 18:23:10 dtydb2 avahi-daemon[13096]: Registering new address record for 169.254.188.250 on bond1.
Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Registering new address record for 10.4.124.242 on bond2.
Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Withdrawing address record for 10.4.124.242 on bond2.
Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Registering new address record for 10.4.124.242 on bond2.
Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Withdrawing address record for 10.4.124.242 on bond2.
Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Registering new address record for 10.4.124.242 on bond2.
node2
Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Registering new address record for 10.4.124.242 on bond2.
Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Withdrawing address record for 10.4.124.242 on bond2.
Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Registering new address record for 10.4.124.242 on bond2.
Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Withdrawing address record for 10.4.124.242 on bond2.
Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Registering new address record for 10.4.124.242 on bond2.
Nov 23 18:23:34 dtydb1 avahi-daemon[13132]: Withdrawing address record for 10.4.124.242 on bond2.
不能连接ASM，所有重启
ORA-15064: communication failure with ASMinstance
ORA-03113: end-of-file on communicationchannel
ASM日志
and the ASM instance has the alert info
Wed Nov 23 18:22:29 2011
NOTE: client exited [13858]
Wed Nov 23 18:22:29 2011
NOTE: ASMB process exiting, either shutdown is in progress
NOTE: or foreground connected to ASMB was killed.
Wed Nov 23 18:22:29 2011
PMON (ospid: 13797): terminating the instance due to error 481
Wed Nov 23 18:22:29 2011
ORA-1092 : opitsk aborting process
Wed Nov 23 18:22:30 2011
ORA-1092 : opitsk aborting process
Wed Nov 23 18:22:30 2011
ORA-1092 : opitsk aborting process
Wed Nov 23 18:22:30 2011
ORA-1092 : opitsk aborting process
Wed Nov 23 18:22:30 2011
License high water mark = 16
Instance terminated by PMON, pid = 13797
USER (ospid: 9488): terminating the instance
Instance terminated by USER, pid = 948
ocssd.log：has a disk HB, but no network HB,
2011-11-23 18:22:20.512: [ CSSD][1111939392]clssnmPollingThread: node dtydb1 (1) is impending reconfig, flag 394254, misstime 15910
2011-11-23 18:22:20.512: [ CSSD][1111939392]clssnmPollingThread: local diskTimeout set to 27000 ms, remote disk timeout set to 27000, impending reconfig status(1)
2011-11-23 18:22:20.512: [ CSSD][1106696512]clssnmvDHBValidateNCopy: node 1, dtydb1, has a disk HB, but no network HB, DHB has rcfg 216519746, wrtcnt, 1004978, LATS 1030715744, lastSeqNo 946497, uniqueness 1321449141, timestamp 1322043740/933687024
2011-11-23 18:22:21.515: [ CS