一:版本信息
操作系统版本:AIX 61009
数据库版本:11.2.0.3.11(RAC)
二:错误描述
1) crsctl stat res -t命令查看crs资源状态的时候,发现"ora.LISTENER.lsnr"资源状态异常:
ora.LISTENER.lsnr
ONLINE OFFLINE ****1 ##实例1
ONLINE OFFLINE ****2 ##实例2
2)检查监听状态正常
lsnrctl LSNRCTL for IBM/AIX RISC System/6000: Version 11.2.0.3.0 - Production on 11-NOV-2014 10:53:26 Copyright (c) 1991, 2011, Oracle. All rights reserved. Welcome to LSNRCTL, type "help" for information. LSNRCTL> set password Password: The command completed successfully LSNRCTL> status Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER))) STATUS of the LISTENER ------------------------ Alias LISTENER Version TNSLSNR for IBM/AIX RISC System/6000: Version 11.2.0.3.0 - Production Start Date 06-NOV-2014 17:22:05 Uptime 4 days 17 hr. 31 min. 48 sec Trace Level off Security ON: Password SNMP OFF Listener Parameter File /oracle/app/11.2.0/grid/network/admin/listener.ora Listener Log File /oracle/app/11.2.0/grid/log/diag/tnslsnr/bidb4/listener/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=***.***.**.**)(PORT=1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=***.***.**.***)(PORT=1521))) Services Summary... Service "+ASM" has 1 instance(s). Instance "+ASM2", status READY, has 1 handler(s) for this service... Service "bidb" has 1 instance(s). Instance "bidb2", status READY, has 1 handler(s) for this service... The command completed successfully
3)检查监听日志,发现目前监听还能正常接受连接
su - grid
cd $ORACLE_HOME/log/diag/tnslsnr/主机名/listener/alert
host_id='bidb4'
4)检查 CRS alert日志发现有如下报错:
/oracle/app/11.2.0/grid/bin/oraagent.bin(7274512)]CRS-5016:Process "/oracle/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/oracle/app/11.2.0/grid/bin/oraagent.bin" f
or action "start" failed: details at "(:CLSN00010:)" in "/oracle/app/11.2.0/grid/log/bidb4/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-11-07 15:03:21.139:
[/oracle/app/11.2.0/grid/bin/oraagent.bin(7274512)]CRS-5016:Process "/oracle/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/oracle/app/11.2.0/grid/bin/oraagent.bin" f
or action "start" failed: details at "(:CLSN00010:)" in "/oracle/app/11.2.0/grid/log/bidb4/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-11-07 15:03:21.267:
[/oracle/app/11.2.0/grid/bin/oraagent.bin(7274512)]CRS-5016:Process "/oracle/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/oracle/app/11.2.0/grid/bin/oraagent.bin" f
or action "check" failed: details at "(:CLSN00010:)" in "/oracle/app/11.2.0/grid/log/bidb4/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-11-07 15:08:01.278:
##标红的部分报,监听检查报错,这个就跟我的问题联系上了。根据提示 查看oracle/app/11.2.0/grid/log/bidb4/agent/crsd/oraagent_grid/下的oraagent_grid日志(注意可能你要找的信息已经被刷出当前日志了,根据时间点查看相应日志。例如我需要的相关信息就在oraagent_grid.l02日志文件中)
5)查看oraagent_grid.l02日志
根据时间点,查找与"[ora.LISTENER.lsnr]"