昨天去南京某客户那里调优新上线的业务
数据库,在查看alert.log日志时发现在过去的一段时间里,每过几个小时或间隔一段时间,就会报类似以下的内容:
Thu Aug 21 09:01:26 2014 WARNING: Heavy swapping observed on system in last 5 mins. pct of memory swapped in [8.42%] pct of memory swapped out [2.16%]. Please make sure there is no memory pressure and the SGA and PGA are configured correctly. Look at DBRM trace file for more details.
Thu Aug 21 14:56:27 2014 WARNING: Heavy swapping observed on system in last 5 mins. pct of memory swapped in [5.40%] pct of memory swapped out [8.63%]. Please make sure there is no memory pressure and the SGA and PGA are configured correctly. Look at DBRM trace file for more details.
......
Sat Oct 18 22:13:48 2014 WARNING: Heavy swapping observed on system in last 5 mins. pct of memory swapped in [7.76%] pct of memory swapped out [0.33%]. Please make sure there is no memory pressure and the SGA and PGA are configured correctly. Look at DBRM trace file for more details.
客户的环境是IBM P570,AIX 6.1,安装了Oracle 11.2.0.3单实例数据库,物理内存64G,仅仅分配了20G给SGA,采用memory自动管理
查阅了一下MOS,发现是AIX平台上的一个bug,相关文档为:[1508575.1]
对应的数据库和平台:
Oracle Database - Enterprise Edition - Version 11.2.0.3 to 11.2.0.3 [Release 11.2] IBM AIX on POWER Systems (64-bit)
症状:
There is new warning message in alert.log in 11.2.0.3 similar to
WARNING: Heavy swapping observed on system in last 5 mins. pct of memory swapped in [2.08%] pct of memory swapped out [0.12%]. Please make sure there is no memory pressure and the SGA and PGA are configured correctly. Look at DBRM trace file for more details.
On AIX platform this message can be seen even when there is no virtual memory swapping at all. --物理内存足够,而且根本没有使用swap交换空间
You may compare the vmstat from AIX level with DBRM trace file entries to see the differences.
原因:
The issue is caused by unpublished Bug:14731911.
Swap usage messages are based on statistics that do not reflect the actual usage.
The v$osstat does not reflect proper stats for the swap space paging.
解决方法:
Apply Patch:11801934 on top of your IBM AIX on POWER Systems (64-bit) platform.
P.S: Bug is port-specific. --这个bug是针对端口指定的平台的
The issue is fixed in patchset 11.2.0.4 and release 12.1. --说是在12.1的patch中修复了,但实际上12.1还是会有这个问题,会有ora-700错误,详见文档:[ID 1919850.1]
来看一下BUG:14731911的描述:
Bug 属性
B - Defect
2 - Severe Loss of Service
11.2.0.3
96 - Closed, Duplicate Bug
212 - IBM AIX on POWER Systems (64-bit)
2012-10-8
2014-10-11
11801934
11.2.0.3
Port-Specific
Oracle
与此 Bug 相关的知识, 补丁程序和 Bug
相关产品
Oracle Database Products
Oracle Database Suite
Oracle Database
5 - Oracle Database - Enterprise Edition
Hdr: 14731911 11.2.0.3 RDBMS 11.2.0.3 VOS PRODID-5 PORTID-212 11801934
Abstract: FALSE SWAP WARNING MESSAGES PRINTED TO ALERT.LOG ON AIX
*** 10/08/12 04:52 am ***
BUG TYPE CHOSEN
===============
Code
SubComponent: Virtual Operating System
======================================
DETAILED PROBLEM DESCRIPTION
============================
Oracle process seems to check wrong OS local statistic (which include also
FILESYSTEM caching etc.)
Alert log shows WARNING: Heavy swapping observed on system in last 5 mins.
pct of memory swapped in [2.08%] pct of memory swapped out [0.12%].
Please make sure there is no memory pressure and the SGA and PGA
are configured correctly. Look at DBRM trace file for more details.
but this is not reflected at OS level.
DI