Steps in the Emergency Performance Method

Steps in the Emergency Performance Method

(copy from: http://download-west.oracle.com/docs/cd/B10501_01/server.920/a96532/ch3.htm#9798 )

1. Survey the performance problem and collect the symptoms of the performance problem.

This process should include the following:

* User feedback on how the system is underperforming. Is the problem throughput or response time?
* Ask the question, "What has changed since we last had good performance?" This answer can give clues to the problem; however, getting unbiased answers in an escalated situation can be difficult.


2. Sanity-check the hardware utilization of all components of the application system.

Check where the highest CPU utilization is, and check the disk, memory usage, and network performance on all the system components. This quick process identifies which tier is causing the problem. If the problem is in the application, then shift analysis to application debugging. Otherwise, move on to database server analysis.

3. Determine if the database server is constrained on CPU or if it is spending time waiting on wait events.

(a) If the database server is CPU-constrained, then investigate the following:

* Sessions that are consuming large amounts of CPU at the operating system level
* Sessions or statements that perform many buffer gets at the database level (check V$SESSTAT, V$SQL)
* Execution plan changes causing sub-optimal SQL execution (these can be difficult to locate)
* Incorrect setting of initialization parameters
* Algorithmic issues as a result of code changes or upgrades of all components

(b) If the database sessions are waiting on events, then follow the wait events listed in V$SESSION_WAIT to determine what is causing serialization.

In cases of massive contention for the library cache, it might not be possible to logon or submit SQL to the database. In this case, use historical data to determine why there is suddenly contention on this latch. If most waits are for I/O, then sample the SQL being run by the sessions that are performing all of the I/Os.


4. Apply emergency action to stabilize the system.

This could involve actions that take parts of the application off-line or restrict the workload that can be applied to the system. It could also involve a system restart or the termination of job in process. These naturally have service level implications.

5. Validate that the system is stable.

Having made changes and restrictions to the system, validate that the system is now stable, and collect a reference set of statistics for the database. Now follow the rigorous performance method described earlier in this book to bring back all functionality and users to the system. This process may require significant application re-engineering before it is complete.