IBM AIX Continuous Availability Features - IBM Redbooks

2.2.1 Error checking
Run-Time Error Checking
The Run-Time Error Checking (RTEC) facility provides service personnel with a method to
manipulate debug capabilities that are already built into product binaries. RTEC provides
powerful first failure data capture and second failure data capture error detection features.
The basic RTEC framework is introduced in AIX V5.3 TL3, and has now been extended with
additional features. RTEC features include the Consistency Checker and Xmalloc Debug
features. Features are generally tunable with the errctrl command.
Some features also have attributes or commands specific to a given subsystem, such as the
sodebug command associated with new socket debugging capabilities. The enhanced socket
debugging facilities are described in the AIX publications, which can be found online at the
following site:
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp
Kernel stack overflow detection
Beginning with the AIX V5.3 TL5 package, the kernel provides enhanced logic to detect stack
overflows. All running AIX code maintains an area of memory called a stack, which is used to
store data necessary for the execution of the code. As the code runs, this stack grows and
shrinks. It is possible for a stack to grow beyond its maximum size and overwrite other data.
These problems can be difficult to service. AIX V5.3 TL5 introduces an asynchronous
run-time error checking capability to examine if certain kernel stacks have overflowed. The
default action upon overflow detection is to log an entry in the AIX error log. The stack
overflow run-time error checking feature is controlled by the ml.stack_overflow component.
AIX V6.1 improves kernel stack overflow detection so that some stacks are guarded with a
synchronous overflow detection capability. Additionally, when the recovery framework is
enabled, some kernel stack overflows that previously were fatal are now fully recoverable.
Kernel no-execute protection
Also introduced in the AIX V5.3 TL5 package, no-execute protection is set for various kernel
data areas that should never be treated as executable code. This exploits the page-level
execution enable/disable hardware feature. The benefit is immediate detection if erroneous
device driver or kernel code inadvertently make a stray branch onto one of these pages.
Previously the behavior would likely lead to a crash, but was undefined.
This enhancement improves kernel reliability and serviceability by catching attempts to
execute invalid addresses immediately, before they have a chance to cause further damage
or create a difficult-to-debug secondary failure. This feature is largely transparent to the user,
because most of the data areas being protected should clearly be non-executable.
2.2.2 Extended Error Handling
In 2001, IBM introduced a methodology that uses a combination of system firmware and
Extended Error Handling (EEH) device drivers that allow recovery from intermittent PCI bus
errors. This approach works by recovering and resetting the adapter, thereby initiating system
recovery for a permanent PCI bus error. Rather than failing immediately, the faulty device is
“frozen” and restarted, preventing a machine check. POWER6 technology extends this
capability to PCIe bus errors.
20 IBM AIX Continuous Availability Features