Fault Management on Sun's Xeon 5500-based Systems
By user9148476 on Apr 14, 2009
Sun's Xeon 5500 servers and blades have two fault management precincts - the service processor (SP) and the host operating system. This new line of systems has SP-based fault detection and diagnosis for several subsystems, providing a solid base level of fault management irrespective of the host operating system (but you're all running Solaris or OpenSolaris, right? :) A quick rundown of the subsystems that are fault managed:
- Nehalem CPUs: The SP detects and diagnoses all processor uncorrectable errors (UEs). Correctable errors (CEs) are remanded to the host operating system. If the OS is Solaris, processors can be offlined by Solaris FMA if CEs occur too frequently. Also, Solaris FMA will capture error state on CPU UEs and report them upon the next Solaris restart.
- Memory: As with CPUs, memory UEs are diagnosed and reported within the SP. Unlike CPU errors, memory UEs are not visible to the host operating system.
- IO Subsystem: Errors in the IO Hub (Tylersburg) itself are detected and reported via the SP. PCI/PCIE fabric errors are handled by the host operating system (via AER).
- Power, Cooling & Environmentals: The SP detects and diagnoses problems with the bulk power supplies and fan trays. It also monitors the various component sensors (temp, voltage, etc.), reporting components that have gone out of tolerance.
The service processor provides coverage for a good portion of the errors in the system, yet the host operating system can augment the SP, notably in the area of recovery and/or isolation of problematic resources. If you're at all familiar with Solaris, you'll know that Solaris has the capacity to offline individual processor strands (no further software threads are scheduled on the affected strand), retire individual pages of memory (8KB granularity), and cease using problematic IO devices (configurations & active usage permitting).
Another interesting look at Sun's new systems is the fault management functionality for various host operating systems - what diagnosis and recovery features are available straight out of the box:
|Host Operating System|
|CPU Uncorrectable||Yes||Yes||Yes||Memory Correctable||Yes||Yes||Yes|
|IO Hub (Tylersburg)||Yes||Yes||Yes|
|PCI/PCIE Fabric (assuming AER support)||Yes||Yes||Yes 1|
|CPU Strand Offline||Yes||No||No||Memory Page Retire||Yes||No||No||IO Device Retire||Yes||No||No|
|1 Provided Linux kernel has PCI AER support|
Irrespective of your choice of host operating system, Sun's Xeon 5500 suite of systems provides solid fault detection and reporting capabilities. And when coupled with Solaris, fault resilience is improved thanks its recovery and isolation capabilities.
Click here for more Sun blogs about Sun's new Xeon 5500 systems.
1 If running OpenSolaris on an Ultra 27, it is recommended to install build 111 or later to include the fix to CR 6804867 This fix is also planned for a future Solaris 10 Update release.