Dependability Benchmarking for Computer Systems
By relling on Aug 20, 2008
Over the past few years, a number of people have been working to develop benchmarks for dependability of computer systems. After all, why should the performance guys have all of the fun? We've collected a number of papers on the subject in a new book, Dependability Benchmarking for Computer Systems, available from the IEEE Computer Society Press and Wiley.
The table of contents includes:
- The Autonomic Computing Benchmark
- Analytical Reliability, Availability, and Serviceability Benchmarks
- System Recovery Benchmarks
- Dependability Benchmarking Using Environmental Test Tools
- Dependability Benchmark for OLTP Systems
- Dependability Benchmarking of Web Servers
- Dependability Benchmark of Automotive Engine Control Systems
- Toward Evaluating the Dependability of Anomaly Detectors
- Vajra: Evaluating Byzantine-Fault-Tolerant Distributed Systems
- User-Relevant Software Reliability Benchmarking
- Interface Robustness Testing: Experience and Lessons Learned from the Ballista Project
- Windows and Linux Robustness Benchmarks with Respect to Application Erroneous Behavior
- DeBERT: Dependability Benchmarking of Embedded Real-Time Off-the-Shelf Components for Space Applications
- Benchmarking the Impact of Faulty Drivers: Application to the Linux Kernel
- Benchmarking the Operating System against Faults Impacting Operating System Functions
- Neutron Soft Error Rate Characterization of Microprocessors
Wow, you can see that there has been a lot of work, by a lot of people to measure system dependability and improve system designs.
The work described in Chapter 2, Analytical Reliability, Availability, and Serviceability Benchmarks, can be seen as we are beginning to publish these benchmark results in various product white papers:
Performance benchmarks have proven useful in driving innovation in the computer industry, and I think dependability benchmarks can do likewise. If you feel that these benchmarks are valuable, then please drop me a note, or better yet, ask your computer vendors for some benchmark results.
I'd like to thank all of the contributors to the book, the IEEE, and Wiley. Karama Kanoun and Lisa Spainhower worked tirelessly to get all of the works compiled (herding the cats) and interfaced with the publisher, great job! Ira Pramanick, Jim Mauro, William Bryson, and Dong Tang collaborated with me on Chapters 2 & 3, thanks team!