Throughput computing series: Defining Throughput Computing
By glennf on Jan 08, 2008
What is throughput computing?Oxford American defines "throughput" as:
"The amount of materials or items passing through a system or process"
Misguided throughput metrics
- Latency or Response time is not a throughput metric.
- CPU % is not a throughput metric.
- IO wait% is definitely not a throughput metric... or anything other than a measure of idle time
- The "Load average" of a system is not a measure of throughput. OK... you get the idea.
Job level parallelismJob level parallelism is about taking a single job and breaking it into multiple pieces. Say you have 10,000 letters to put stamps on. If it takes 3 seconds per letter, you would need 30,000 seconds or more than 8 hours to complete the task. Now consider you are a teacher and you bring the letters to class. There are 20 students in the class so each student will place stamps on 500 letters. With only 500 letters to complete per student, the job can be done in only 1500 seconds or 25 minutes.
In terms of throughput, one person processes one letter every 3 seconds or 60/3 = 20 letters per minute... and a class of 20 students can process 20\*20 = 400 letters per minute.
ConcurrencyConcurrency comes from running multiple jobs or applications together on a system. A job may be single-threaded or use multiple threads of execution as discussed above. These jobs need not be related or even from the same application. To further increase concurrency, virtualization is often used to run multiple concurrent OS images on the same machine in-order to take advantage of modern multi-threaded systems.
Putting it all together with Chip Multi-ThreadingDenis Sheanan sums up Sun's throughput computing initiative in his paper on CMT as:
"Sun’s Throughput Computing initiative represents a new paradigm in system design, with a
focus toward maximizing the overall throughput of key commercial workloads rather than the speed of a single thread of execution. Chip multi-threading (CMT) processor technology is key to this approach, providing a new thread-rich environment that drives application throughput and processor resource utilization while effectively masking memory access latencies."
- Developing and Tuning Applications on UltraSPARC T1 Chip Multi-threading Systems
- Improving Application Efficiency Through Chip Multi-Threading
- CMT Tuning and resources.
- Chip Level Multiprocessing.
- Chip Multi-Threading CPU utilization and Calculating CMT utilization with performance counters
- Ali-G discusses Niagara2