Sun Constellation System: Petascale Computing Done Right
By Josh Simons on Jun 26, 2007
Today Sun is revealing in a technology preview at the International Supercomputing Conference in Dresden the approach we will use to build TACC's 500+ TFLOP Ranger system later this year. And other large machines that have not yet been announced.
We call systems built with this approach Sun Constellation Systems. Such systems can scale from TeraFLOPS up into the PetaFLOPs range. To my mind, Sun's approach to Petascale starts with this:
Yes, it's a connector. Specifically, this connector allows three 4X InfiniBand links to be run across a single cable, rather than three separate cables. The cable is also both higher quality and significantly less bulky than the three separate cables taken together. The connector itself is also mechanically and electrically superior to standard InfiniBand connectors. Sun (by which I mean AndyB and others working closely with him) has put a lot of thought into this because to effectively build a petascale system one needs to closely examine current approaches and assess whether just "doing more of the same" really gets you where you need to be or whether new thinking is needed.
The above dense cabling approach complements our ultra-dense blade server and ultra-dense InfiniBand switch components. The blade server supports 48 blades (768 cores) in a single rack/chassis (the rack is the chassis in this case) and the switch sports over 3000 4X Infiniband ports with all of the complexity of a multi-stage network compressed into a single chassis, removing the need for a huge number of cables and a large number of intermediate, discrete InfiniBand switches. Compared to a conventional approach to building a large cluster, the Sun Constellation system uses 1/6 the cables, 20% smaller footprint, and one switch rather than 300 (not a typo.)
The switch component is a double-wide chassis and it looks like this:
The blade chassis looks like this:
I'm walking over to the show floor at ISC here in Dresden in an hour so and will post some photos later tonight.