ARSC's New x86_64 Supercomputer

Newby

Greg Newby, Chief Scientist at the Arctic Region Supercomputing Center (ARSC)at the University of Alaska Fairbanks spoke today about Midnight, their new Sun-based HPC supercomputer. For the curious, the name comes from "Land of the Midnight Sun."

Midnight contains 2312 computational cores or 413 nodes, configured in 19 racks. The aggregate peak performance of its dual core AMD processors is about 12.1 TFLOPs. It was interesting to see that ARSC had opted for a mix of node sizes for Midnight. There are 55 X4600 nodes, each with 16 cores and 64 GB memory. In addition, there are 358 4-core X2200 m2 nodes, each with 16GB memory. It's also noteworthy that ARSC opted for a generous memory configuration of 4 GB per core rather than the more standard 1-2 GB/core.

The system uses a 4X SDR InfiniBand interconnect built around two Voltaire 288-port switches. The interconnect is used for both MPI communication and for connection to the Lustre cluster file system. One switch is configured with 256 attached X2200 m2 nodes while the other hosts the X4600 nodes and the balance of the X2200m2 systems.

The software stack used on Midnight includes the following components:

  • PBS Pro is used to manage the X4600s and the two groups of X2200 m2 systems as three separate resource pools for scheduling purposes
  • SUSE 9 is run on the computational nodes while Red Hat is used on the X4500 (Thumper) storage servers used by Lustre
  • A Voltaire-provided MPI library
  • A variety of compiler suites, including Pathscale, PGI, and Sun Studio
Six x4500 Thumper storage nodes are used to provide about 144 TBytes of raw storage for Lustre. The name space is divided into two pools: a short-term scratch space that is periodically purged and a long-term shared scratch space that is not purged. Neither area is backed up. NFS is used for home directories and Sun's QFS is used to provide back up capabilities. Shared QFS was described as "wonderful." Always nice from a customer.

In closing, Greg shared some of his experiences with testing and accepting such a large system and ended by noting that Sun must improve its support of Linux-based systems in order to be successful with these engagements. As it happened, Peter Braam (founder of Cluster File Systems, maker of Lustre, and now a VP at Sun) was the next speaker at the Consortium. More later on what Peter said in response to Greg's comments regarding Sun's support for Linux.


Comments:

thanks

Posted by mirc on February 20, 2008 at 07:26 PM EST #

thank you very much

Posted by mirc indir on February 20, 2008 at 07:26 PM EST #

thanks for apologies

Posted by forum on February 20, 2008 at 07:26 PM EST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Josh Simons

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today