Tuesday Mar 03, 2009

Lustre Parallel File Sytem for CFD analysis Part 2

As already said, Lustre stores files, or blocks of files, that are considered as objects, of one or more OSTs. This is called striping in the Lustre terminology. You will need striping :

- if your file is too large to be stored on a single OST.
- if the required aggregate bandwidth for a single file cannot be offered by a single OST.
- if a client, i.e., your program running on the cluster, needs more bandwidth than a single OSS can offer.

Lustre allows you to configure the number of stripes, the stripesize and the servers (OSS) to use, for every file, directory or directory-tree.

Practically, the smallest recommended stripe size is 512 KB because Lustre tries to batch I/O into 512 KB chunks over the network. This is a good amount of data to transfer at one time.

Perhaps you will see a problem, that the file that you are using is considered as a single object. In this case, the file (or object) is stored on a single OST, i.e., a single disk, and you do not see any performance improvement.[Read More]

Tuesday Feb 24, 2009

Lustre Parallel File Sytem for CFD analysis

Whether you are looking at crash simulations, implicit-explicit computations, or CFD analysis, all computing numerical solutions for very different physical models, they have in common, that the size of the data sets becomes bigger and bigger. This is true for the input data, the temporarily computed scratch files, and the final output data. Generally, I/O times have been considered small compared to the runs times of the solver. This may not be longer true today. Not all ISV codes propose a parallel I/O option, and if, it is not always easy to use.

Look for example at the StarCD input files for StarCD V3, V4 and finally StarCCM+ : The same geometry, a 34M cell case, uses 3.5GB for StarCD V3 .geom file and climbs up to 5.1GB when converted into the V4 .ccmg file (or the CCM+ .sim file).[Read More]
About

Be more productive with the Sun High-Performance Computing platform.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today