Lustre Parallel File Sytem for CFD analysis Part 2
By Gunter Roeth on Mar 03, 2009
As already said, Lustre stores files, or blocks of files, that are considered as objects, of one or more OSTs. This is called striping in the Lustre terminology. You will need striping :
- if your file is too large to be stored on a single OST.
- if the required aggregate bandwidth for a single file cannot be offered by a single OST.
- if a client, i.e., your program running on the cluster, needs more bandwidth than a single OSS can offer.
Lustre allows you to configure the number of stripes, the stripesize and the servers (OSS) to use, for every file, directory or directory-tree.
Practically, the smallest recommended stripe size is 512 KB because Lustre tries to batch I/O into 512 KB chunks over the network. This is a good amount of data to transfer at one time.
Perhaps you will see a problem, that the file that you are using is considered as a single object. In this case, the file (or object) is stored on a single OST, i.e., a single disk, and you do not see any performance improvement.
You can directly query the files that you are using with
lfs getstripe <filename>
If you see a single OST, well then your I/O is not striped against multiple OSTs and all goes sequentially.
You can change this, by stopping and restarting the application in a new directory with a different stripe count.
lsf setstripe --count=-1 .
A stripe-count of -1 means that all available OSTs should be used. If you rerun the lsf getstripe command now, you should see
that the objects are distributed over all OSTs. Of course, you could have set the new stripe to a single file only. Directories are propagating the striping features. In the example above, all files and directories created from "rundir" will have the same new settings.
Check the manual pages (205pp.) for more options, and how to optimize striping for individual files.