Halliburton ProMAX Oil & Gas Application Fast on Sun 6048/X6275 Cluster

Significance of Results

The ProMAX family of seismic data processing tools is the most widely used Oil and Gas Industry seismic processing application. ProMAX is used for multiple applications, from field processing and quality control, to interpretive project-oriented reprocessing at oil companies and production processing at service companies. ProMAX is integrated with Halliburton's OpenWorks Geoscience Oracle Database to access prestack seismic data and populate the database with seismic images. This shows the powerful combination of scientific computing merged with commercial database technology.

A cluster of 48 Sun Blade X6275 server modules in a Sun Blade 6048 Modular System was configured with QDR Infiniband and a Lustre File System to demonstrate performance on ProMAX.

  • The 3D Prestack Kirchhoff Time Migration showed excellent scalability while utilizing the QDR Infiniband Lustre Filesystem.

    • 70808 Traces : 144x improvement going from 1 to 72 nodes
    • 283232 Traces : 98x improvement going from 1 to 96 nodes

  • The super linear scalability is attributed in part to data caching effects.

  • Improved the performance of the current production release of the ProMAX 3D Kirchhoff Time Migration by a factor up to 1.7x through recompilation of the source code using the Intel 11.1 compilers.

High Performance ProMAX allows Halliburton's GeoProbe interpretation application to perform migrations "on the fly" while pulling additional mapping data, well logs, and reservoir data from the OpenWorks Oracle Database.

  • Improves velocity modeling throughput for performing iterative Kirchhoff Migrations

  • Sun Grid Engine can be used to optimize the throughput of multiple migrations and maximize the return on investment of a Sun Blade 6048 Modular System.

Enabling hyperthreading and running 16 threads per node can benefit current and potential ProMAX users running on a Sun Blade X6275 configuration. In the tests run with the code rebuild, hyperthreading outperformed non-hyperthreading by as much as 27%.

Performance Landscape

Note: Results are all run with 16 Threads per Node with HyperThreading Enabled.


ProMAX 3D Prestack Kirchhoff Time Migration
SMP Threads and PVM Mode
Nodes Procs 70808 Traces 283232 Traces
Current Release
Execution Time
(sec)
Code Rebuild
Execution Time
(sec)
Current Release
Execution Time
(sec)
Code Rebuild
Execution Time
(sec)
96 192

18 13
72 144 3 2 22 16
48 96 4 3 38 23
24 48 11 7 76 48
16 32 23 14 117 72
12 24 37 23 165 100
8 16 62 35 258 150
4 8 129 78 514 343
1 2 486 288 2022 1278


ProMAX 3D Prestack Kirchhoff Time Migration
Scalability
Nodes Procs 70808 Traces 283232 Traces
Current Release
Speedup
1-Node
Code Rebuild
Speedup
1-Node
Current Release
Speedup
1-Node
Code Rebuild
Speedup
1-Node
96 192

112 98
72 144 162 144 92 80
48 96 121 96 53 55
24 48 44 41 26 26
16 32 21 20 17 18
12 24 13 12 12 13
8 16 8 8 8 8
4 8 4 4 4 4
1 2 1 1 1 1



ProMAX 3D Prestack Kirchhoff Time Migration
283232 Traces
Hyperthreading Performance Comparison
Nodes Procs Current Release Code Rebuild
8 Threads per
Node Perf
(sec)
16 Threads per
Node Perf
(sec)
8 Threads per
Node Perf
(sec)
16 Threads per
Node Perf
(sec)
24 48 95 76 59 48
16 32 144 117 91 72
12 24 196 165 122 100
8 16 310 258 190 150
4 8 629 514 390 343
1 2 2518 2022 1554 1278

Results and Configuration Summary

Hardware Configuration:
    Sun Blade 6048 Modular System with
      48 x Sun Blade X6275 (Vayu) Server Modules, each with
        4 x 2.93 GHz Intel Xeon QC X5570 processors
        12 x 4 GB memory at 1333 MHz
        2 x 24 GB Internal Flash
    QDR InfiniBand
    Lustre 1.8.0.1 File System

Software Configuration:

    OS: 64-bit SUSE Linux Enterprise Server SLES 10 SP 2
    PVM: Parallel Virtual Machine
    Resource Management: Sun Grid Engine
    Compiler: GNU C++ 4.1.2, Intel 11.1 Compilers
    Database: OpenWorks Database requires Oracle 10g Enterprise Edition
    Additional Libraries: pthreads 2.4, Java 1.6.0_01, BLAS, Stanford Exploration Project Libraries

Benchmark Description

This benchmark compares the current production release of ProMAX built with the GNU C++ and Fortran compilers to builds with the Intel Fortran and C++ compilers. Two different problem sizes were evaluated with the ProMAX 3D Prestack Kirchhoff Time Migration:
  • 70808 traces with 8 msec sample interval and trace length of 4992 msec
  • 283232 traces with 8 msec sample interval and trace length of 4992 msec

The ProMAX processing parameters used for this benchmark:
  • Input data set = Shots
    Minimum output inline = 65
    Maximum output inline = 85
    Inline output sampling interval = 1
    Minimum output xline = 1
    Maximum output xline = 200
    Xline output sampling interval = 1
    Antialias inline spacing = 15
    Antialias xline spacing = 15
    Stretch Mute Aperature Limit with Maximum Stretch = 15
    Image Gather Type = Full Offset Image Traces
    No Block Moveout
    Number of Alias Bands = 10
    3D Amplitude Phase Correction
    No compression
    Maximum Number of Cache Blocks = 500000

The compiler flags used for the various builds:
  • The Current Production Release Code was built with GNU Fortran and C++ flags.
    -O3 -m64 -march=x86-64 -mieee-fp -mfpmath=sse -msse2 -fforce-addr -fno-inline-functions

  • The application was rebuilt with the Intel Fortran and C++ flags.
    -xSSE4.2 -O3 -ipo -no-prec-div -static -m64 -ftz -fast-transcendentals -fp-speculation=fast

Key Points and Best Practices

Super linear scalability of the 70808 trace case for the larger node runs can be attributed to the fact that the dataset decomposition fits in cache which is shared by multiple threads per core.

See Also

Disclosure Statement

The following are trademarks or registered trademarks of Halliburton/Landmark Graphics: ProMAX, GeoProbe, OpenWorks.
Comments:

Post a Comment:
Comments are closed for this entry.
About

BestPerf is the source of Oracle performance expertise. In this blog, Oracle's Strategic Applications Engineering group explores Oracle's performance results and shares best practices learned from working on Enterprise-wide Applications.

Index Pages
Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today