Performance Comparison: SunStudio vs. GCC on BYTE benchmark (Nbench)


I am often asked about performance differences between Sun Studio Compilers and GCC. And whereas with performance, a single answer never works across the board, I am attempting to put out as much comparative information as I can, to show some of the differences (and hopefully advantages over GCC) as I can.
I have announced previous Sun Studio based SPEC World Record numbers in postings here (like this World Record SPECfp number and this mention of SPEC CPU2006 World Records ), and about STREAM (as in here), but these were not comparative numbers (vs GCC), so this is an attempt to fill that gap.
The first attempt is to take BYTE magazine's BYTEmark benchmark programs that are freely available at this location.
The benchmarks are designed to expose the capabilities of a system's CPU, FPU and memory system and were derived directly, without algorithmic change, from the BYTE web site.
The tests used here were ported to Linux and are actually run on a Solaris 10 system . The HW used here was a SunFire X4100 box with a dual-core 2.4GHz Opteron chip in a standard configuration.
In the following tables, the numbers are all Ratios index against a baseline of AMD K6/233 with 512KB L2-cache, gcc2.7.2.3 and libc-5.4.38 system.
Being ratios, Higher number is Better and so also in the Ratio's column, a ratio > 1 means SunStudio is better than GCC

Test
GCC4.1
SunStudio11
Ratio(SS11/GCC)
Numeric Sort
12.68
10.63
0.84
String Sort
16.07
20.93
1.30
Bitfield
15.15
16.15
1.06
FP Emulator
14.46
31.77
2.19
Fourier
11.69
25.55
2.19
Assignment
37.59
25.11
0.67
Idea
18.52
32.87
1.77
Huffman
15.10
17.02
1.13
Neural Net
24.87
33.98
1.37
Lu Decomposition
45.55
50.20
1.10
Memory Index
20.78
20.397 0.98
Integer Index
15.09
20.844
1.39
FP Index
23.772
35.190
1.48

Numeric Sort, FP Emulator, Idea and Huffman are part of Integer Index
String Sort, Bitfield and Assignment make up the Memory Index
The other tests are part of FP Index

Flags used for each were:
Sun Studio11: -fast -xarch=amd64
GCC-O3 -s -Wall -fomit-frame-pointer -funroll-loops


Comments:

Um, are these timings? Smaller better? Or what...

Posted by Toby on September 26, 2006 at 08:17 AM PDT #

Vijay, do you have these numbers for Solaris as well? Btw, it might be helpful to readers to note that these are "scores" and larger numbers mean better performance.

Posted by Kristofer Spinka on September 26, 2006 at 08:24 AM PDT #

Ugh, sorry about that. It got dropped in the cut-and-paste from the editor to the Roller on which this blog was entered.
I've corrected it above. The numbers are ratios and so larger is better; similarly in the last column a ratio >1 implies Sun Studio is doing a better job than GCC

Posted by Vijay Tatkar on September 26, 2006 at 08:28 AM PDT #

I highlighted the fact that they are actually run on a Solaris 10 box to make that clearer.

Posted by guest on September 26, 2006 at 08:38 AM PDT #

Nice data, and interesting results! Out of curiosity, I wonder how these results compare to previous Sun Studio compilers. I mean, looking at these results it shows that Studio compares quite nicely against GCC, but I don't know if that's always been the case. For those developers who had adhered to GCC because they had a less-than-optimal experience with Studio, I wonder how much better performance they'd see with the current Studio? Great info.

Posted by George Drapeau on September 26, 2006 at 03:35 PM PDT #

Good point!
One of the things that makes this hard is keeping track of which compiler to compare against. Earlier SunStudio compilers (before SunStudio 10) would not be able to exploit AMD64 ABI. Thus they probably need to be compared against earlier (current at that time) versions of GCC running 32bits.
This is a bit harder; I'll see if I can post 32bit versions of these results for both!

Posted by guest on September 27, 2006 at 01:44 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

I have worked with Sun and Oracle for 25 years now; in compilers and tools organization for most of these years followed by a couple of years in Cloud Computing. I am now in ISV Engineering, where our primary task is to improve synergy between Oracle Sun Systems and our rich ISV ecosystem

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Interesting Links