A long time ago I described how misaligned loads appeared in profiles of 32-bit applications. Since then we've changed the level of detail presented in the Performance Analyzer. When I wrote the article the time spent on-cpu that wasn't User time was grouped as System time. We've now started showing more detail - and more detail is good.
Here's a similar bit of code:
static int i,j;
volatile double *d;
void main ()
for (i=0;i < j; i++)
This code stores into a misaligned double, and that's all we need in order to generate misaligned traps and see how they are shown in the performance analyzer. Here's the hot loop:
Load Object: a.out
User CPU Time Trap CPU Time Name
1.131 0.510 [?] 10928: inc 4, %i1
0. 0. [?] 1092c: ldd [%i2], %f2
0.811 0.380 [?] 10930: faddd %f2, %f0, %f4
0. 0. [?] 10934: std %f4, [%i2]
0.911 0.480 [?] 10938: ldd [%i2], %f2
1.121 0.370 [?] 1093c: faddd %f2, %f0, %f4
0. 0. [?] 10940: std %f4, [%i2]
0.761 0.410 [?] 10944: ldd [%i2], %f2
0.911 0.410 [?] 10948: faddd %f2, %f0, %f4
0.010 0. [?] 1094c: std %f4, [%i2]
0.941 0.450 [?] 10950: ldd [%i2], %f2
1.111 0.380 [?] 10954: faddd %f2, %f0, %f4
0. 0. [?] 10958: cmp %i1, %i5
0. 0. [?] 1095c: ble,pt %icc,0x10928
0. 0. [?] 10960: std %f4, [%i2]
So the first thing to notice is that we're now reporting Trap time rather than aggregating it into System time. This is useful because trap time is intrinsically different from system time, so it's worth displaying it differently. Fortunately the new overview screen highlights the trap time, so it's easy to recognise when to look for it.
Now, you should be familiar with the "previous instruction is to blame rule" for interpreting the output from the performance analyzer. Dealing with traps is no different, the time spent on the next instruction is due to the trap of the previous instruction. So the final load in the loop takes about 1.1s of user time and 0.38s of trap time.
Slight side track about the "blame the last instruction" rule. For misaligned accesses the problem instruction traps and its action is emulated. So the next instruction executed is the instruction following the misaligned access. That's why we see time attributed to the following instruction. However, there are situations where an instruction is retried after a trap, in those cases the next instruction is the instruction that caused the trap. Examples of this are TLB misses or save/restore instructions.
If we recompile the code as 64-bit and set
-xmemalign=8i, then we get a different profile:
User CPU Time Name
For 64-bit code the misaligned operations are fixed in user-land. One (unexpected) advantage of this is that you can take a look at the routines that call the trap handler and identify exactly where the misaligned memory operations are:
0.480main + 0x00000078
0.450main + 0x00000054
0.410main + 0x0000006C
0.380main + 0x00000060
0.370main + 0x00000088
0.310main + 0x0000005C
0.270main + 0x00000068
0.260main + 0x00000074
This is really useful if there are a number of sites and your objective is to fix them in order of performance impact.