Subnormal numbers

Under IEEE-754, floating point numbers are represented in binary as:

Number = signbit \* mantissa \* 2exponent

There are potentially multiple ways of representing the same number, using decimal as an example, the number 0.1 could be represented as 1\*10-1 or 0.1\*100 or even 0.01 \* 10. The standard dictates that the numbers are always stored with the first bit as a one. In decimal that corresponds to the 1\*10-1 example.

Now suppose that the lowest exponent that can be represented is -100. So the smallest number that can be represented in normal form is 1\*10-100. However, if we relax the constraint that the leading bit be a one, then we can actually represent smaller numbers in the same space. Taking a decimal example we could represent 0.1\*10-100. This is called a subnormal number. The purpose of having subnormal numbers is to smooth the gap between the smallest normal number and zero.

It is very important to realise that subnormal numbers are represented with less precision than normal numbers. In fact, they are trading reduced precision for their smaller size. Hence calculations that use subnormal numbers are not going to have the same precision as calculations on normal numbers. So an application which does significant computation on subnormal numbers is probably worth investigating to see if rescaling (i.e. multiplying the numbers by some scaling factor) would yield fewer subnormals, and more accurate results.

The following program will eventually generated subnormal numbers:

#include <stdio.h>

void main()
{
  double d=1.0;
  while (d>0) {printf("%e\\n",d); d=d/2.0;}
}

Compiling and running this program will produce output that looks like:

$ cc -O ft.c
$ a.out
...
3.952525e-323
1.976263e-323
9.881313e-324
4.940656e-324

The downside with subnormal numbers is that computation on them is often deferred to software - which is significantly slower. As outlined above, this should not be a problem since computations on subnormal numbers should be both rare and treated with suspicion.

However, sometimes subnormals come out as artifacts of calculations, for example subtracting two numbers that should be equal, but due to rounding errors are just slightly different. In these cases the program might want to flush the subnormal numbers to zero, and eliminate the computation on them. There is a compiler flag that needs to be used when building the main routine called -fns which enables the hardware to flush subnormals to zero. Recompiling the above code with this flag yields the following output:

$ cc -O -fns ft.c
$ a.out
...
1.780059e-307
8.900295e-308
4.450148e-308
2.225074e-308

Notice that the smallest number when subnormals are flushed to zero is 2e-308 rather than 5e-324 that is attained when subnormals are enabled.

Comments:

A quick search of IEEE-754 shows that it makes no mention of "subnormals". They do talk about "denormalized" numbers, that seem to be what you are talking about.
I think that there are far more subtle effects of using denormals than you included in your blog, and that an article on floating point should include a strong warning to the effect that the use of floating point in general is not for the faint of heart.
I suggest thorough study, especially if you are going to consider using denormals.

Posted by Michael Lyle on August 29, 2008 at 04:24 AM PDT #

Yes denormal == subnormal
http://en.wikipedia.org/wiki/Denormal_number

Yes, the recommended text is "what every computer scientist should know about floating point"
http://docs.sun.com/source/806-3568/ncg_goldberg.html

Thanks!

Darryl.

Posted by Darryl Gove on August 29, 2008 at 04:29 AM PDT #

Post a Comment:
Comments are closed for this entry.
About

Darryl Gove is a senior engineer in the Solaris Studio team, working on optimising applications and benchmarks for current and future processors. He is also the author of the books:
Multicore Application Programming
Solaris Application Programming
The Developer's Edge

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
5
6
8
9
10
12
13
14
15
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks
The Developer's Edge
Solaris Application Programming
Publications
Webcasts
Presentations
OpenSPARC Book
Multicore Application Programming
Docs