Under IEEE-754, floating point numbers are represented in binary as:
Number = signbit \* mantissa \* 2^{exponent}
There are potentially multiple ways of representing the same number, using decimal as an example, the number 0.1 could be represented as 1\*10^{-1} or 0.1\*10^{0} or even 0.01 \* 10
Now suppose that the lowest exponent that can be represented is -100. So the smallest number that can be represented in normal form is 1\*10^{-100}. However, if we relax the constraint that the leading bit be a one, then we can actually represent smaller numbers in the same space. Taking a decimal example we could represent 0.1\*10^{-100}. This is called a subnormal number. The purpose of having subnormal numbers is to smooth the gap between the smallest normal number and zero.
It is very important to realise that subnormal numbers are represented with less precision than normal numbers. In fact, they are trading reduced precision for their smaller size. Hence calculations that use subnormal numbers are not going to have the same precision as calculations on normal numbers. So an application which does significant computation on subnormal numbers is probably worth investigating to see if rescaling (i.e. multiplying the numbers by some scaling factor) would yield fewer subnormals, and more accurate results.
The following program will eventually generated subnormal numbers:
#include <stdio.h>
void main()
{
double d=1.0;
while (d>0) {printf("%e\\n",d); d=d/2.0;}
}
Compiling and running this program will produce output that looks like:
$ cc -O ft.c
$ a.out
...
3.952525e-323
1.976263e-323
9.881313e-324
4.940656e-324
The downside with subnormal numbers is that computation on them is often deferred to software - which is significantly slower. As outlined above, this should not be a problem since computations on subnormal numbers should be both rare and treated with suspicion.
However, sometimes subnormals come out as artifacts of calculations, for example subtracting two numbers that should be equal, but due to rounding errors are just slightly different. In these cases the program might want to flush the subnormal numbers to zero, and eliminate the computation on them. There is a compiler flag that needs to be used when building the main routine called -fns
which enables the hardware to flush subnormals to zero. Recompiling the above code with this flag yields the following output:
$ cc -O -fns ft.c
$ a.out
...
1.780059e-307
8.900295e-308
4.450148e-308
2.225074e-308
Notice that the smallest number when subnormals are flushed to zero is 2e^{-308} rather than 5e^{-324} that is attained when subnormals are enabled.
A quick search of IEEE-754 shows that it makes no mention of "subnormals". They do talk about "denormalized" numbers, that seem to be what you are talking about.
I think that there are far more subtle effects of using denormals than you included in your blog, and that an article on floating point should include a strong warning to the effect that the use of floating point in general is not for the faint of heart.
I suggest thorough study, especially if you are going to consider using denormals.
Yes denormal == subnormal
http://en.wikipedia.org/wiki/Denormal_number
Yes, the recommended text is "what every computer scientist should know about floating point"
http://docs.sun.com/source/806-3568/ncg_goldberg.html
Thanks!
Darryl.