• Work |
    August 28, 2008

Subnormal numbers

Under IEEE-754, floating point numbers are represented in binary as:

Number = signbit \* mantissa \* 2exponent

There are potentially multiple ways of representing the same number, using decimal as an example, the number 0.1 could be represented as 1\*10-1 or 0.1\*100 or even 0.01 \* 10. The standard dictates that the numbers are always stored with the first bit as a one. In decimal that corresponds to the 1\*10-1 example.

Now suppose that the lowest exponent that can be represented is -100. So the smallest number that can be represented in normal form is 1\*10-100. However, if we relax the constraint that the leading bit be a one, then we can actually represent smaller numbers in the same space. Taking a decimal example we could represent 0.1\*10-100. This is called a subnormal number. The purpose of having subnormal numbers is to smooth the gap between the smallest normal number and zero.

It is very important to realise that subnormal numbers are represented with less precision than normal numbers. In fact, they are trading reduced precision for their smaller size. Hence calculations that use subnormal numbers are not going to have the same precision as calculations on normal numbers. So an application which does significant computation on subnormal numbers is probably worth investigating to see if rescaling (i.e. multiplying the numbers by some scaling factor) would yield fewer subnormals, and more accurate results.

The following program will eventually generated subnormal numbers:

#include <stdio.h>
void main()
double d=1.0;
while (d>0) {printf("%e\\n",d); d=d/2.0;}

Compiling and running this program will produce output that looks like:

$ cc -O ft.c
$ a.out

The downside with subnormal numbers is that computation on them is often deferred to software - which is significantly slower. As outlined above, this should not be a problem since computations on subnormal numbers should be both rare and treated with suspicion.

However, sometimes subnormals come out as artifacts of calculations, for example subtracting two numbers that should be equal, but due to rounding errors are just slightly different. In these cases the program might want to flush the subnormal numbers to zero, and eliminate the computation on them. There is a compiler flag that needs to be used when building the main routine called -fns which enables the hardware to flush subnormals to zero. Recompiling the above code with this flag yields the following output:

$ cc -O -fns ft.c
$ a.out

Notice that the smallest number when subnormals are flushed to zero is 2e-308 rather than 5e-324 that is attained when subnormals are enabled.

Join the discussion

Comments ( 2 )
  • Michael Lyle Friday, August 29, 2008

    A quick search of IEEE-754 shows that it makes no mention of "subnormals". They do talk about "denormalized" numbers, that seem to be what you are talking about.

    I think that there are far more subtle effects of using denormals than you included in your blog, and that an article on floating point should include a strong warning to the effect that the use of floating point in general is not for the faint of heart.

    I suggest thorough study, especially if you are going to consider using denormals.

  • Darryl Gove Friday, August 29, 2008

    Yes denormal == subnormal


    Yes, the recommended text is "what every computer scientist should know about floating point"




Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha

Integrated Cloud Applications & Platform Services