Precision : Of Singles and Doubles


Typically, a single precision digit (aka float32) has a decimal digits precision from 6-9 digits. If you type in:


In Matlab, you’ll find that it gives 1.1920929e-07. This is 2^-23.
Put simply, this is the smallest value required to to effect the next largest single-precision number, when the number is 1.

So for 980e6 for example, the smallest value required to represent the next largest single-precision number will be

980e6 * eps('single') == 1.1682510e+02.

Basically, if you have a single-precision value 980e6, the smallest confirmed granularity is ~116.8!  So when you are doing calculations involving large numbers, it will be good to keep this in mind. When you have something like this:

single( 980e6) + single(k(1))

single(k(1)) needs to be > ~116.8 in order to effect change upon single(980e6).
Basically I’ve said the same thing several different ways.

eps(‘double’) is 2^-52

which is much smaller.

For double-precision, it would be 980e6 * eps(‘double’) == 2.176037128265307e-07. Pretty small. Not something to fret about.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s