Double precision – decimal places

An IEEE double has 53 significant bits (that’s the value of DBL_MANT_DIG in <cfloat>). That’s approximately 15.95 decimal digits (log10(253)); the implementation sets DBL_DIG to 15, not 16, because it has to round down. So you have nearly an extra decimal digit of precision (beyond what’s implied by DBL_DIG==15) because of that. The nextafter() function … Read more

How to calculate float type precision and does it make sense?

The MSDN documentation is nonsensical and wrong. Bad concept. Binary-floating-point format does not have any precision in decimal digits because it has no decimal digits at all. It represents numbers with a sign, a fixed number of binary digits (bits), and an exponent for a power of two. Wrong on the high end. The floating-point … Read more

Is floating-point math consistent in C#? Can it be?

I know of no way to way to make normal floating points deterministic in .net. The JITter is allowed to create code that behaves differently on different platforms(or between different versions of .net). So using normal floats in deterministic .net code is not possible. The workarounds I considered: Implement FixedPoint32 in C#. While this is … Read more

What range of numbers can be represented in a 16-, 32- and 64-bit IEEE-754 systems?

For a given IEEE-754 floating point number X, if 2^E <= abs(X) < 2^(E+1) then the distance from X to the next largest representable floating point number (epsilon) is: epsilon = 2^(E-52) % For a 64-bit float (double precision) epsilon = 2^(E-23) % For a 32-bit float (single precision) epsilon = 2^(E-10) % For a … Read more

Which is the first integer that an IEEE 754 float is incapable of representing exactly?

2mantissa bits + 1 + 1 The +1 in the exponent (mantissa bits + 1) is because, if the mantissa contains abcdef… the number it represents is actually 1.abcdef… × 2^e, providing an extra implicit bit of precision. Therefore, the first integer that cannot be accurately represented and will be rounded is: For float, 16,777,217 … Read more

tech