precision
Why is a round-trip conversion via a string not safe for a double?
I found the bug. .NET does the following in clr\src\vm\comnumber.cpp: DoubleToNumber(value, DOUBLE_PRECISION, &number); if (number.scale == (int) SCALE_NAN) { gc.refRetVal = gc.numfmt->sNaN; goto lExit; } if (number.scale == SCALE_INF) { gc.refRetVal = (number.sign? gc.numfmt->sNegativeInfinity: gc.numfmt->sPositiveInfinity); goto lExit; } NumberToDouble(&number, &dTest); if (dTest == value) { gc.refRetVal = NumberToString(&number, ‘G’, DOUBLE_PRECISION, gc.numfmt); goto lExit; } DoubleToNumber(value, … Read more
Get DateTime.Now with milliseconds precision
How can I exactly construct a time stamp of actual time with milliseconds precision? I suspect you mean millisecond accuracy. DateTime has a lot of precision, but is fairly coarse in terms of accuracy. Generally speaking, you can’t. Usually the system clock (which is where DateTime.Now gets its data from) has a resolution of around … Read more
What is the standard (or best supported) big number (arbitrary precision) library for Lua?
Using lbc instead of lmapm would be easier because lbc is self-contained. local bc = require”bc” s=bc.pow(2,1000):tostring() z=0 for i=1,#s do z=z+s:byte(i)-(“0”):byte(1) end print(z)
Set back default floating point print precision in C++
You can get the precision before you change it, with std::ios_base::precision and then use that to change it back later. You can see this in action with: #include <ios> #include <iostream> #include <iomanip> int main (void) { double d = 3.141592653589; std::streamsize ss = std::cout.precision(); std::cout << “Initial precision = ” << ss << ‘\n’; … Read more
How best to sum up lots of floating point numbers?
For “more precise”: this recipe in the Python Cookbook has summation algorithms which keep the full precision (by keeping track of the subtotals). Code is in Python but even if you don’t know Python it’s clear enough to adapt to any other language. All the details are given in this paper.
What class to use for money representation?
Never use a floating point number to represent money. Floating numbers do not represent numbers in decimal notation accurately. You would end with a nightmare of compound rounding errors, and unable to reliably convert between currencies. See Martin Fowler’s short essay on the subject. If you decide to write your own class, I recommend basing … Read more