Scary calculation in OS X.4 Calculator

[Ignacio Azpiazu]

Trying to figure out why this happens and if it can have any impact on CAD. In the computers and systems I've been able to check out:

OS X.3.# you go to the Calculator, type 200000 * 1.12 and get 224000.
OS X.4, same thing, you get a weird 224000.000000000029.

(In View, precision needs to be set at 11 or higher.)

The widget calculator (not sure how it handles precision) and Excel at 16 decimals seem to be doing OK.

[Anonymous]

Precision errors are common, but usually inconsequential. Remember that the decimal values are being translated to binary and there are limits to how much overhead programmers want to take on. I wouldn't worry about it. In the real world accuracy past 4 to 5 places is rarely needed and past 11 is pretty much unknown.

[TomWaltz]

If I can find a contractor that can build to that accuracy, I'm betting he's a different kind of alien than the president likes to talk about.

[Thomas Holm]

Ignacio wrote:
OS X.3.# you go to the Calculator, type 200000 * 1.12 and get 224000.
OS X.4, same thing, you get a weird 224000.000000000029.

It's a known problem with the Calculator in OSX.4. I think there was another one of the same kind in OSX.3, but I don't remember the figures. It's not a hardware or system problem. Therefore, other calculators work.
I guess Apple don't think people use the Calculator much. At least not enough to give thsi problem enough priority. I think it's embarrassing for them.

[Ignacio Azpiazu]

Thomas wrote:
It's a known problem with the Calculator in OSX.4.

Phew, thanks.