Another day, another question
I entered 0.8 * 6 * 8 into the calculator and got 38.4. Why when I do this:
self.label_3.text = 0.8*6*8
i see 38.40000000000001? Same result for:
self.label_3.text = str(0.8*6*8)
Another day, another question
I entered 0.8 * 6 * 8 into the calculator and got 38.4. Why when I do this:
self.label_3.text = 0.8*6*8
i see 38.40000000000001? Same result for:
self.label_3.text = str(0.8*6*8)
Welcome to the joys of floating point arithmetic. This is a general problem with binary mathematics, not a problem with Anvil!
But it’s correct in REPL:
>>> 0.8*6*8
38.4
What’s the difference?
You think in decimal. For you it’s easy to write 1/5=0.2
, but it’s difficult to write 1/3=0.33333
…
Computers think in binary and have the same problems, just with different fractions.
I’m guessing that storing 0.8 in binary requires an infinite number of digits, just like storing 1/3 in decimal does.
When you deal with non integer numbers, you should never assume they are what you think they are or that two numbers that you assume are equal, are actually equal.
Instead of if x == y
you should always do if abs(x - y) < 0.001
or whatever tolerance is acceptable for you. And instead of str(x)
you should do f'{x:.3f}'
. Unfortunately python doesn’t offer a quick way to get rid of both decimal point and trailing zeros, so I usually create a little formatting function that formats the numbers however the app needs.
The repl does some smart formatting for you.
Now I understand. Thanks for the explanation!