Stange rounding of numbers in Confirmation

I’m using Confirm to ask, before rolling the change.

If you type into a numerical input box a number like 8,2; 8.2 or 8,3.
8,2 or 8.2 will be transformed into 8.199999999999999
and 8.3 to 8.300000…1

Even if you try to round it:

print(round(self.tb_new_target.text,2))

Exact same output.

I’ve tested different numbers and they work.

It is impossible to round 8.2 to 8.2, but you can format the number 8.2 into the string “8.2” :thinking:

I know, it sounds weird. Check this post for some details: What is the result of multiplying 0.8*6*8? - #4 by stefano.menci

Ok, maybe it is impossible, but how it is possible, that 8.2 or 8.3 are being autoamticly converted into 8.199999999999999 or 8.300000…1?

To solve round problems I like to round it to significant numbers:

round(VALUE, -int(math.floor(math.log10(abs(VALUE)))) + significant_settings)

You can read more about the issues with floating point numbers here: 15. Floating Point Arithmetic: Issues and Limitations — Python 3.11.4 documentation

Rather than trying to round numbers for display, use f strings and specify how many decimal points you want to display, e.g. f"{self.tb_new_target.text:.2f}", to convert the floating point number into a string that has the right number of decimal places.

3 Likes

No, you are not solving the problem with a rounding. 8.199999 IS the rounding of 8.2!

Can you round 1/3 to… something in decimal?
No, you can’t.

8.2 in binary is similar to 1/3 in decimal: it’s stored in memory as an infinite repetition of ones and zeros, just like 1/3 is written with an infinite repetition of threes.

You can decide to write down a finite number of threes, 2, 10 or 1000, but you can’t write down an infinite number of threes. It’s impossible to store the number 1/3 in decimal.

The same goes for 8.2 in binary: computers store numbers with a finite number of digits (bits), just like you write down 0.3333 with a finite number of digits, which means that 8.2 is impossible to store in binary and is rounded to 8.19999999.

A decimal numerical representation of 1/3 will never be exactly 1/3, will be a little off, depending on how many digits you decide to use.

A binary numerical representation of 8.2 will never be exactly 8.2, will be a little off, depending on how many digits you decide to use.

3 Likes

Ok, I wasn’t aware of the binary storing problem.

I will use the f strings to solve such problems. Thanks for help

You would think they solved this common python problem when they introduced decimal as a standard library type but it turns out not really.
Also decimal.Decimal is not supported in skulpt anyways, so rounding, min, or max -ing with f-strings is the more common solution.

2 Likes

I would like to add a clarification: decimals are not fixed floats.

The original question asks about showing a number in the UI. The problem here is not how the number is stored or its precision. The problem is how it appears in the UI, and the solution is what I said in my first answer: format the number into a string.

Perhaps @KR1 has good reasons to use floats, and shouldn’t switch to decimals (even if they existed in Skulpt) just because they don’t look pretty in the UI.

Decimals have some limitations, different from the limitations of the floats. You are not solving the problem, you are trading some limitations for other limitations.

The real solution is to know what’s going on, and to do the right thing to deal with it.

Decimals are typically used in financial applications, where high precision means something different from other applications that typically use floats.

1 Like

Yes, thanks for the clarification, I was really lamenting that they created decimals, which sounds like the solution a person new to python would be attracted to, but they are really for exactly as you said, more precise usually financial applications.
I just meant they don’t fix what a new user might think they do.