What I’m trying to do:
To display a field in my data table, I want to round it to have 3 decimals.
What I’ve tried and what’s not working:
The code is simple:
self.label_5.text = f"{round(round(self.item['discount_factor_t']*100, 3)} %"
with discount_factor_t being the column from the data table
The result is really weird
You can see that it goes well with most figures, except for 0.999 and 0.995. I checked and the original figures were indeed 0.999 and 0.995. What went wrong?
For maximum calculation speed and precision, computer hardware generally uses binary floating-point. The drawback is that decimal fractions (like 0.1 or 0.01) don’t have an exact counterpart in the binary system. The value that is stored in the database, therefore, is exceedingly close. Typically, it’s within +/- one part in 2^55 of the decimal version.
Humans don’t read binary very well, so we typically convert from binary back to decimal for display and printing. Again, there is often no perfectly exact match, and when there isn’t, you see artifacts like these.
For more details, see
What Every Programmer Should Know About Floating-Point Arithmetic
or
Why don’t my numbers add up?
There are plenty of programming tricks one can use to get around this. Most of them end up converting figures into some whole number multiple of a base figure (in your case, it would be 0.001), and storing that whole number, instead. Whole numbers do have exact representation in the binary system, so they’re not subject to this “problem”.
For example, Stripe takes currency amounts not in dollars, which would have fractional parts, but in pennies.
Also, if interested in (one of many) fixes:
You probably want to look here for the long dry boring and incredibly accurate and detailed pythonic documentation.
Or preferably here for the cliffsnotes version with the actual answer to fix the display issue of your specific problem.
As you are discovering, floating point numbers are weird, and it’s sometimes impossible to exactly represent a decimal fraction using them. Rather than tangle with all the scary details of IEEE floating point, let’s leave the numbers as they are and focus on converting them to a pleasing textual representation.
So our problem is really displaying the number correctly. Conveniently, there’s a simple way to do this!
self.label_5.text = f"{self.item['discount_factor_t']*100:.1f}"
That displays the number as text with 1 point of precision after the decimal point.
Hope that helps!