Hello fellow community!
I’ve recently had problem with my script.
Why does this function, when the base is clearly 8 and height is 8, return the result 63.99999??
Is it a conversion error, or what? Shouldn’t the area be 64?
Try adding this line in:
Debug.LogFormat("base {0:F12} Height {1:F12}", b, h);
Debug.LogFormat("b < 8: {0}, h < 8: {1}", b < 8f, h < 8f);
The first line will force printing more digits, and I bet one or both of the boolean values from the second line will be true.
The lesson here is to recognize that a floating point number converted to a base-10 string representation rarely represents the actual value exactly. Your base and height could easily both be 7.9999993, which when printed as a decimal string with no more than 5 digits after the decimal point, is rounded up to a perfect 8. But when multiplied together produce approximately 63.9999888, which when rounded and printed the same way doesn’t quite make it to 64.
The other lesson is that you should be very careful when comparing floating point values for equality, and often for crossing a threshold also. You need to allow for some small amount of error creeping in. For example, if you’re trying to make sure that area equals 64, you likely will want to instead make sure that area is within a tiny range around 64:
if (Mathf.Abs(area - 64f) < 0.00001f)
In your particular case, why does it matter that the area isn’t exactly 64? If it just bugs you, don’t worry about it; nothing is behaving in an unexpected way. If it’s making your code misbehave, you’ll need to reexamine your comparisons, as described above.
And if you’re ambitious about learning more, there’s always the highly cited article What Every Computer Scientist Should Know About Floating-Point Arithmetic.
1 Like