C# Floating Point Confusion

Consider this seemingly simple code:

using UnityEngine;

public class TestFloat : MonoBehaviour {
	private float x = 0.2f;
	private float y = 0.1f;

	public float SumFun() {
		return x + y;
	}

	void Update() {
		float t = SumFun();
		if (t != SumFun())
			Debug.Log("Wow");
	}
}

The question is, why does it ever print anything? The C# standard in 11.1.5 says that floating point operations may be performed in higher precision than the result type of the operation, but in this case it shouldn’t matter because both t and the result of SumFun() should convert to the same value in higher precision, and the result should still be equal.

It appears to me that the x+y in SumFun() is being computed at higher precision, and this higher precision result is being compared to t. But the problem with this is that SumFun() returns a float and not some higher precision value. Surely the compiler can’t just change the semantics of the program like that? I’m just as surely confused.

UPDATE: I tried this program in Visual Studio 2008 on a PC and it doesn’t print anything. So Unity is differing from .Net in this case.

UPDATE: For even more weirdness, change the comparison to:

void Update() {
	float t = SumFun();
	if (t != (float)SumFun())
		Debug.Log("Wow");
}

Casting the float result of SumFun() to a float makes it work!

UPDATE:
I decided to summarize my thoughts about the ensuing discussion since I never got a clear answer as to why the code prints anything.

Most responders point out that floating-point is complicated, or at least more complicated than integer math. In particular I was warned away from relying on == (and !=) between floats. Reasons given include: 1) That certain decimal numbers aren’t representable as floats (and therefore won’t compare as equal. 0.1 being such a number) 2) Floating point arithmetic is therefore order dependent (e.g. 1.0 - 0.1 + 0.1 can give different results depending on the order it is evaluated). 3) Floating point arithmetic is rarely exact, even with high precision.

These are all true facts, but I don’t think they really have anything to do with my code sample. If you changed all occurrences of “float” to “short”, then the program would still be just as wrong, but no one would complain about doing == on shorts (note: I’m not claiming that the program would actually fail if you did that). My point being that == (or !=) between two float values is O.K. as long as the two float values are exactly the same number, and I claim that the above code should ensure that they are. The reason is that the return type of SumFun() is “float”, which should cause any higher precision intermediate value to be truncated before being returned. At this point ‘t’ and the result of SumFun() should both be identical. And if you still think that you cannot compare identical floats with ==, then why does the the x86 include the FCOM instruction?

Now, I may be wrong about C#, and that is really what this question was about. But if in C# it is O.K. for the compiler to not perform the truncate to float when returning a value from SumFun(), then it would make the language very difficult to use. By the same logic you could also just ignore a cast to float, which would be absurd.

So, if Unity isn’t actually compiling this code correctly then it is a bug.

But still, I’m sure my insistence that it is O.K. to use == with floats in certain circumstances won’t sit well with some. I think it comes from being bitten by too many difficult to find bugs involving floating-point math. But another more subtle reason is that mathematics and floating-point on a computer are two different things, and it is easy to forget that. On a computer, a floating-point number is just a bit-pattern, whereas in mathematics it is an abstract idea. The IEEE standard describes how floating-point values are processed by a computer. The rules are complete and deterministic. Given two floats (a and b), the standard completely defines the result (c) of any given computation: c = a op b. a, b, and c are all just bit-patterns. 0.625 is 0x3F200000, and Pi is 0x40490FDA. I can add .625 to Pi and get 3.766593 which is 0x40710FDB. There is no reason I shouldn’t then be able to compare the result to 3.766593, as long as I can ensure it is the same bit-pattern. Unfortunately, computer languages aren’t always so precisely defined, and 11.1.5 of the C# specification seems to leave wiggle room for the compiler to do just about anything. However, 11.1.5 doesn’t say that the result of a floating-point operation may be returned at higher precision, only that the computation may be performed at higher precision.

@macfanpro referred me to the excellent paper “What Every Computer Scientist Should Know About Floating Point Arithmetic” by David Goldberg. I was already familiar with this paper and think that everyone really should read it, but especially language designers and compiler writers :slight_smile: Also, David Eberly is/was working on a book titled “Numerical Computing for Games and Science”. I hope it gets published.

Well, the reason that you are having this problem is due to complicated problem surrounding the float type. For a complicated explanation, look at this: What Every Computer Scientist Should Know About Floating-Point Arithmetic

For a simple explanation: mikeash.com: Friday Q&A 2011-01-04: Practical Floating Point

According to the C# standard, the compiler is free to perform floating-point computation at higher precision than requested, and that includes copying values around.

You should never rely on the result of a floating point calculation being exactly equal to any particular value, including some other value produced by an apparently identical computation. If you must check equality, use Mathf.Approximately() to improve robustness.

I know this is coming late, but I saw an article that seems to really answer your question beyond the whole “floating point can be inexact stuff” hand-waving:

http://blogs.msdn.com/b/davidnotario/archive/2005/08/08/449092.aspx

“it’s still a language choice to give ‘predictability’ … C#, for example, does nothing, if you want narrowing, you will have to insert (float) and (double) casts by hand.”

This explains why t and SumFun() don’t evaluate to equal: in your case, t probably got stored in memory whereas the second call to SumFun() got stored in a higher-precision register. This also explains why when you added an explicit (float) cast, you got the expected results. I find it a strange choice for C# to perform narrowing when casting to float but not when returning a float, but apparently it’s “by design” (although admittedly I don’t know which part of the C# standard makes it so).

So your workaround of casting the result to float is actually what you are supposed to have to do to get reliable results in this case, by design. You do not need to use Mathf.Approximately in cases where you expect an exact equality check to be a meaningful operation; you just have to be a little more careful and explicit about it in C# than in other languages.

Floating point values are seldom exact, no matter the precision, which is why they have this: http://unity3d.com/support/documentation/ScriptReference/Mathf.Approximately.html

This is fundamental computer science stuff. Some floating point numbers can’t actually be represented in a 32 or 64 bit floating point value. It’s always a matter of approximations.

http://unity3d.com/support/documentation/ScriptReference/Mathf.Epsilon.html
http://unity3d.com/support/documentation/ScriptReference/Mathf.Approximately.html

As a practical matter, I’ve found that if you ever think you need to compare a non-int to a specific value, there’s a better solution that doesn’t do that. Either you really should have an int, you really wanted to say either only > or <, or you really do want speed of 0.01 to count as 0. Plus, zero is often a safe value (if(Input.GetAxis("Horizontal")==0) works reliably for me.)

But, yeah, you’ve put your finger on it. Even through (7.0/3)*3 isn’t 7, and isn’t (7.0/4)*4, it should always be equal to another (7.0/3)*3. For your float t = SumFun(); if (t != SumFun()) which is sometimes true, it’s probably doing inlining then promoting t to a double. The cast fixing it seems to confirm that.

I’d guess if you read all the manuals, it really is breaking a C# rule, but every implementation has a list of things it doesn’t do that no one expected it to do anyway.

Usually the resolution of expressions is done in double precision in order to minimize rounding errors. It seems that in unity the function results are also returned as double, even when they are defined as float - it explains why casting the function result to float eliminated the mismatch.
Anyway, comparing floating point values is always tricky, doesn’t matter which computer or compiler you’re using - sooner or later you will get in trouble. The rule of thumb is: never rely on floating point comparisons! I’ve been programming for more than 30 years and, believe me, Mathf.Approximately is an Unity’s handy feature, not a work around.

There is no such thing as comparing the two deterministically. They both represent the two signed limits approaching the number from either side, so an approximation is the best you’re gonna get according to number theory.