So... how 'does' float precision work, like really?

Unity Engine
1

So I am doing a calculation like this:

var rotation1 = Quaternion.Euler(0f, 0f, 16.32496f);
var rotation2 = Quaternion.Euler(45f, 45f, 45f);

var result = Quaternion.Inverse(rotation1) * rotation2 * Vector3.one;

Debug.Log(result.ToString("F6")); // returns incorrect (1.240768, -0.058215, 1.207107);

The result I expected was (1.240884, -0.058220, 1.207220). “Aight: float precision”, I lazily thought. But just out of curiosity, I did this:

var result = Quaternion.Euler(90f, 0f, 0f) * Quaternion.Inverse(Quaternion.Euler(0f, 90f, 0f)) * Quaternion.Inverse(Quaternion.Euler(0f, 0f, 90f)) * Vector3.one;

Debug.Log(result.ToString("F6")); // returns a perfect (-1.000000, -1.000000, -1.000000);

What’s going on here?! How can the second equation not have any imprecision?

Ultimately, is it possible to get a correct result on that first equation? I expect a no, but I’d like to understand the difference between these equations and why this happens.

2

There’s a good explanation of how floats work here, as well as the math involved:

3

Thanks!

After having read the article and still wondering about that result I got, I ran it again… and oh gosh this is quite embarrassing.

var result = Quaternion.Euler(90f, 0f, 0f) * Quaternion.Inverse(Quaternion.Euler(0f, 90f, 0f)) * Quaternion.Inverse(Quaternion.Euler(0f, 0f, 90f)) * Vector3.one;

// Let's see what F7 has to say...
Debug.Log(result.ToString("F7")); // returns an imperfect (-0.9999996, -0.9999996, -0.9999996)

7064170--839299--gordon facepalm.jpg

2 Likes