What’s wrong with Vector3.normalized, or my understanding of it?

Questions & Answers
1

I ran into this issue when trying to normalize a small vector. At first I thought it was a precision issue, but when I tested it myself, as shown in the code below; my normalized values are computed just fine, but Vector3.Normalize just returns Vector3.zero.

I could understand some loss of precision for the sake of performance, but this result is just plain wrong. Do I really need to roll my own Vector3 TrulyNormalized(this Vector3 v) extension function? Should I submit this as a bug? OR - What am I misunderstanding / incorrectly-assuming?

I do see in the docs it explicitly states: “If this vector is too small to be normalized it will be set to zero.” but I really don’t know what “too small” is, nor why this should be the case.

Test code:

    float x = 5E-16f;
    float y = 1E-16f;
    float z = 0;
    float d = x / y;
    float magsqd = x * x + y * y + z * z;
    float mag = Mathf.Sqrt(magsqd);
    string log = (" x: " + x.ToString()+ " y: " + y.ToString()+ " z: " + z.ToString()+ "

sanity check x/y: " + d.ToString() + "
xyz.magsqd: " + magsqd.ToString() + " xyz.mag: " + mag.ToString() );
float an = x / mag;
float bn = y / mag;
float cn = z / mag;
log += ("
My normalized vector xn: " + an.ToString() + " yn: " + bn.ToString() + " zn: " + cn.ToString() );
Vector3 vec = new Vector3(x, y, z);
Vector3 vecNorm = vec.normalized;
log += ("
Vector (sanity check) vx: " + vec.x.ToString() + " vy: " + vec.y.ToString() + " vz: " + vec.z.ToString());
log += ("
Unity Normalized Vector: vxN: " + vecNorm.x.ToString() + " vyN: " + vecNorm.y.ToString() + " vzN: " + vecNorm.z.ToString());
Debug.Log(log);

OutputResults:

x: 5E-16 y: 1E-16 z: 0
 sanity check x/y: 5
 xyz.magsqd: 2.6E-31 xyz.mag: 5.09902E-16
 My normalized vector  xn: 0.9805806 yn: 0.1961161 zn: 0
 Vector (sanity check) vx: 5E-16 vy: 1E-16 vz: 0
 Unity Normalized Vector: vxN: 0 vyN: 0 vzN: 0
3

Edit/update:

Unity defines Vector3 equality/equivilence using this class operator, Unity - Scripting API: Vector3.operator ==, which states two vectors are equal if they fall within a certain range of each other. Regardless of the purpose (optimization, help beginners, whatever…), this is now a mathematical definition.

In order to maintain the consistency of this definition, it is necessary to specifically check against it, during some operations. As an example, the normal operation will always need to check if the vector we are normalizing has a magnitude of zero, because Vector3.zero is the only invalid vector we can input, (and in order to avoid a divide by zero error). Though one CAN check the magnitude against the value 0.0f, we could equivalently express the conditional, as checking to see if the input vector is EQUAL to Vector3.zero.

So, in order to be consistent with the unity definition of vector3 equality, but avoid extra operations, the normalize function checks the magnitude directly against the defined equality range.

(Never would have seen this without Owen’s help.)

Additional FP computation details, and my moaning, remain, below.

End edit:

Normalizing a Vector is a universally standard mathematical operation. It is clearly defined, the same way, in all fields of math.

Except in Unity.

Unity decided to change this universally accepted formula, such that arbitrarily “small” vectors return Vector3.zero when normalized. Apparently, this is not a bug, it’s “by design”. (see Eric’s answer for the actual unity code.)

I have yet to get a good reason for this choice. But, since I don’t trust myself enough; if I get more than let say… 4 up-votes on this answer, and no “good reason” is provided, I WILL submit it as a bug. Please down-vote this answer if you think I’m wrong, and there is a good reason for this limit (though a detailed explanation on why, would be most appreciated.)

After much analysis, research and tests, I have concluded that the generalized claim that this is a “32 bit floating-point limitation”: is simply incorrect! (Sorry Eric.) Sample code in my OP question provides a limited proof of this, code in other professional-level software projects lends weight to this argument, and a rigorous analysis of the FP operations involved confirms it. (Analysis details below).

Another possible reason provided is that this is intended to somehow help us detect inaccurate FP values. But this requires a whole bunch of assumptions about the vector being passed in to the normalize function. These assumptions turned out to be incorrect, in at LEAST one real-world case (which is how I ran across this issue).

Both explanations seem inconsistent with the rest of Unity; everywhere else in my code, it is MY duty to ensue FP accuracy. I agree with this philosophy; and ensuring FP accuracy is most certainly NOT the duty of a normalize vector function.

It seems unbelievable, but if you want a function that performs the STANDARD vector normalization function, consistently, you actually DO need to provide your own. (Easy 'nuff to do, just very surprised it’s necessary.)

Some FP analysis details:

The limit applied by unity affects a Vector3 with a magnitude of less than 1e-5, which is FAR greater than where FP limits are actually hit.
The 32 FP limit is 2^-126, or approximately 1.17e-38: this is the smallest number we can represent, with maximum precision, using a 32bit-FP.

0x0080 0000  = 1.0 × 2−126 ≈ 1.175494351 × 10−38 (min normalized positive value in single precision)

This means that if we square a number (part of the normalize operation) that is smaller than (sqrt(2^-126)), or around 1.32e-19 in decimal, the result will be too small to be represented, with full accuracy, by a FP.
So, if we compared say… 2e-19 against the original vector components, we could determine if any of them might be too small to be squared with full accuracy.

For the normalization operation, there is a safe exception we can make. We don’t mind when accuracy-loss happens to components that are only a tiny fraction of the other components. What matters, is the RELATIVE values of the vector components. In other words, if one component is “small enough” relative to another, it’s contribution is less than the least significant digit of the input vector’s magnitude. In this case, we don’t care if the smaller component loses accuracy; because its value is simply not significant enough to affect the result at all.

This “significance” is, obviously, related to the number of “significant” digits a float can store: 7.2 decimal, lets round up to 8 for worst-cases. The value at which our 2e-19 is too small to be significant, is 2e(-19 + -8) = 2e-11.

So those are our limitations, and exceptions:

  • Exception: If any component is
    greater than 2e-11, you will not lose
    accuracy.
  • Limit: Otherwise, if any
    component is less than 2e-19, you might
    lose accuracy.

Guesses on what unity is doing:

One COULD use the vector’s magnitude to check for the exception: If the vector to be normalized has a magnitude that is greater than a certain value, we KNOW, at least one component is so large that any possible loss-of-accuracy, in other components will NOT be significant. The “certain value” magnitude to check against should also be 2e-11, since 2e-11= sqrt(2e-11^2+0+0).

So perhaps, Unity is checking for the exception, and for optimization purposes, using it as the limit?

Even then, I’m still not sure where the 1e-5 comes from. The only relation I can see to this number, is that it happens to be the approximate value of the least significant digit of a 32bit-float, when using an exponent value of 0. I guess this number DOES represent the LSB value of the normalized output vector’s largest component, but the value is actually being compared against the input vector, not the output vector: so, NOT actually relevant to the computation.

IMHO it’s NOT actually the job of a normalize function to check my FP accuracy, but still, I HOPE I’m wrong about this FP stuff, since it’s the only possibly-valid reason I can think of for this limitation. A sample Vector3 value, that my analysis would say works with full accuracy, but actually fails to be properly processed by a standard normalize function would be the best proof I’m wrong.

4

I think the trick is that Unity doesn’t want you to use their normalize built-ins:
https://docs.unity3d.com/Manual/DirectionDistanceFromOneObjectToAnother.html

That seems like an obscure reference, but it really is what everyone does. You’re going to need the magnitude anyway, so just divide by it yourself.

So who uses the normalize built-ins? I think it’s new users who are somewhat math-phobic and like to see the commands (you can normalize without calling normalize? what?) The most common use is (targetPos-myPos).normalized. You’re stating with two points. If they’re that close together, Unity decided to force you to check for almost-equal.

It’s maybe a funny change, but Unity is a game engine, and it’s documented.

It’s a little like how the built-in lerp clamps to 0-1. Normal lerp functions don’t do that - everyone knows using 1.5 overshoots by 50%. I assume they added it since most new programmers use lerp for an A to B motion.

2

Precision is the cause, and there’s no bug. It’s not about performance, but rather due to the limitations of 32-bit floating point, which isn’t specific to Unity. Hence the code for Normalize is this:

public void Normalize ()
{
	float num = Vector3.Magnitude (this);
	if (num > 1E-05f)
	{
		this /= num;
	}
	else
	{
		this = Vector3.zero;
	}
}

It’s trivial to decompile UnityEngine.dll, so it’s simple to see what it’s “really” doing, except in those cases where the function wraps native code.

5

Try to find the length of vector (3.0e-25f, 4.0e-25f,0.0f) which is expected to be 5.0e-25f.

Now that’s an example of vector that cannot be normalised the usual way.

The usual formula returns a length of zero.

It’s a fundamental issue with floating point.

Let me explain a bit more: there are non zero vectors that can return a length of zero. This is due to the fact that floating point numbers have limited precision.

We can try using doubles which are more precise. Cool. Then the same problems appears at a smaller scale. We did not get rid of the problem, we just made it smaller.

To get rid of the problem, we could think of floating points whose precision can increase arbitrarily. But increasing the precision arbitrarily means to increase the time it takes to compute arbitrarily and that in practice means dropping frames if you hit the wrong case.

I think that what unity did was just pragmatic:
If you know what you are doing it’s documented and you can roll your own solutions instead of using theirs.
On the other hand you are not going to fully get rid of the problem, so let’s just choose a limit and everything below that is just zero so you save a few headaches and in most cases it won’t be a problem as a few digits of precision are ok in a lot of cases.
Choosing a higher limit than the actual limit for floating point numbers is also a pragmatic choice: recursion makes precision errors explode.

To me it makes sense. I am not saying that I would do the same if I were to write a vector library but at least it does not seem too strange to me.

Unless you have objects at very different scales in the same scene you can forget about this issue for the most part.

Problem is that for example in VR you can have a room with a very tiny object and a very big one. and you may be looking outside a window and see even bigger objects. Or you can have objects that move by a very small distance per second.

In those cases you have to be aware of the problem and roll your own solution. I don’t think that there is a known and accepted solution that works in every single possible case exactly because floating point numbers don’t behave exactly the same as real numbers and on the other hand people generally expect floating points to behave the same.