I've noticed something odd with floats

Scripting
, ,
1

While tracking down a bug, I was having a variable broadcast it’s value into the debug log. Said variable was simply LegProgress += speed * Time.deltaTime; which was called in FixedUpdate.

Now, the speed was set to just 5, and with fixed update at the default 0.02, this variable is simply adding 0.1 to itself each iteration.

But yet, when it got to 2.7, the next value was 2.799999 instead of 2.8. And then later, after 3.799999 it dropped to 3.899998.

Now, I do understand that there is a fundamental error in precision with floating point numbers, but this is manifesting itself merely at the value of 2.8f. That seems really odd to me; surely we have enough precision to add 0.1f to 2.7f and get 2.8f. I mean, I can move object positions in my scene to such a simple value.

But then it gets worse at merely 1.0 later. If 2.8 were the point where we lost precision from floating point values, wouldn’t the next drop in accuracy come in around 5.3?

And then there is one more oddity:

The very first time this log was called, it declared the first value to be 0.09999999, but then the very next log is 0.2 and 0.3 and so on. If my original calculation of 5 * 0.02 isn’t giving me a result of 0.1, then why isn’t the whole sequence off? Why do I get 26 consecutive entries that properly calculate out to 0.1 intervals, but not the first? And then they drop again ten calculations later?

This just makes no sense to me. I thought that floating point precision lost accuracy with high digit numbers, this is just two. And shouldn’t the lack of precision be at a consistent/technically-predictable rate?

(PS, I found my bug, I neglected to set a different variable when the object was spawned, in case anyone was curious.)

2

Some numbers, even small ones, cannot be represented accurately in floating point form. 2.8 is one such number:

image
image876×270 13.3 KB

Site used: IEEE-754 Floating Point Converter

3 Likes
3

Building on what spiney199 said, approximately half of all floating-point numbers fall within the range (-1,1). As you move further from ±1, precision decreases. In the context of floating-point arithmetic, 2.8 is considered a relatively large number.

You will encounter smaller precision errors when adding 0.01 meters to 0.28 meters compared to adding 0.1 decimeters to 2.8 decimeters. So, it’s better to try and use values in the (-1,1) range whenever possible when precision is important.

3 Likes
4

Okay then, two questions:

One, if this were a variable where that precision of being exactly 2.8 was necessary, or at least if it was hurting me to lose precision at that point, how would I want to go about this? (In my current use I think I’m fine, but I’m curious for future reference.)
Offhand I can think of using an int and storing the value as 28 (or larger if needed) and then when I apply the value in the game world I convert that to a float value and divide by ten. But I don’t know if that would be problematic for other reasons.

Two, how is Unity handling these values? When I set an object’s position in the transform, I can give it a value of 2.8 and it is set to 2.8. Is Unity just lying to me about the value stored in the inspector? Or does it have another layer of precision that it is utilizing somehow?

5

That is Fixed point arithmetic and it is a great simple solution. In fact, most early fractional systems were done that way.

  • store the 2.8 as an integer, but it is 28
  • upon display or transfer out of your “fixed point area,” divide it by 10

Unity transfers your floating point numbers into its internal game engine, and in most cases the internal engine uses the same IEEE single precision numbers, but that isn’t technically guaranteed.

If you MUST have precision, then you MUST store it and never trust it to another API or subsystem unless that system guarantees it will return the same value. Unity makes no such guarantee because it wouldn’t be useful to 99.99% of all games. For instance, parenting and deparenting might do a whole pile of computations on a Transform’s position, then undo those same computations, which will (in floating point) almost always lead to a slightly-different number.

1 Like
6

Floating-point arithmetic can be thought of as a form of “rounding.” A 32-bit floating point (fp32) value can represent at most 2^32 distinct numbers. The idea is to have only certain numbers within a given range, and when a number that doesn’t exist in that range is needed, it is “rounded” to the closest representable value (rounding here is used loosely).

For applications requiring exact precision, such as financial calculations in banking apps, the decimal type is used: https://learn.microsoft.com/en-us/dotnet/api/system.decimal?view=net-9.0. However, decimal has a much smaller range of values and is significantly slower in computational performance.

In general, fixed-point arithmetic (as Kurt mentioned) or a similar technique is often used when precision is critical. The calculations are performed in a type that provides the required precision, and only at the final step is the result cast to the type used by the API. This ensures that precision loss occurs only in the final conversion.

For example, in AI, where FP16 is used for storage, numbers are first cast to FP32 before calculations. All computations are performed in FP32 to minimize precision loss since FP16 can represent at most 2^16 values (and in reality, some values are reserved for special cases like NaN). After the calculations are complete, the final result is cast back to FP16. This means that more precision is only lost at the final casting step, as FP16 has lower precision than FP32.

Don’t know if 2.8 is stored somewhere and eventually assigned in the transform, but when it is assigned in a float, 2.8 doesn’t exist, it gets replaced with the nearest value that exists.

1 Like
7

The moment that you assign 2.8 into a float/Vector3 variable in code, the loss of precision occurs immediately, and there’s no way for Unity’s Inspector code to know later on that you meant 2.8 instead of 2.7999999523162841796875. Similarly, when a Vector3 field is visualized in the inspector, its component values are constantly being backed by floats, so there’s no temporary additional precision in this case either.

Unity isn’t necessarily so much lying to you about the values of float fields, as it’s simply rounding them to make them more easily human readable. It’ll only visualize you up to 7 significant digits of the float when converting it into a string for display in the Inspector. So 0.123456789 becomes "0.1234568", 123456789 becomes "1.234568e+08" etc. When you copy a float value to the clipboard using the Inspector, it also actually copies this rounded string representation of the float, instead of the exact value of the float based on which it was generated.

3 Likes
8

I think everything was already said. Though I’d like to add some general facts about numbers. Whenever you have a fraction of integers in a certain number system, any fraction that has a prime divisor that is not part of the base, it means the fraction can not be represented in that number system. In the decimal system we know about the common issues like 1/3 which is 0.33333… Though actually any divisor that has other prime factors than 2 or 5 has the same issue as the decimal expansion would result in an infinite recursion of some sort. For example 1/7 is 0.142857142857… so those “142857” are repeating infinitely. 1/11 is 0.090909…, 1/13 is 0,076923076923…

So only divisors which are composed of factors from the base would result in “nice” finite decimal expansions.

The binary system of course has only one prime factor and that is 2. So when you want to represent the fraction 1/10 you will run in a problem as you can not reach 1/5 by repeatedly dividing by 2. So many numbers which have no issues in our decimal system would have an infinite recursion in binary and since we have limited digits we can only approximate the value. 0.1 decimal would be 0.000110011001100110011…

You could literally figure those values out by trial and error. 1/8 (0.125) is too large 1/16 (0.0625) too small. Combining 1/16 and 1/32 (0,03125) we’re still to small (0.09375). However adding the next digit (1/64== 0.015625) would be too large. Same for 1/128. However adding 1/256 (0.0078125) and 1/512 (0.00390625) does get us closer to 0.1. That’s why we have that “0011” repeating but you can never actually get to 0.1 that way.

Another important part is that we have a fix number of digits in our binary number. The more digits are required “before” the binary point, the less digits you have behind it. For 32 bit floats I once made this table to visualize it. The smallest possible change is always flipping the last bit on the far right. When the number is 8 millions or larger, there are no digits left behind the decimal point. So we can’t even add 0.5 to that value, only whole numbers. And it gets worse. Over 16 million we loose the next bit, so we can only add or subtract 2. For most applications this doesn’t matter. Though when you really need high precision you can use the decimal type. Though as mentioned it has a smaller range as it’s essentially an integer with an additional decimal divisor / floating point. For more info on the decimal type have a look at this.

3 Likes
9

unity needs to work on 64 bit CPUs with SSE2 tech which is something from 25 years ago (the SSE2). for better precision in math there are new tech developed but obviously you can’t limit the engine to these new features especially if you want to support mobile platforms that may not have them for years after they are released in desktop cpus

for example

10

This makes absolutely no sense. Everything in the transform exists solely for rendering calculations, all of which are performed on the GPU. The GPU handles these computations, and CPU registers have nothing to do with GPU calculations.

AVX on CPUs was developed to enable single instructions to process multiple calculations simultaneously or perform individual calculations with higher precision. GPUs, on the other hand, are inherently designed for parallel execution but are particularly slow for anything beyond FP32, unless we’re talking about workstation GPUs.

Even if Unity supported higher-precision floating-point numbers, such as FP64 (the only higher precision supported by GPU hardware), it would be so slow due to GPU processing limitations that it would be unusable in a modern game. Double-precision calculations on GPUs are 30 to 40 times slower than FP32, meaning that a game using doubles, even if Unity supported them, would run as if it were on a GPU from around 2008.

1 Like
11

Apart from the floating point question, shouldn’t you use Time.fixedDeltaTime instead? :thinking:

12

From the docs on Time.deltaTime:

When this is called from inside MonoBehaviour.FixedUpdate, it returns Time.fixedDeltaTime.

2 Likes