FBX Importer Converts RGBA Floats to Ints and Destroys Precision - How to Fix?

I am storing some float values in RGBA vertex colors and exporting them from 3ds Max into an FBX. The values show up correctly in the .fbx file. Specifically some of the values are nice simple values like 0.5 and 0.25.

When I import the FBX file into unity, I discover that the values have been changed. 0.5 becomes 0.502. I assume this must be because the importer is temporarily converting the values to 0-255 Ints and then back to floats when storing them in the mesh. 0.5 becomes 128. 128/255 = 0.50196 which gets rounded to 0.502.

It is critical that these values stay accurate. I need 0.500. NOT 0.502.

Is there a way to fix this problem?

What do you need that much precision for? Can you store them in another set of UVs instead?

No I can’t store them in another set of UVs. I want the other UVs to be reserved for lightmaps. Plus, I’m storing four values, so I’m using all four R G B and A channels.

I need the precision for specialized mapping transformations. It never even occurred to me that Unity would be casting the floats to Int before storing them.

Here’s the C# Script I’m using in Unity to print the mesh vertex color values to check them:

Mesh mesh = GetComponent().mesh;
int i = 0;
while (i < mesh.colors.Length) {
print(mesh.colors*);*

  • i++;*
    }
    mesh.colors stores a float value apparently because that’s what gets printed out. It seems very absurd that the .colors array would be stored as floats… even though Unity’s importer is temporarily casting to Int and then storing as float. It makes storing as float worthless and impairs data packing solutions.

OK, maybe mesh.colors doesn’t store floats after all. I changed the loop to this:

while (i < mesh.colors.Length) {
mesh.colors = new Color(0.5F, 0.5F, 0.5F, 0.5F);
print(mesh.colors*);*
i++;
}
The script doesn’t show any errors so I’m left to believe the new Color line is storing to mesh.colors. But the print line doesn’t print 0.5; it prints 0.502!!
I guess there’s some kind of implicit cast going on when .colors is accessed. :frowning:

It will get converted to .502 regardless; there’s no way to keep it as a “pure” float, because the mesh actually stores colors as bytes. For this reason I’d recommend using Mesh.colors32, which is the “real” vertex color data. It’s faster and uses less memory than Mesh.colors since there’s no superfluous float conversion.

–Eric

Thanks, Eric.

Very disappointing… :frowning: But I appreciate you chiming in to clear things up!

RESOLVED:

I’m just following up to explain how I got around the issue. It just so happens that in my particular case, I can get away with values that fall on a grid with the smallest value being 1/128 which is 0.0078125. So I’ve added a couple lines in the shader to snap the float values to a multiple of 0.0078125.

float remainder = fmod ( i.color.r, 0.0078125);
i.color.r = (i.color.r - remainder);

I do this for each of the four color channels. The cost of fmod is kind of obnoxious since it takes a few processor cycles, but it’s worth it for the gains of storing data in the RGB channel in my case.