Vertex color changing script leaving artifacts of replaced color.

I’ve working on a character customization script, where you can change the vertex colors of an object. I’ve tried a number of ways, but every time it leaves artifacts, except for one way, which only allows you to replace every instance of a specific color in a mesh. This doesn’t allow me to isolate specific parts of a mesh, as after updating the colors in the mesh, each those parts of the mesh are now grouped with other parts that match the same color.

The following is a couple of the ways I’ve tried, with the TryAgain method working, but failing me in the ways described above.

To start, I’ve created a cube, subdivided, where each of the six sides of the cube is a different color consisting of multiple actual faces, and then applied a vertex color shader.

When I replace the color, the edges of the cube near the next side of the cube are the spots where the artifacts from the previous color remain, leaving a splotchy mess. It was my understanding that to preserve hard edges, Unity splits the faces of the mesh, and so instead of a cube having 8 vertices, when imported from Blender it reads as 24, which lead me to believe that it should work, but after the result I had, I even split the faces manually and imported the cube and was left with the same result.

After numerous attempts, I even looped through every vertex and separated the indices into different lists based on color, and still, artifacts. This is the failed method that lies below.

The goal is to figure out what I’m doing wrong in the ChangeColor method so that it will completely fill the face as it does in the TryAgain method, while still allowing me to divide and change the colors of the mesh by vertex, rather than just looping through and replacing all instances of a specific color.

A assigns the mesh, B uses the method that leaves artifacts, Space will restore the original colors of the mesh, and C uses the method that doesn’t leave artifacts, however doesn’t allow me to isolate the color to change by group of vertices/face. Keys 1-6 will change which face is adjusted. Everything is public for simplicity sake the only thing you need to do is attach the script and drag a GameObject with a mesh renderer into the activeMeshObject in the inspector. As previously stated I used a a sub divided cube with each side consisting of subdivided faces of the same color, but anything with a vertex color shader and faces that are flood filled with a single color should demonstrate the problem, provided there’s only six different colors.

There’s definitely a chance I’m just making some stupid mistake, any help is appreciated.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Linq;


public class ColorChangeDebug : MonoBehaviour
{

    public GameObject activeMeshObject;
    public Mesh activeMesh;

    public Color fillColor;

    public Color[] originalColors;

    public bool colorsChanged;

    public List<int> distinctVertColorIndexList1;
    public List<int> distinctVertColorIndexList2;
    public List<int> distinctVertColorIndexList3;
    public List<int> distinctVertColorIndexList4;
    public List<int> distinctVertColorIndexList5;
    public List<int> distinctVertColorIndexList6;
 
    public List<int> vertIndexList;

    public int numberOfIndexLists;

    public Color colorToReplace;

    public List<Color> distinctMeshColors;
    public List<Color> meshColors;
    public List<Vector3> meshVerts;

    public int currentIndexList;
    public int cubeSide;

    void Start()
    {

        fillColor = Color.black;
   

    }

    void Update()
    {

        if (Input.GetKeyUp(KeyCode.A)) { AssignMesh(); }


        if (Input.GetKeyUp(KeyCode.Space) && colorsChanged == true)  //Restores original Colors
        {

            activeMesh.colors = originalColors;
            colorsChanged = false;

        }

        else if (Input.GetKeyUp(KeyCode.B) && colorsChanged == false) //Leaves artifacts
        {

            ChangeColor();
            colorsChanged = true;

        }

        else if (Input.GetKeyUp(KeyCode.C) && colorsChanged == false) //Gets proper hard edges and leaves no artifacts, but won't allow me to target specific vertices of a mesh.
        {

            TryAgain();
            colorsChanged = true;

        }

        if (Input.GetKeyUp(KeyCode.Keypad1)) { currentIndexList = 1; UpdateIndexList(); }               //Changes side of cube that will change color.
        else if (Input.GetKeyUp(KeyCode.Keypad2)) { currentIndexList = 2; UpdateIndexList(); }
        else if (Input.GetKeyUp(KeyCode.Keypad3)) { currentIndexList = 3; UpdateIndexList(); }
        else if (Input.GetKeyUp(KeyCode.Keypad4)) { currentIndexList = 4; UpdateIndexList(); }
        else if (Input.GetKeyUp(KeyCode.Keypad5)) { currentIndexList = 5; UpdateIndexList(); }
        else if (Input.GetKeyUp(KeyCode.Keypad6)) { currentIndexList = 6; UpdateIndexList(); }

    }


    void AssignMesh()
    {

        activeMesh = activeMeshObject.GetComponent<MeshFilter>().sharedMesh;
        Mesh clonedMesh = (Mesh)Instantiate(activeMesh);
        activeMeshObject.GetComponent<MeshFilter>().sharedMesh = clonedMesh;
        activeMesh = clonedMesh;
        originalColors = activeMesh.colors;
        GetMeshColors();
        GetVertListsByColor();


    }


    void ChangeColor()
    {

        activeMesh.GetColors(meshColors);

        foreach (int index in vertIndexList)
        {

            meshColors[index] = fillColor;

        }

        activeMesh.SetColors(meshColors);

    }


    void TryAgain()
    {


        Color[] temporaryColorArray = activeMesh.colors;
        colorToReplace = distinctMeshColors[cubeSide];

        foreach (Color col in temporaryColorArray)
        {

            if (col == colorToReplace)
            {

                int index = System.Array.IndexOf(temporaryColorArray, col);
                Color temporaryCol = fillColor;
                temporaryColorArray[index] = temporaryCol;

            }

        }

        activeMesh.colors = temporaryColorArray;

    }


    void UpdateIndexList()
    {
        switch (currentIndexList)
        {
            case 1:
                vertIndexList = distinctVertColorIndexList1;
                cubeSide = 0;
                break;
            case 2:
                vertIndexList = distinctVertColorIndexList2;
                cubeSide = 1;
                break;
            case 3:
                vertIndexList = distinctVertColorIndexList3;
                cubeSide = 2;
                break;
            case 4:
                vertIndexList = distinctVertColorIndexList4;
                cubeSide = 3;
                break;
            case 5:
                vertIndexList = distinctVertColorIndexList5;
                cubeSide = 4;
                break;
            case 6:
                vertIndexList = distinctVertColorIndexList6;
                cubeSide = 5;
                break;
        }
    }


    void GetVertListsByColor()
    {
        currentIndexList = 1;
        UpdateIndexList();

        foreach (Color distinctCol in distinctMeshColors)
        {

            foreach (Vector3 vert in meshVerts)
            {
                int vertIndex = meshVerts.IndexOf(vert);
            
                if (meshColors[vertIndex] == distinctCol)
                {

                    vertIndexList.Add(vertIndex);

                }

            }

            currentIndexList++;
            UpdateIndexList();

        }
    }


    void GetMeshColors()
    {
        if (activeMesh != null)
        {
            activeMesh.GetColors(meshColors);
            activeMesh.GetVertices(meshVerts);
            distinctMeshColors = meshColors.Distinct().ToList();

        }
    }

}

I think some screenshots of the cube would help clarify what exactly is meant by “artifacts”. You might be seeing some sort of bleed from the shader, or some lighting effect, etc, or it could be an actual problem with this logic itself - screenshots will help a lot.

It’s not lighting or the shader, as mesh.colors when sent into a list shows that some of the verts are just unaffected by the method. here’s a screen shot demonstrating what I mean by artifacts caused by the ChangeColor method. The shown sides of the cube were green, red, and blue, and the method should flood fill the green side of the cube.

The GetVertListsByColor method should theoretically fill a different list of indices for every distinct color in the mesh, but when I loop through and change them, some of them simply don’t change.

GetVertListsByColor may be bugged, not giving you all of the vertices. Try logging the .Count of each of those vertex lists; do they all contain the expected number of vertices?

One potential problem I see is with ==. Color is a set of 3 floating point values and you should almost never expect == to work on floats reliably. Floating point imprecision is a common cause of rookie mistakes; basically, sometimes floating point math can result in something like 2.0f * 3.5f resulting in something like 6.9999998f instead of 7.0f, and this is just a fact of life for floating point math on CPUs and you have to expect this and work around it. You’re using == on like 189 which might be triggering this glitch. (I’m not sure why it would trigger there but not on line 129, but it’s possible than the floats lose some “resolution” after the game has started, which might make them all actually be == the same value)

You can account for this by checking the R, G, and B values to see if they’re within some range (like 0.01f) rather than using a direct == between the colors.

In this scenario you might find it beneficial to use the vertex’s normal to build your vertex lists rather than colors. With normals you could use Vector3.angle to compare each vertex’s normal to the expected vector (such as Vector3.up); if it’s <90 then that’s the side you’re on. This might be more reliable than using pre-existing colors as each vertex would be essentially guaranteed to fall into ONE of the slots; you wouldn’t have any gaps. (Disregard this advice if you think you might be using something besides a simple cube one day)

One other curiosity. You say:

Does this cube just have 8 (or just 24) vertices total? If so, then there is very likely something non-vertex-related going wrong somehow. Looking at your screenshot, if this is an issue of vertices being “left behind”, then there are a LOT more vertices than implies by the quoted line; eyeballing I’d say it looks like 64 vertices per face? If there are only 4 vertices per face it just wouldn’t be possible for the shapes shown in the screenshot to be caused by vertex colors. I’m thinking a shadow map bug or something along those lines.

1 Like

the cube is sub divided, so that there’s 36 faces per side, though once it gets into unity that doubles to 72 due to triangulation, I was just making the point that in order to maintain a flood fill with hard edges, even if the cube was modeled so that each side was a single face, from what I’ve heard there should be no shared vertices that would be responsible for causing the issue.

i can’t use the normals like that, because i’m not actually going to just use this on a cube, the goal is to make it apply to a skinned mesh renderer, but I was having the same problem, so I simplified it for the sake of making it easier to demonstrate the problem on a cube.

i’ll give your suggestion about floating point values a shot though and report back by tomorrow. thank you for taking a look, i know 200 lines of code is kind of a hassle to wade through.

BTW, you could reduce that a lot if you make your vertex list a List<List> instead - use currentIndexList as the first index (except make it 0-5 instead of 1-6). You could get rid of all of the swapping in and out of vertIndexList, using distinctVertColorIndexLists[currentIndexList] in its place. e.g.:

 // equivalent to line 107
foreach (int index in distinctVertColorIndexLists[currentIndexList]) {

I didn’t want to overcomplicate the fix by bringing this up earlier but it’ll be a nice improvement and will make your code more easily maintained.

I converted everything to Color32 to do away with floats completely, and still got the same result.

When I look in the inspector, the index counts should all be 49, but they’re all different numbers that add up to the correct total, which doesn’t make any sense to me. I’m thinking it may be a graphics issue as I don’t understand why the GetVertListsByColor method would randomly put different verts in the wrong list, as I’m checking for the color of the verts before hand.

A work around I’ve thought about is just using the method that works, and creating multiple slight variations on the lists of colors, so that even if the colors appear almost the same visually, they never match up, and therefore are never grouped together, which will allow me to maintain changing different parts of the mesh. this leads to annoyances when designing the vert colors of the mesh to make sure to never use any color in the 6 palettes i’ll have to create, but it should work.

Why 49? A cube has 8 corners. If the faces share the vertices you should have 8 vertices. You could also give each face its own vertex to avoid interpolation artifacts (afaik called “faceted look”). Then you should have 32 vertices. Or do I understand you wrong?

Each face has 7x7 vertices.

As Ray pointed out the cube is subdivided. I originally just used the 24 vert cube, but wasn’t quite understanding what was happening with the splotchy vertex colors, so I subdivided it, which confirmed that the artifacts only appear on the vertices that were adjacent to the next side of the cube.

I should probably make a post in graphics and ask why Unity is grouping my vertices wrong, but I thought there was a chance that I was just making a coding mistake. So far one has yet to be pointed out to me, aside from a possibility of floating point errors, but after replacing my color value with a Color32, which doesn’t deal with floats, the problem still remains.

Edit: Also, a non sub divided cube imported into Unity, at least from Blender will always have 24 vertices no matter what, there are no shared vertices.