Duplicating a Mesh including blendshapes to send via network

Hello all,

I’m stuck trying to save all information of a mesh (imported from an fbx) to a serializable format so that I can send that information via network. Don’t worry, I only have to do this once, so file size is not an issue for now. Copying vertices, normals, tangents, and so on is not an issue and works just as expected.

However, when I want to copy the blendshapes, this happens:

How the original looks with blendshape “jawopen” modified:
alt text

How the copy looks:
alt text

As you can see, while something is modified when altering the blendshape weights, some information is lost here.

Does anyone know what the issue might be here?

For testing, I made a simple script that copies all information from the current mesh to a new one.

public class MeshHolder
{
    public string name;
    public Vector3[] vertices;
    public Vector3[] normals;
    public Vector4[] tangents;
    public Vector2[] uv;
    public int[] triangles;
    public BlendShapeHolder[] BlendShapeHolders;
    public Matrix4x4[] bindposes;
    public BoneWeight[] boneWeights;
    public Bounds bounds;
    public int blendshapeCount;
}

public class BlendShapeHolder
{
    public string name;
    public float weight;
    public int frameCount;
    public int vertexCount;
    public Vector3[][] dVertices;
    public Vector3[][] dNormals;
    public Vector3[][] dTangents;
}

public void SaveMesh(){

Mesh myMesh = CurrentMesh.sharedMesh;

            MeshHolder.normals = myMesh.normals;
            MeshHolder.vertices = myMesh.vertices;
            MeshHolder.uv = myMesh.uv;
            MeshHolder.tangents = myMesh.tangents;
            MeshHolder.triangles = myMesh.triangles;
            MeshHolder.bindposes = myMesh.bindposes;
            MeshHolder.boneWeights = myMesh.boneWeights;
            MeshHolder.bounds = myMesh.bounds;
            MeshHolder.blendshapeCount = myMesh.blendShapeCount;

            Vector3[] dVertices = new Vector3[myMesh.vertexCount];
            Vector3[] dNormals = new Vector3[myMesh.vertexCount];
            Vector3[] dTangents = new Vector3[myMesh.vertexCount];
            MeshHolder.BlendShapeHolders = new BlendShapeHolder[myMesh.blendShapeCount];
            for (int shape = 0; shape < myMesh.blendShapeCount; shape++)
            {
                BlendShapeHolder holder = new BlendShapeHolder();
                holder.dVertices = new Vector3[myMesh.blendShapeCount][];
                holder.dTangents = new Vector3[myMesh.blendShapeCount][];
                holder.dNormals = new Vector3[myMesh.blendShapeCount][];
                holder.frameCount = myMesh.GetBlendShapeFrameCount(shape);
                for (int frame = 0; frame < holder.frameCount; frame++)
                {
                   holder.name = myMesh.GetBlendShapeName(shape);
                    float frameWeight = myMesh.GetBlendShapeFrameWeight(shape, frame);
                    holder.weight = frameWeight;
                    holder.dVertices[frame] = new Vector3[myMesh.vertices.Length];
                    holder.dTangents[frame] = new Vector3[myMesh.vertices.Length];
                    holder.dNormals[frame] = new Vector3[myMesh.vertices.Length];
                    myMesh.GetBlendShapeFrameVertices(shape, frame, dVertices, dNormals, dTangents);
                    for (int k = 0; k < holder.dVertices[frame].Length; k++)
                    {
                        holder.dVertices[frame][k] = dVertices[k];
                        holder.dNormals[frame][k] = dNormals[k];
                        holder.dTangents[frame][k] = dTangents[k];
                    }
                }

                MeshHolder.BlendShapeHolders[shape] = holder;
            }
}

… and transfers it to the new one.

public void LoadMesh(){
newMesh = new Mesh();
            newMesh.vertices = MeshHolder.vertices;
            newMesh.normals = MeshHolder.normals;
            newMesh.uv = MeshHolder.uv;
            newMesh.tangents = MeshHolder.tangents;
            newMesh.triangles = MeshHolder.triangles;
            newMesh.bindposes = MeshHolder.bindposes;
            newMesh.boneWeights = MeshHolder.boneWeights;
            newMesh.bounds = MeshHolder.bounds;
            for (int shape = 0; shape < MeshHolder.blendshapeCount; shape++)
            {
                int frameCount = MeshHolder.BlendShapeHolders[shape].frameCount;
                for (int frame = 0; frame < frameCount; frame++)
                {
                    BlendShapeHolder holder = MeshHolder.BlendShapeHolders[shape];
                    string shapeName = holder.name;
                    float frameWeight = holder.weight;
                    Vector3[] dVertices = holder.dVertices[frame];
                    Vector3[] dNormals = holder.dNormals[frame];
                    Vector3[] dTangents = holder.dTangents[frame];
                    newMesh.AddBlendShapeFrame(shapeName, frameWeight, dVertices, dNormals, dTangents);
                }
}

I’m linking you @Bunny83 , as you’ve posted the mesh serializer that I’ve tried. Using it, this above seen error occurs.

I’m a bit confused what your question is about. Did you use my MeshSerializer or did you use your code? If you use your code, how do you actually send it over the network? Also what kind of network connection / transport layer do you use? Maybe you have issues with the index format? By default a new mesh uses 16bit indices so you are restricted to 64k vertices. You don’t seem to care about submeshes (maybe your mesh doesn’t have any?!) Are you sure your mesh is made of triangles? Unity now support different topologies like Quads.

Have you tried serializing and deserializing the mesh on the same machine, right away from the freshly serialized data? Just to ensure it’s not a pure transport issue?

So, I came up with a solution after all. As a reminder, my main issue was that when copying data from a mesh object to a new mesh object, I seem to lose some information that affects the animation of the blendshapes as seen in the above images.

The solution: Do not use a new mesh at all but use a prefab of a working mesh, instantiate it, whenever you need to create a new object and then switch out all the information from the instantianted mesh with the copied mesh.

As an example: I saved one avatar head with working blendshapes as a prefab. Whenever I send mesh data from another avatar head through the network, I do not use this data to create a new mesh, but I instantiate the prefab and override the prefab’s mesh data with the data sent through the network :wink: