Runtime procedural mesh generation with DOTS

I’ve tried looking up info on this, but most guides are from 2020-2022 and there’s like one from 2023. Given DOTS goes through breaking changes faster than I can count, it’s difficult to reference them. Most of them also use partial DOTS and have direct access to managed data at some point.

My use case is relatively simple as far as procedural meshes go. I’m making a submarine editor and I’d like to use hollow shapes for the walls so it’s easy for players to make curves. Issue is, when you scale a hollow shape the thickness is going to change. The formula for fixing that is easy, but applying it isn’t.

I’m a newcomer to DOTS, in that this is my first system ever, so this is a pain in the rear. I’m using an ISystem with an EntityQuery to run a job to handle the mesh generation. So far, I’ve got it to only trigger on the first load and any time you subsequently change the scale of the LocalToWorld matrix. Good start.

Now for the meat of the question. How can I modify a mesh with a job? I’m guessing it has something to do with the MeshData API, but finding info on how to use it with ECS is like looking into a dark forest from above with a helicopter on a foggy night. No luck. So I’m here to ask from anyone who knows.

Here’s my system code:

using Unity.Entities;
using Unity.Mathematics;
using Unity.Collections;
using Unity.Burst;
using Unity.Transforms;
using Unity.Rendering;
using UnityEngine;

namespace BlueAbyss
{
    [RequireMatchingQueriesForUpdate]
    partial struct HollowShapeSystem : ISystem
    {
        EntityQuery query;

        public void OnCreate(ref SystemState state)
        {
            query = new EntityQueryBuilder(Allocator.Temp)
                .WithAllRW<HollowShape>()
                .WithAll<LocalToWorld>()
                .Build(ref state);
            query.SetChangedVersionFilter(typeof(LocalToWorld)); // We only care if the entity changes scale.
        }

        [BurstCompile]
        public void OnUpdate(ref SystemState state)
        {
            HollowShapeJob hollowShapeJob = new();
            hollowShapeJob.ScheduleParallel(query);
        }
    }

    [BurstCompile]
    public partial struct HollowShapeJob : IJobEntity
    {
        public void Execute(ref HollowShape hollowShape, in LocalToWorld localToWorld)
        {
            float3 currentScale = localToWorld.Value.Scale();
            if (currentScale.Equals(hollowShape.LastScale)) return;
            hollowShape.LastScale = currentScale;

            // Past this point, mesh generation should occur.
        }
    }
}

And here’s my component code:

using Unity.Burst;
using Unity.Entities;
using Unity.Mathematics;
using UnityEngine;

namespace BlueAbyss
{
    public class HollowShapeAuthoring : MonoBehaviour
    {
        public float Thickness = 0.2f;
        public AxisFlags AxisFlags = (AxisFlags)~0;

        public class Baker : Baker<HollowShapeAuthoring>
        {
            public override void Bake(HollowShapeAuthoring authoring)
            {
                var entity = GetEntity(TransformUsageFlags.Renderable);

                AddComponent(entity, new HollowShape
                {
                    Thickness = authoring.Thickness,
                    AxisFlags = authoring.AxisFlags,
                });
            }
        }
    }

    public struct HollowShape : IComponentData
    {
        public float Thickness;
        public AxisFlags AxisFlags;
        public float3 LastScale;
    }

    [System.Flags]
    public enum AxisFlags
    {
        X = 1 << 0,
        Y = 1 << 1,
        Z = 1 << 2,
    }
}

You have to allocate a MeshDataArray with the number of meshes you want to modify. Then in a job, you write the MeshData. And then after the job, you apply the MeshDataArray to the List of meshes you want to replace their data for.

You also want to make sure you have unique meshes for each entity that you register with EntitiesGraphicsSystem and assign to the entity MaterialMeshInfo component.

This isn’t a great example because it is tightly tied to some other high-performance mechanisms I’ve built, but I’ve been working on a new feature for my framework that converts dynamic buffer mesh data into actual meshes that are automatically assigned to the entities, complete with Mesh object pooling behind-the-scenes. I’ll share it anyways just in case it helps somehow. Latios-Framework/Kinemation/Systems/Culling/UploadUniqueMeshesSystem.cs at prerelease/0.12.0 · Dreaming381/Latios-Framework · GitHub

Hmm, I think I’m going to need an unique mesh for every entity with a HollowShape component. Generally, having two hollow shapes with exactly the same scale is not something I’m going to optimize for. It’s cumbersome to handle with very little benefit.

I know how to copy a mesh and I can do that in a baker since it’s managed, but how do I then assign that mesh to the entity? Normally, that’s handled by a MeshFilter, but I can’t be sure whether the MeshFilter baking is before or after my own and bakers can’t have inter-dependencies. I have no clue how to manually set up an entity for rendering in a baker either, specifically because I’d just be creating a RenderMeshArray for every single entity since a baker only knows about one entity at a time and I don’t know how to share a RenderMeshArray across entities like that.

I saw something about baker systems being able to run before bakers do, so I could swap out the mesh in a mesh filter and have it handled for me, but I have no clue how to use baker systems like that.

Once that’s done I still have to handle the whole mesh modification part, but at least I won’t have to actually create or swap any meshes so that part should be easier. I hope.

Don’t try to bake the unique meshes. Just create them at runtime.

Well, I did at least manage to get the number of meshes I need. Not sure if this is the right way, though, and it feels a tad off to be entirely honest. For beginners, I’m losing access to the entity I’m actually trying to create the mesh for. Which makes this unusable, but at least I got the count out of the job I think.

using Unity.Entities;
using Unity.Mathematics;
using Unity.Collections;
using Unity.Burst;
using Unity.Transforms;
using Unity.Rendering;
using Unity.Jobs;
using UnityEngine;

namespace BlueAbyss
{
    [RequireMatchingQueriesForUpdate]
    partial struct HollowShapeSystem : ISystem
    {
        EntityQuery query;

        public void OnCreate(ref SystemState state)
        {
            query = new EntityQueryBuilder(Allocator.Temp)
                .WithAllRW<HollowShape>()
                .WithAll<LocalToWorld, MaterialMeshInfo>()
                .Build(ref state);
            query.SetChangedVersionFilter(typeof(LocalToWorld)); // We only care if the entity changes scale.
        }

        [BurstCompile]
        public void OnUpdate(ref SystemState state)
        {
            NativeList<MaterialMeshInfo> materialMeshInfo = new(query.CalculateEntityCount(), Allocator.TempJob);

            InfoJob infoJob = new()
            {
                MaterialMeshInfo = materialMeshInfo.AsParallelWriter(),
            };
            state.Dependency = infoJob.ScheduleParallel(query, state.Dependency);
            state.CompleteDependency();

            var meshDataArray = Mesh.AllocateWritableMeshData(materialMeshInfo.Length);
        }

        [BurstCompile]
        public partial struct InfoJob : IJobEntity
        {
            public NativeList<MaterialMeshInfo>.ParallelWriter MaterialMeshInfo;

            public void Execute(HollowShape hollowShape, in LocalToWorld localToWorld, in MaterialMeshInfo materialMeshInfo)
            {
                float3 currentScale = localToWorld.Value.Scale();
                if (currentScale.Equals(hollowShape.LastScale)) return;
                hollowShape.LastScale = currentScale;

                MaterialMeshInfo.AddNoResize(materialMeshInfo);
            }
        }
    }
}

Do you need an actual different meshes?

Why not leverage power an advantage of DOTS, by using same mesh and use instead a shader, to actually change the size and thikness of the walls?

I wish I could, but I need collisions. This is entirely possible in a shader and I actually did prototype it using Shader Graph. Sadly, it’s more of a cool looking toy than anything functional.

That said, I’m thinking of different ways to approach this for reasons I won’t get into right now given I’m about to head to bed.

Colliders modification at runtime is more complex, than just mesh modification.
Specially if is none uniform scaling.

If is just multiplier scale, is not an issue.
But modifying vertices, you need build new colliers and push to blob asset store, for physics to handle it.

Solved it. Still missing AxisFlags usage and whatnot, but it’s enough to count for a solution.

Working component code:

using Unity.Burst;
using Unity.Entities;
using Unity.Mathematics;
using Unity.Rendering;
using UnityEngine;
using UnityEngine.Rendering;

namespace BlueAbyss
{
    public class HollowShapeAuthoring : MonoBehaviour
    {
        public float Thickness = 0.2f;
        public AxisFlags AxisFlags = (AxisFlags)~0;

        public class Baker : Baker<HollowShapeAuthoring>
        {
            public override void Bake(HollowShapeAuthoring authoring)
            {
                var meshFilter = GetComponent<MeshFilter>();

                if (!meshFilter) return;

                var entity = GetEntity(TransformUsageFlags.Renderable);

                var extents = meshFilter.sharedMesh.bounds.extents;
                var highest = Mathf.Max(extents.x, extents.y, extents.z);

                AddComponent(entity, new HollowShape
                {
                    Thickness = authoring.Thickness,
                    ShapeExtents = highest,
                    AxisFlags = authoring.AxisFlags,
                });

                AddComponentObject(entity, new HollowShapeMesh
                {
                    Mesh = meshFilter.sharedMesh,
                });
            }
        }
    }

    public struct HollowShape : IComponentData
    {
        public float3 Thickness;
        public float3 ShapeExtents;
        public AxisFlags AxisFlags;
        public float3 LastScale;
        public BatchMeshID BatchMeshID;
    }

    public class HollowShapeMesh : IComponentData
    {
        public Mesh Mesh;
    }

    [System.Flags]
    public enum AxisFlags
    {
        X = 1 << 0,
        Y = 1 << 1,
        Z = 1 << 2,
    }
}

Working system code:

using Unity.Entities;
using Unity.Mathematics;
using Unity.Collections;
using Unity.Burst;
using Unity.Transforms;
using Unity.Rendering;
using Unity.Jobs;
using UnityEngine;
using UnityEngine.Rendering;

namespace BlueAbyss
{
    [RequireMatchingQueriesForUpdate]
    partial struct HollowShapeSystem : ISystem
    {
        EntityQuery query;

        EntityTypeHandle entityType;

        ComponentLookup<HollowShape> hollowShapeLookup;
        ComponentLookup<MaterialMeshInfo> materialMeshInfoLookup;

        ComponentTypeHandle<HollowShape> hollowShapeType;
        ComponentTypeHandle<LocalToWorld> localToWorldType;

        public void OnCreate(ref SystemState state)
        {
            query = new EntityQueryBuilder(Allocator.Temp)
                .WithAllRW<HollowShape>()
                .WithAll<LocalToWorld>()
                .Build(ref state);
            query.SetChangedVersionFilter(typeof(LocalToWorld)); // We only care if the entity changes scale.

            entityType = state.GetEntityTypeHandle();

            hollowShapeLookup = state.GetComponentLookup<HollowShape>();
            materialMeshInfoLookup = state.GetComponentLookup<MaterialMeshInfo>();

            hollowShapeType = state.GetComponentTypeHandle<HollowShape>();
            localToWorldType = state.GetComponentTypeHandle<LocalToWorld>(true);
        }

        public void OnUpdate(ref SystemState state)
        {
            var chunks = query.ToArchetypeChunkArray(Allocator.TempJob);

            entityType.Update(ref state);
            hollowShapeType.Update(ref state);
            localToWorldType.Update(ref state);

            NativeList<Entity> validEntities = new(query.CalculateEntityCount(), Allocator.TempJob);

            EntityValidationJob entityValidationJob = new()
            {
                Chunks = chunks,
                ValidEntities = validEntities.AsParallelWriter(),
                EntityType = entityType,
                HollowShapeType = hollowShapeType,
                LocalToWorldType = localToWorldType,
            };

            entityValidationJob.Run(chunks.Length);

            if (validEntities.IsEmpty)
            {
                validEntities.Dispose();
                return;
            }

            hollowShapeLookup.Update(ref state);
            materialMeshInfoLookup.Update(ref state);

            NativeArray<Entity> entities = validEntities.ToArray(Allocator.TempJob);
            validEntities.Dispose();

            var meshes = new Mesh[entities.Length];

            for (int i = 0; i < entities.Length; i++)
            {
                var entity = entities[i];
                var hollowShapeMesh = state.EntityManager.GetComponentObject<HollowShapeMesh>(entity);
                meshes[i] = hollowShapeMesh.Mesh;
            }

            var sourceMeshDataArray = Mesh.AcquireReadOnlyMeshData(meshes);
            var targetMeshDataArray = Mesh.AllocateWritableMeshData(entities.Length);

            NativeArray<VertexAttributeDescriptor> vertexLayout = new(4, Allocator.TempJob);

            vertexLayout[0] = new(VertexAttribute.Position, VertexAttributeFormat.Float32);
            vertexLayout[1] = new(VertexAttribute.Normal, VertexAttributeFormat.Float32);
            vertexLayout[2] = new(VertexAttribute.Tangent, VertexAttributeFormat.Float32, 4);
            vertexLayout[3] = new(VertexAttribute.Color, VertexAttributeFormat.UNorm8, 4);

            HollowShapeJob hollowShapeJob = new()
            {
                Entities = entities,
                HollowShapeLookup = hollowShapeLookup,
                SourceMeshDataArray = sourceMeshDataArray,
                TargetMeshDataArray = targetMeshDataArray,
                VertexLayout = vertexLayout,
            };

            state.Dependency = hollowShapeJob.Schedule(entities.Length, 1, state.Dependency);
            state.Dependency.Complete();

            sourceMeshDataArray.Dispose();

            for (int i = 0; i < meshes.Length; i++)
            {
                meshes[i] = new();
            }

            Mesh.ApplyAndDisposeWritableMeshData(targetMeshDataArray, meshes, MeshUpdateFlags.DontRecalculateBounds);

            var entitiesGraphicsSystem = state.World.GetExistingSystemManaged<EntitiesGraphicsSystem>();

            NativeArray<BatchMeshID> meshIDs = new(meshes.Length, Allocator.TempJob);

            for (int i = 0; i < meshes.Length; i++)
            {
                meshIDs[i] = entitiesGraphicsSystem.RegisterMesh(meshes[i]);
            }

            MeshApplicationJob meshApplicationJob = new()
            {
                Entities = entities,
                MeshIDs = meshIDs,
                HollowShapeLookup = hollowShapeLookup,
                MaterialMeshInfoLookup = materialMeshInfoLookup,
            };

            meshApplicationJob.Run(meshIDs.Length);

            foreach (var meshID in meshIDs) // The mesh application job swaps in the old IDs.
            {
                Object.Destroy(entitiesGraphicsSystem.GetMesh(meshID));
                entitiesGraphicsSystem.UnregisterMesh(meshID);
            }

            meshIDs.Dispose();
        }

        [BurstCompile]
        struct EntityValidationJob : IJobParallelFor
        {
            [ReadOnly, DeallocateOnJobCompletion]
            public NativeArray<ArchetypeChunk> Chunks;

            public NativeList<Entity>.ParallelWriter ValidEntities;

            public EntityTypeHandle EntityType;

            public ComponentTypeHandle<HollowShape> HollowShapeType;

            [ReadOnly]
            public ComponentTypeHandle<LocalToWorld> LocalToWorldType;

            public void Execute(int index)
            {
                var chunk = Chunks[index];
                var chunkEntities = chunk.GetNativeArray(EntityType);
                var chunkHollowShape = chunk.GetNativeArray(ref HollowShapeType);
                var chunkLocalToWorld = chunk.GetNativeArray(ref LocalToWorldType);
                var chunkCount = chunk.Count;

                for (int i = 0; i < chunkCount; i++)
                {
                    var hollowShape = chunkHollowShape[i];
                    var localToWorld = chunkLocalToWorld[i];

                    var currentScale = localToWorld.Value.Scale();
                    if (currentScale.Equals(hollowShape.LastScale)) continue;
                    hollowShape.LastScale = currentScale;

                    ValidEntities.AddNoResize(chunkEntities[i]);

                    chunkHollowShape[i] = hollowShape;
                }
            }
        }

        struct MeshGatheringJob : IJob
        {
            [ReadOnly]
            public NativeArray<Entity> Entities;

            public EntityManager EntityManager;

            public Mesh.MeshDataArray OutputMeshDataArray;

            public void Execute()
            {
                var meshes = new Mesh[Entities.Length];

                for (int i = 0; i < Entities.Length; i++)
                {
                    var entity = Entities[i];
                    var hollowShapeMesh = EntityManager.GetComponentObject<HollowShapeMesh>(entity);
                    meshes[i] = hollowShapeMesh.Mesh;
                }

                OutputMeshDataArray = Mesh.AcquireReadOnlyMeshData(meshes);
            }
        }

        struct Vertex
        {
            public float3 Position;
            public float3 Normal;
            public float4 Tangent;
            public Color32 Color;
        }

        [BurstCompile]
        struct HollowShapeJob : IJobParallelFor
        {
            [ReadOnly]
            public NativeArray<Entity> Entities;

            [ReadOnly]
            public ComponentLookup<HollowShape> HollowShapeLookup;

            [ReadOnly]
            public Mesh.MeshDataArray SourceMeshDataArray;

            public Mesh.MeshDataArray TargetMeshDataArray;

            [DeallocateOnJobCompletion]
            public NativeArray<VertexAttributeDescriptor> VertexLayout;

            public void Execute(int index)
            {
                var entity = Entities[index];
                var hollowShape = HollowShapeLookup[entity];

                var sourceMeshData = SourceMeshDataArray[index];
                var targetMeshData = TargetMeshDataArray[index];

                var sourceIndices = sourceMeshData.GetIndexData<ushort>();
                var sourceVertices = sourceMeshData.GetVertexData<Vertex>();

                targetMeshData.SetIndexBufferParams(sourceIndices.Length, sourceMeshData.indexFormat);
                targetMeshData.SetVertexBufferParams(sourceVertices.Length, VertexLayout);

                var targetIndices = targetMeshData.GetIndexData<ushort>();
                var targetVertices = targetMeshData.GetVertexData<Vertex>();

                var insideColor = new Color32(255, 255, 255, 255);

                for (int i = 0; i < sourceIndices.Length; i++)
                {
                    targetIndices[i] = sourceIndices[i];
                }

                for (int i = 0; i < sourceVertices.Length; i++)
                {
                    var vertex = sourceVertices[i];
                    
                    if (IsRGBEqual(in vertex.Color, in insideColor))
                    {
                        vertex.Position = (hollowShape.ShapeExtents - hollowShape.Thickness / hollowShape.LastScale) * math.normalizesafe(vertex.Position);
                    }

                    targetVertices[i] = vertex;
                }

                targetMeshData.subMeshCount = sourceMeshData.subMeshCount;

                for (int i = 0; i < sourceMeshData.subMeshCount; i++)
                {
                    targetMeshData.SetSubMesh(i, sourceMeshData.GetSubMesh(i));
                }
            }

            bool IsRGBEqual(in Color32 a, in Color32 b)
            {
                return a.r == b.r && a.b == b.b && a.g == b.g;
            }
        }

        [BurstCompile]
        struct MeshApplicationJob : IJobParallelFor
        {
            [ReadOnly, DeallocateOnJobCompletion]
            public NativeArray<Entity> Entities;

            public NativeArray<BatchMeshID> MeshIDs;

            public ComponentLookup<HollowShape> HollowShapeLookup;
            public ComponentLookup<MaterialMeshInfo> MaterialMeshInfoLookup;

            public void Execute(int index)
            {
                var entity = Entities[index];
                var hollowShape = HollowShapeLookup[entity];
                var materialMeshInfo = MaterialMeshInfoLookup[entity];
                var meshID = MeshIDs[index];

                MeshIDs[index] = hollowShape.BatchMeshID;
                hollowShape.BatchMeshID = meshID;

                materialMeshInfo.MeshID = meshID;

                HollowShapeLookup[entity] = hollowShape;
                MaterialMeshInfoLookup[entity] = materialMeshInfo;
            }
        }
    }
}