Finding data loss in parallel burstcompiled code

[Ambrose998800]

Hi. I am on a tutorial to write a multi-core processed procedural terrain creation code using burstcompile and the Delaunay triangulation algorithm.
The terrain is created in multiple steps (heightmap → normals → triangles) and the code runs without error, but also without (final) output. Heightmap and normals are computet, stored in a list and displayed by gizmos. But the triangles output list comes out empty (count = 0). The code runs without problems, the profiler shows a lot of work is done by the triangulation script, but at the end the NativeList shows a length of 0 (before it is disposed of course). Somewhere the data is lost or the output isn’t transfered back to the list. How can I find that?
The static burst compile methods wont allow me to check list size or debug any part of the code (error: boxing a valuetype is not supported), so I can’t look till what point the output comes and where the data is going lost.

How can I check the ouput on different steps in the burstcompiled code?

Here is the tutorial and I checked my scripts and they seem the same (I wrote down the code myself, no copy paste). But somewhere is a issue I can’t find.

Generating procedural terrains with decimated meshes

[DreamingImLatios]

@Ambrose998800

(error: boxing a valuetype is not supported)

You need to show actual code and error message if you want us to help you with that.

[Ambrose998800]

Sure, but as I said, there is no error message. It works perfect, but without output. With a input to calculate a terrain with 128x128 points it prints me:
Vertex NativeList = 16’384 points calculated.
Triangles NativeList = 0 tris calculated (but takes 14 jobs to work on it for 150 ms showing in the profiler).

Here is my code. The check for the output is in the CreateTerrain() method:

Summary

using Unity.Jobs;
using UnityEngine;
using Unity.Burst;
using Unity.Profiling;
using Unity.Mathematics;
using Unity.Collections;
using System.Collections;
using System.Collections.Generic;

/// <summary>
/// Thanks to David from germany for the tutorial and providing this script on blog.rearth.dev
/// </summary>


public class TerrainGenerator01 : MonoBehaviour
{
    public bool StartGeneration;

    public GridSettings Settings;

    [WriteOnly] public NativeArray<half> Heights;

    List<VertexData> CachedPoints = new List<VertexData>();

    List<TriCache> CachedTriangles = new List<TriCache>();

    public struct TriCache
    {
        public float3 Pos1;
        public float3 Pos2;
        public float3 Pos3;
    }

    [System.Serializable]
    public struct GridSettings
    {
        public ushort Count;

        public float Distance;

        public float HeightScale;

        public float NoiseStretch;

        public ushort CoreGridSpacing;

        public float NormalReduceThreshold;
    }

    [System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Sequential)]
    public struct VertexData
    {
        public float3 Position;

        public float3 Normal;
    }

    void Start()
    {

    }

    void Update()
    {
        if (Input.GetKeyDown(KeyCode.G)) StartGeneration = true;

        if (StartGeneration)
        {
            StartGeneration = false;

            CreateTerrain();
        }
    }

    void CreateTerrain()
    {
        float StartTime = Time.realtimeSinceStartup;

        int PointCount = Settings.Count * Settings.Count;

        Heights = new NativeArray<half>(PointCount, Allocator.TempJob);

        try
        {
            var HeightHandle = new HeightSampler
            {
                SamplerSettings = Settings,

                SamplerHeights = Heights
            };

            JobHandle HeightJob = HeightHandle.Schedule(PointCount, Settings.Count);



            NativeList<VertexData> Vertices = new NativeList<VertexData>(PointCount, Allocator.TempJob);

            var PointToVertexRefs = new NativeParallelHashMap<int2, int>(PointCount, Allocator.TempJob);

            var NormalsHandle = new NormalPassJob
            {
                NormalSettings = Settings,

                NormalHeights = Heights,

                Vertices = Vertices.AsParallelWriter(),

                PointToVertexReference = PointToVertexRefs.AsParallelWriter()

            }.Schedule(PointCount, Settings.Count, HeightJob);

            NormalsHandle.Complete();

            print("Verts: " + Vertices.Length);



            int TriangleCountHalf = (Settings.Count - 1) * (Settings.Count - 1);

            NativeList<int3> Triangles = new NativeList<int3>(TriangleCountHalf * 2, Allocator.TempJob);

            int PatchCountPerLine = (Settings.Count - 1) / Settings.CoreGridSpacing;

            int PatchCount = PatchCountPerLine * PatchCountPerLine;

            var TriangulationHandle = new PatchTriangulationJob
            {
                TriSettings = Settings,

                VertexRef = PointToVertexRefs,

                Triangles = Triangles.AsParallelWriter(),

                EdgeComputeMarker = new ProfilerMarker("EdgeComputeMarker"),

                InvalidSearchMarker = new ProfilerMarker("InvalidSearchMarker"),

                VertexGatherMarker = new ProfilerMarker("VertexGatherMarker"),

                TriangulationMarker = new ProfilerMarker("TriangulationMarker"),

                HoleTriangulationMarker = new ProfilerMarker("HoleTriangulationMarker")

            }.Schedule(PatchCount, 1, NormalsHandle);

            TriangulationHandle.Complete();

            print("Tris: " + Triangles.Length);



            StorePoints(in Vertices);

            StoreTriangles(in Triangles);

            PointToVertexRefs.Dispose();

            Triangles.Dispose();

            Vertices.Dispose();
        }
        finally
        {
            print(((Time.realtimeSinceStartup - StartTime) * 1000f).ToString("F1") + "ms for " + Heights.Length + " points");

            Heights.Dispose();
        }
    }

    void StorePoints(in NativeList<VertexData> Vertices)
    {
        CachedPoints.Clear();

        foreach (VertexData Data in Vertices)
        {
            CachedPoints.Add(Data);
        }
    }

    void StoreTriangles(in NativeList<int3> Triangles)
    {
        CachedTriangles.Clear();

        foreach (int3 Triangle in Triangles)
        {
            //TriCache TriPos = new TriCache
            //{
            //    Pos1 = Vertices[Triangle.x].Position,
            //    Pos2 = Vertices[Triangle.y].Position,
            //    Pos3 = Vertices[Triangle.z].Position,
            //};

            //CachedTriangles.Add(TriPos);
        }
    }

    //Multi-Thread sampler
    [BurstCompile]
    public struct HeightSampler : IJobParallelFor
    {
        public GridSettings SamplerSettings;

        [WriteOnly] public NativeArray<half> SamplerHeights;

        public void Execute(int Index)
        {
            int2 Coord = LinearArrayHelper.ReverseLinearIndex(Index, SamplerSettings.Count);

            int X = Coord.x;

            int Y = Coord.y;

            float2 LocalPos = new float2(X, Y);

            float HeightSample = noise.snoise(LocalPos / SamplerSettings.NoiseStretch);

            SamplerHeights[Index] = (half)HeightSample;
        }
    }

    [BurstCompile]
    public struct NormalPassJob : IJobParallelFor
    {
        public GridSettings NormalSettings;

        [ReadOnly] public NativeArray<half> NormalHeights;

        [WriteOnly] public NativeList<VertexData>.ParallelWriter Vertices;

        [WriteOnly] public NativeParallelHashMap<int2, int>.ParallelWriter PointToVertexReference;

        public void Execute(int Index)
        {
            int2 RevIndex = LinearArrayHelper.ReverseLinearIndex(Index, NormalSettings.Count);

            int X = RevIndex.x;
            int Y = RevIndex.y;

            bool Border = X == 0 || X == NormalSettings.Count || Y == 0 || Y == NormalSettings.Count;

            half OwnHeight = NormalHeights[Index];
            float3 CenterPos = new float3(0f, OwnHeight * NormalSettings.HeightScale, 0f);

            int SampleAIndex = LinearArrayHelper.GetLinearIndexSafe(X - 1, Y, NormalSettings.Count);
            float SampleA = NormalHeights[SampleAIndex] * NormalSettings.HeightScale;
            float3 PosA = new float3(-NormalSettings.Distance, SampleA, 0f);

            int SampleBIndex = LinearArrayHelper.GetLinearIndexSafe(X + 1, Y, NormalSettings.Count);
            float SampleB = NormalHeights[SampleBIndex] * NormalSettings.HeightScale;
            float3 PosB = new float3(NormalSettings.Distance, SampleB, 0f);

            int SampleCIndex = LinearArrayHelper.GetLinearIndexSafe(X, Y - 1, NormalSettings.Count);
            float SampleC = NormalHeights[SampleCIndex] * NormalSettings.HeightScale;
            float3 PosC = new float3(0f, SampleC, -NormalSettings.Distance);

            int SampleDIndex = LinearArrayHelper.GetLinearIndexSafe(X, Y + 1, NormalSettings.Count);
            float SampleD = NormalHeights[SampleDIndex] * NormalSettings.HeightScale;
            float3 PosD = new float3(0f, SampleD, NormalSettings.Distance);

            float3 NormalA = math.cross(PosC - CenterPos, PosA - CenterPos);
            float3 NormalB = math.cross(PosD - CenterPos, PosB - CenterPos);

            float3 Normal = math.normalize(NormalA + NormalB);
            float Angle = Vector3.Angle(NormalA, NormalB);

            bool IsCoreGridPoint = X % NormalSettings.CoreGridSpacing == 0 && Y % NormalSettings.CoreGridSpacing == 0;
            bool SkipPoint = !Border && !IsCoreGridPoint && Angle < NormalSettings.NormalReduceThreshold;

            if (SkipPoint) return;

            float3 LocalPos = new float3(X * NormalSettings.Distance, OwnHeight * NormalSettings.HeightScale, Y * NormalSettings.Distance);

            VertexData NewData = new VertexData
            {
                Normal = Normal,

                Position = LocalPos
            };

            int Idx = UnsafeListHelper.AddWithIndex(ref Vertices, in NewData);

            PointToVertexReference.TryAdd(RevIndex, Idx);
        }
    }

    [BurstCompile]
    public struct PatchTriangulationJob : IJobParallelFor
    {
        public GridSettings TriSettings;

        [ReadOnly] public NativeParallelHashMap<int2, int> VertexRef;

        [WriteOnly] public NativeList<int3>.ParallelWriter Triangles;

        public ProfilerMarker EdgeComputeMarker;
        public ProfilerMarker VertexGatherMarker;
        public ProfilerMarker InvalidSearchMarker;
        public ProfilerMarker TriangulationMarker;
        public ProfilerMarker HoleTriangulationMarker;

        public void Execute(int PatchIndex)
        {
            VertexGatherMarker.Begin();

            int PatchCountPerLine = (TriSettings.Count - 1) / TriSettings.CoreGridSpacing;

            int2 PatchPosition = LinearArrayHelper.ReverseLinearIndex(PatchIndex, PatchCountPerLine);

            int PatchLineVertCount = (TriSettings.Count - 1) / PatchCountPerLine;

            int2 StartVertex = PatchPosition * PatchLineVertCount;

            NativeList<int2> PatchVertices = new NativeList<int2>(PatchLineVertCount * PatchLineVertCount / 2, Allocator.Temp);

            float Size = math.length(new int2(PatchLineVertCount + 1));

            //Get all non-skipped vertices
            for (int x = StartVertex.x; x < StartVertex.x + PatchLineVertCount + 1; x++)
            {
                for (int y = StartVertex.y; y < StartVertex.y + PatchLineVertCount + 1; y++)
                {
                    int2 CandidatePos = new int2(x, y);

                    bool IsValid = VertexRef.ContainsKey(CandidatePos);

                    if (IsValid) PatchVertices.Add(CandidatePos);
                }
            }

            VertexGatherMarker.End();

            TriangulationMarker.Begin();

            var Triangulation = TriangulatorAlgorithm01.Delaunay(ref PatchVertices, (int)Size, ref InvalidSearchMarker, ref EdgeComputeMarker, ref HoleTriangulationMarker);

            //Add triangles to global triangle list, while getting the correct global indices
            for (int i = 0; i < Triangulation.Length; i++)
            {
                var Triangle = Triangulation[i];

                int3 Indices = Triangle.Indices;

                int2 PosA = PatchVertices[Indices.x];
                int2 PosB = PatchVertices[Indices.y];
                int2 PosC = PatchVertices[Indices.z];

                var IndexA = VertexRef[PosA];
                var IndexB = VertexRef[PosB];
                var IndexC = VertexRef[PosC];

                int3 GlobalIndices = new int3(IndexA, IndexB, IndexC);

                Triangles.AddNoResize(GlobalIndices);
            }

            TriangulationMarker.End();
        }
    }

    //Support functions
    public static class LinearArrayHelper
    {
        public static int GetLinearIndex(int x, int y, int count)
        {
            return y + x * count;
        }

        public static int GetLinearIndex(int2 pos, int count)
        {
            return pos.y + pos.x * count;
        }

        public static int GetLinearIndexSafe(int x, int y, int count)
        {
            x = math.clamp(x, 0, count - 1);

            y = math.clamp(y, 0, count - 1);

            return y + x * count;
        }

        public static int2 ReverseLinearIndex(int index, int count)
        {
            int y = index % count;

            int x = index / count;

            return new int2(x, y);
        }
    }

    public static class UnsafeListHelper
    {
        public static unsafe int AddWithIndex<T>(ref NativeList<T>.ParallelWriter List, in T Element) where T : unmanaged
        {
            var ListData = List.ListData;

            int Idx = System.Threading.Interlocked.Increment(ref ListData->m_length) - 1;

            Unity.Collections.LowLevel.Unsafe.UnsafeUtility.WriteArrayElement(ListData->Ptr, Idx, Element);

            return Idx;
        }

        public static unsafe void Add<T>(ref NativeList<T>.ParallelWriter List, in T Element) where T : unmanaged
        {
            var ListData = List.ListData;

            int Idx = System.Threading.Interlocked.Increment(ref ListData->m_length) - 1;

            Unity.Collections.LowLevel.Unsafe.UnsafeUtility.WriteArrayElement(ListData->Ptr, Idx, Element);
        }
    }

    void OnDrawGizmos()
    {
        //Draw normals
        foreach (VertexData Vertex in CachedPoints) Gizmos.DrawRay(Vertex.Position, Vertex.Normal);

        //Draw triangles
        foreach (TriCache Tri in CachedTriangles)
        {
            Gizmos.DrawLine(Tri.Pos1 + (float3)transform.position, Tri.Pos2 + (float3)transform.position);
            Gizmos.DrawLine(Tri.Pos2 + (float3)transform.position, Tri.Pos3 + (float3)transform.position);
            Gizmos.DrawLine(Tri.Pos3 + (float3)transform.position, Tri.Pos1 + (float3)transform.position);
        }
    }
}



using Unity.Jobs;
using UnityEngine;
using Unity.Burst;
using Unity.Profiling;
using Unity.Mathematics;
using Unity.Collections;
using System.Collections;
using System.Collections.Generic;
using System.Runtime.CompilerServices;

//[BurstCompile]
public static class TriangulatorAlgorithm01
{
    public static NativeList<TriangleData> Delaunay(ref NativeList<int2> Points, in int Size, ref ProfilerMarker InvalidSearchMarker, ref ProfilerMarker EdgeComputeMarker, ref ProfilerMarker HoleTriangulationMarker)
    {
        NativeList<TriangleData> Triangles = new NativeList<TriangleData>(Points.Length * 2, Allocator.Temp);

        int2 FirstPoint = Points[0];

        int2 SuperTriangleA = new int2(Size / 2, -3 * Size) + FirstPoint;
        int2 SuperTriangleB = new int2(-3 * Size, 3 * Size) + FirstPoint;
        int2 SuperTriangleC = new int2(3 * Size, 3 * Size) + FirstPoint;

        int SuperTriangleIndexStart = Points.Length;

        Points.Add(SuperTriangleA);
        Points.Add(SuperTriangleB);
        Points.Add(SuperTriangleC);

        //var SuperTri = insertTriangle
        //Triangles.Add(SuperTri);

        for (int i = 0; i < Points.Length; i++)
        {
            int2 Point = Points[i];

            NativeList<int> BadTris = new NativeList<int>(2, Allocator.Temp);

            InvalidSearchMarker.Begin();

            //Find bad triangles
            for (int TriIndex = Triangles.Length - 1; TriIndex >= 0; TriIndex--)
            {
                TriangleData Triangle = Triangles[TriIndex];

                float DistSq = math.distancesq(Triangle.Center, Point);

                bool IsInside = DistSq < Triangle.RadiusSq;

                if (IsInside) BadTris.Add(TriIndex);
            }

            InvalidSearchMarker.End();

            //int2 values are vertex positions (represents edges)
            NativeParallelHashSet<int2> Polygon = new NativeParallelHashSet<int2>(6, Allocator.Temp);

            EdgeComputeMarker.Begin();

            //Compute valid polygons
            for (int BadTriIterator = 0; BadTriIterator < BadTris.Length; BadTriIterator++)
            {
                int BadTriIndex = BadTris[BadTriIterator];

                TriangleData BadTriangle = Triangles[BadTriIndex];

                if (!IsEdgeShared(BadTriangle.EdgeA, BadTriIndex, in BadTris, in Triangles)) Polygon.Add(BadTriangle.EdgeA);
                if (!IsEdgeShared(BadTriangle.EdgeB, BadTriIndex, in BadTris, in Triangles)) Polygon.Add(BadTriangle.EdgeB);
                if (!IsEdgeShared(BadTriangle.EdgeC, BadTriIndex, in BadTris, in Triangles)) Polygon.Add(BadTriangle.EdgeC);
            }

            EdgeComputeMarker.End();

            BadTris.Sort();

            //Remove tris that overlap with the new point
            for (int n = BadTris.Length - 1; n >= 0; n--)
            {
                int BadTriIndex = BadTris[n];

                Triangles.RemoveAtSwapBack(BadTriIndex);
            }

            HoleTriangulationMarker.Begin();

            //Triangulate new hole
            using var PolyIterator = Polygon.ToNativeArray(Allocator.Temp).GetEnumerator();

            while (PolyIterator.MoveNext())
            {
                int2 Edge = PolyIterator.Current;

                int3 NewTriIndices = new int3(Edge.x, Edge.y, i);

                TriangleData Tri = InsertTriangle(ref Triangles, NewTriIndices, in Points);

                if (!float.IsNaN(Tri.RadiusSq)) Triangles.Add(Tri);
            }

            HoleTriangulationMarker.End();
        }

        //Clean up
        //Remove triangles from super triangle
        NativeList<int> ToRemove = new NativeList<int>(3, Allocator.Temp);

        for (int r = 0; r < Triangles.Length; r++)
        {
            TriangleData Triangle = Triangles[r];

            if (math.any(Triangle.Indices == SuperTriangleIndexStart) || math.any(Triangle.Indices == SuperTriangleIndexStart + 1) || math.any(Triangle.Indices == SuperTriangleIndexStart + 2))
            {
                ToRemove.Add(r);
            }
        }

        ToRemove.Sort();

        for (int t = ToRemove.Length - 1; t >= 0; t++)
        {
            int RemoveIndex = ToRemove[t];

            Triangles.RemoveAtSwapBack(RemoveIndex);
        }

        //Remove vertices of super-triangle
        //Remove 3 times the same index, since the next item in the list is being moved back
        Points.RemoveRangeSwapBack(SuperTriangleIndexStart, 3);
               
        return Triangles;
    }

    static bool IsEdgeShared(in int2 Edge, in int SelfIndex, in NativeList<int> BadTris, in NativeList<TriangleData> Triangles)
    {
        for (int i = 0; i < BadTris.Length; i++)
        {
            int CurBadTrisIndex = BadTris[i];

            if (CurBadTrisIndex == SelfIndex) continue;

            TriangleData BadTri = Triangles[CurBadTrisIndex];

            if (TriangleContainsEdge(BadTri, Edge)) return true;
        }

        return false;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    static bool TriangleContainsEdge(in TriangleData Tri, in int2 Edge)
    {
        return IsCommonEdge(Edge, Tri.EdgeA) || IsCommonEdge(Edge, Tri.EdgeB) || IsCommonEdge(Edge, Tri.EdgeC);
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    static bool IsCommonEdge(in int2 EdgeA, in int2 EdgeB)
    {
        return math.all(EdgeA == EdgeB) || EdgeA.x == EdgeB.y && EdgeA.y == EdgeB.x;
    }

    static TriangleData InsertTriangle(ref NativeList<TriangleData> List, in int3 Indices, in NativeList<int2> Points)
    {
        int2 PosA = Points[Indices.x];
        int2 PosB = Points[Indices.y];
        int2 PosC = Points[Indices.z];

        float3 V1 = new float3(PosA.x, 0f, PosA.y);
        float3 V2 = new float3(PosB.x, 0f, PosB.y);
        float3 V3 = new float3(PosC.x, 0f, PosC.y);

        Geometry.CircumCircle(V1, V2, V3, out float3 Center, out float Radius);

        TriangleData Tri = new TriangleData
        {
            Indices = Indices,

            RadiusSq = Radius * Radius,

            Center = Center.xz
        };

        return Tri;
    }

    public static class Geometry
    {
        public static void InCenter(float2 P1, float2 P2, float2 P3, out float2 InCenter, out float InRadius)
        {
            float A = math.distance(P1, P2);
            float B = math.distance(P2, P3);
            float C = math.distance(P3, P1);

            float Perimeter = A + B + C;

            float X = (A * P1.x + B * P2.x + C * P3.x) / Perimeter;
            float Y = (A * P1.y + B * P2.y + C * P3.y) / Perimeter;

            InCenter = new float2(X, Y);

            float S = Perimeter / 2f;

            float TriangleArea = math.sqrt(S * (S - A) * (S - B) * (S - C));

            InRadius = TriangleArea / S;
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void CircumCircle(float3 A, float3 B, float3 C, out float3 CircleCenter, out float CircleRadius)
        {
            PerpendicularBisector(A, B - A, out float3 PerpABPos, out float3 PerpABDir);
            PerpendicularBisector(A, C - A, out float3 PerpACPos, out float3 PerpACDir);

            float TAB = Intersection(PerpABPos, PerpABPos + PerpABDir, PerpACPos, PerpACPos + PerpACDir);

            CircleCenter = PerpABPos + PerpABDir * TAB;

            CircleRadius = math.length(CircleCenter - A);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        static void PerpendicularBisector(float3 Pos, float3 Dir, out float3 BisectorPos, out float3 BisectorDir)
        {
            float3 M = Dir * 0.5f;

            float3 Cross = math.normalize(math.cross(math.normalize(Dir), new float3(0f, 1f, 0f)));

            BisectorPos = Pos + M;

            BisectorDir = Cross;
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        static float Intersection(float3 P1, float3 P2, float3 P3, float3 P4)
        {
            float TAB = ((P4.x - P3.x) * (P1.z - P3.z) - (P4.z - P3.z) * (P1.x - P3.x)) / ((P4.z - P3.z) * (P2.x - P1.x) - (P4.x - P3.x) * (P2.z - P1.z));

            return TAB;
        }
    }

    public struct TriangleData
    {
        public int3 Indices;

        internal float RadiusSq; //Squared
        internal float2 Center;

        internal int2 EdgeA => new int2(Indices.xy);
        internal int2 EdgeB => new int2(Indices.yz);
        internal int2 EdgeC => new int2(Indices.zx);
    }
}

[DreamingImLatios]

I was specifically referring to your boxing error. I’m not interested in debugging your code. I’m interested in enabling you to debug your code. The boxing error sounds like a Burst error, in which case your code should still run without Burst, and you can debug it. But I’m suspicious you were trying to use a Debug.Log() in Burst without using string interpolation (which is the only type of string Burst really supports for logging).

[CodeSmile]

You know how to use the debugger? If the triangles aren’t being set just step through your code as it executes to learn why it doesn’t work.

@Ambrose998800

[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Sequential)]

Use a using for this monster. Your IDE should be able to squash that down to
[StructLayout(LayoutKind.Sequential)]

In CreateTerrain() you are using a try/finally which is superfluous, Jobs don’t throw exceptions that you need to handle. You can let any exception trickle through here because any exception will be a bug that you need to fix.

You also call Complete() right after each job, which I take is part of the process but if not, you should know that this is not optimal as it forces the job to run to completion immediately, leaving no room for background processes to run simultaneously.

@Ambrose998800

Triangles.AddNoResize(GlobalIndices);

Is Triangles initialized to a non-zero capacity?