Converting Entities.ForEach(...).ScheduleParallel() to IJobEntityBatch, BufferTypeHandle invalidated

Unity Engine
, ,
1

I am trying to convert an Entities.ForEach().ScheduleParallel() to an IJobEntityBatch and I am having difficulty with invalidated BufferTypeHandles in my job code. I am making this conversion because I run expensive code per entity, but the ForEach method runs a single thread per chunk and all of my entities fit into 1 chunk making the resulting job take forever. If I add junk data to the entities to make them span across multiple chunks, it runs much faster. So I think an IJobEntityBatch change is in order.

For example, the ForEach looks like this:

Entities
     .WithBurst()
     .ForEach((ref PhysicsCollider collider, ref DynamicBuffer<VertexBufferElement> buffer) =>
{
    // Create new shapes and replace them on the collider.
    // This works well if I force the chunk to break up with a bunch of data (16k limit per chunk)
}).ScheduleParallel();

And the job code looks like this:

private struct UpdateColliderJob : IJobEntityBatch
{
    public ComponentTypeHandle<PhysicsCollider> PhysicsColliderTypeHandle;
    [ReadOnly] public BufferTypeHandle<VertexBufferElement> VertexBufferTypeHandle;
 
    [BurstCompile]
    public void Execute(ArchetypeChunk batchInChunk, int batchIndex)
    {
        var physicsColliders = batchInChunk.GetNativeArray(PhysicsColliderTypeHandle);
        var vertexBuffers = batchInChunk.GetBufferAccessor(VertexBufferTypeHandle);

        for (int i = 0; i < batchInChunk.Count; i++)
        {
            // Create new shapes and replace them on the collider,
            // but creating shapes invalidates the VertexBufferTypeHandle.
        }
    }
}

Surely there is a way to make this run multi-threaded other than forcing the entities into separate chunks through data size or archetype differences. Any suggestions?

2

IJobEntityBatch is the right choice. Can you show scheduling code?

1 Like
3 
EntityQuery query;

protected override void OnCreate()
{
    query = GetEntityQuery(
        typeof(PhysicsCollider),
        typeof(VertexBufferElement));
}

protected override void OnUpdate()
{
    Dependency.Complete();

    var job = new UpdateColliderJob
    {
        PhysicsColliderTypeHandle = GetComponentTypeHandle<PhysicsCollider>(),
        VertexBufferTypeHandle = GetBufferTypeHandle<VertexBufferElement>(true),
    };

    Dependency = job.ScheduleParallel(query, 8, Dependency);
    Dependency.Complete();
}

I am manually using Dependency.Complete() simply for testing (using a StopWatch to time the function, but that code is removed to keep it simple).

4

Actually, I am not sure what I did. I reverted back to the old ForEach code, and it seems to be invalidating the buffer as well when creating collision geometry. I must have had a bug or something when I originally tested it, but both seem to be behaving the same way now.

EDIT: ok, I figured it out. When the buffer comes into the ForEach as an “in” variable, I start getting errors when the physics geometry is created as it is read only, and I assume the job is attempting to invalidate it. However as a “ref”, the ForEach works fine and parallelization occurs. I assume it is still being invalidated, but I am done with it by the time a shape is being created, so no issues. With the job, I lose access to the BufferTypeHandle when a buffer is invalidated, so that becomes a problem for the latter entities to be processed in the parallel job.

So there is still a problem where the ForEach works, but the IJobEntityBatch does not. I’ll update the original code to reflect the “ref”.

5

None of the code you posted suggests you are doing anything wrong. Can you post the specific error message you are getting when using “in” for Entities.ForEach? Also, does passing in “false” to GetBufferTypeHandle and removing [ReadOnly] solve any issues? Right now I don’t really understand what you mean by “invalidated”.

6

With the buffer set as an “in” variable, i get this error:
“InvalidOperationException: The BufferTypeHandle has been declared as [ReadOnly] in the job, but you are writing to it.”

I thought that this was invalidation occurring as described in the DynamicBuffer documentation. But it is not. I am able to read/write the DynamicBuffer after shape creation when loaded as ref. In this case, the error is thrown from collider Create() function checking for read/write access to the data being passed to it.

So since invalidation is not the issue, I want to turn towards the job. I thought I was getting a different error yesterday, but today, with the buffer not marked as read only, I am getting errors that the resulting “BlobAssetReference is not valid.”

I appreciate you trying to help and I feel like I have not been providing enough relevant information due to NDA nonsense. While I cannot share the code I am working on (not my property to share), I do want to make a small repro case so that you have something concrete to look at. I’ll use that to submit a bug report as well (I can reliably crash Unity it seems.) I’ll post back with a repro case in a bit.

7

Alright, I have attached a repro case. Here is the system code directly:

using System.Diagnostics;
using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Mathematics;
using Unity.Physics;
using Unity.Physics.Systems;

[UpdateInGroup(typeof(FixedStepSimulationSystemGroup))]
[UpdateBefore(typeof(BuildPhysicsWorld))]
public class TerrainColliderUpdateSystem : SystemBase
{
   EntityQuery query;

   private struct UpdateTerrainColliderJob : IJobEntityBatch
   {
       [ReadOnly] public ComponentTypeHandle<PhysicsCollider> PhysicsColliderTypeHandle;
       public BufferTypeHandle<HeightElement> HeightsTypeHandle;

       [BurstCompile]
       public void Execute(ArchetypeChunk batchInChunk, int batchIndex)
       {
           var physicsColliders = batchInChunk.GetNativeArray(PhysicsColliderTypeHandle);
           var heightsBuffers = batchInChunk.GetBufferAccessor(HeightsTypeHandle);

           for (int i = 0; i < batchInChunk.Count; i++)
           {
               var collider = physicsColliders[i];
               var heightsBuffer = heightsBuffers[i];

               var colliders = Unity.Physics.TerrainCollider.Create(
                       heightsBuffer.Reinterpret<float>().AsNativeArray(),
                       new int2(ReproSetup.Resolution),
                       new float3(1),
                       Unity.Physics.TerrainCollider.CollisionMethod.Triangles
               );

               collider.Value.Dispose();
               collider.Value = colliders;
           }
       }
   }

   protected override void OnCreate()
   {
       query = GetEntityQuery(typeof(PhysicsCollider), typeof(HeightElement));
   }

   protected override void OnUpdate()
   {
       var mode = ReproSetup.Mode;
       Dependency.Complete();
       var stopWatch = new Stopwatch();
       stopWatch.Start();


       if (mode == SystemMode.ForEach)
       {
           Entities
                   .WithBurst()
                   .ForEach((ref PhysicsCollider collider,
                           ref DynamicBuffer<HeightElement> heightsBuffer) =>
                   {
                       var colliders = Unity.Physics.TerrainCollider.Create(
                                           heightsBuffer.Reinterpret<float>().AsNativeArray(),
                                           new int2(ReproSetup.Resolution),
                                           new float3(1),
                                           Unity.Physics.TerrainCollider.CollisionMethod.Triangles
                                   );

                       collider.Value.Dispose();
                       collider.Value = colliders;
                   }).ScheduleParallel();
       }
       else
       {
           var job = new UpdateTerrainColliderJob
           {
               PhysicsColliderTypeHandle = GetComponentTypeHandle<PhysicsCollider>(),
               HeightsTypeHandle = GetBufferTypeHandle<HeightElement>(),
           };

           // Split the chunk up for parallel processing
           Dependency = job.ScheduleParallel(query, 4, Dependency);
       }


       Dependency.Complete();
       stopWatch.Stop();
       UnityEngine.Debug.Log($"System Execution Time: {stopWatch.ElapsedTicks / 10000.0f}");
   }
}

In the SampleScene, the “ReproSetup” game object has a selector to run as “For Each” or “Job”. With “For Each” selected, the workload runs successfully. You can even see that the frametime reduces significantly if you set the InternalCapacity attribute on the DummyElement.cs to 3000 (just breaks up the chunk). So multi-threaded shape changes should work. However, with “Job” selected, we get the following error:

InvalidOperationException: The BlobAssetReference is not valid. Likely it has already been unloaded or released.
Unity.Entities.BlobAssetReferenceData.ValidateNonBurst () (at Library/PackageCache/com.unity.entities@0.17.0-preview.42/Unity.Entities/Blobs.cs:260)
Unity.Entities.BlobAssetReferenceData.ValidateNotNull () (at Library/PackageCache/com.unity.entities@0.17.0-preview.42/Unity.Entities/Blobs.cs:277)
Unity.Entities.BlobAssetReference`1[T].get_Value () (at Library/PackageCache/com.unity.entities@0.17.0-preview.42/Unity.Entities/Blobs.cs:365)
Unity.Physics.Broadphase+PrepareStaticBodyDataJob.ExecuteImpl (System.Int32 index, System.Single aabbMargin, Unity.Collections.NativeArray`1[T] rigidBodies, Unity.Collections.NativeArray`1[T] aabbs, Unity.Collections.NativeArray`1[T] points, Unity.Collections.NativeArray`1[T] filtersOut, Unity.Collections.NativeArray`1[T] respondsToCollisionOut) (at Library/PackageCache/com.unity.physics@0.6.0-preview.3/Unity.Physics/Collision/World/Broadphase.cs:831)
Unity.Physics.Broadphase+PrepareStaticBodyDataJob.Execute (System.Int32 index) (at Library/PackageCache/com.unity.physics@0.6.0-preview.3/Unity.Physics/Collision/World/Broadphase.cs:819)
Unity.Jobs.IJobParallelForDeferExtensions+JobParallelForDeferProducer`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at Library/PackageCache/com.unity.jobs@0.8.0-preview.23/Unity.Jobs/IJobParallelForDefer.cs:62)

EDIT: This is the first error that appears in the log, there are many others afterwards, but I assume they are a result of this error.

7585135–940336–Project.zip (30 KB)

8

For as comfortable as I am with blobs, Unity Physics uses them in a raw and arguable hacky fashion. I don’t personally use Unity Physics so I’m probably not much help, but you may want to post this issue there. There’s a good chance the devs there will be interested and may be able to help you out.

1 Like
9

Sounds good, I will post it over there. Thanks for taking a look!

10

I figured it out. I just needed to add

physicsColliders[i] = collider;

below line 40 in the system code above. I guess the ForEach uses a reference whereas the Job uses a copy.

11

Wow. It is unlike me to miss that. :sweat_smile:

Glad you figured it out!