GPU Character Package with Dynamic Bones

Hello, I recently started to learn about ECS and I’m currently trying to figure out the/a proper way to create GPU skinned characters where I update bone matrices in jobs. I started looking into the GPU Character Package here (GitHub - joeante/Unity.GPUAnimation: Simple but very fast GPU vertex shader based animation system for Unity.Entities).

Currently I set a Dynamic Buffer with float4x4s for each bone per character to each Character Entity like this:

var boneMatrices = BoneMatrixUtility.UpdateBoneMatrices(renderer, renderer.bones);
var buffer = manager.AddBuffer<BoneBufferElement>(entity);
foreach (var m in boneMatrices)
       buffer.Add(new BoneBufferElement() {Value = m});

Q1: I dont need a resizeable container but I havent figured out yet how to put an writeable array (other than a DynamicBuffer) inside an IComponentData (should I better use BlobArray?)

I’m able to pass that to the shader in the GPU Character Render System via ComputeBuffer and Reinterpreting the DynamicBuffer to a NativeArray.
Q2: But how do I update the DynamicBuffer (or NativeArray) from a Job before passing it to the shader. I just tried the following:

    [UpdateBefore(typeof(GpuCharacterRenderSystem))]
    public class BoneSystem : JobComponentSystem
    {
        public struct BoneUpdate : IJobForEachWithEntity<LocalToWorld>
        {
            public BufferFromEntity<BoneBufferElement> Buffer;

            public float time;
          
            public void Execute(Entity entity, int index, [ReadOnly] ref LocalToWorld c0)
            {
                var buffer = Buffer[entity];
                for (var i = 0; i < buffer.Length; i++)
                {
                    var val = buffer[i];
                    val.Value.c3.y += math.sin(time * (index+1) / 2f * i) * .5f;
                    buffer[i] = val;
                  
                }
            }
        }

        protected override JobHandle OnUpdate(JobHandle inputDeps)
        {
            var job = new BoneUpdate()
            {
                time = Time.time,
                Buffer = GetBufferFromEntity<BoneBufferElement>()
            };
            var handle = job.Schedule(query, inputDeps);
            return handle;
        }

        private EntityQuery query;

        protected override void OnCreate()
        {
            base.OnCreate();
            query = EntityManager.CreateEntityQuery(
                ComponentType.ReadOnly<BoneBufferElement>()
            );
        }
    }

but I get:

The container does not support parallel writing. Please use a more suitable container type.
Unity.Jobs.LowLevel.Unsafe.JobsUtility.CreateJobReflectionData (System.Type wrapperJobType, System.Type userJobType, Unity.Jobs.LowLevel.Unsafe.JobType jobType, System.Object managedJobFunction0) (at <86888d7bf865490ca0c08194bf6338db>:0)

Any help is greatly appreciated :slight_smile:

you need to add a readonly attribute above Buffer.

[ReadOnly]
BufferFromEntity<BoneBufferElement> Buffer;

Alternatively you could use a IJobForEach_BC<BoneBufferElement, LocalToWorld> and not use the buffer from entity.

2 Likes

@desertGhost thanks for your help.
I tried to mark it as readonly before but that didnt work because I need to update the matrices of the buffer inside the job

Trying your second suggestion and marking everything as ReadOnly I get

InvalidOperationException: The system Marwi.ECS.SkinnedMesh.GPUAnimation.GpuCharacterRenderSystem
reads Marwi.ECS.SkinnedMesh.GPUAnimation.BoneBufferElement via
 BoneSystem:BoneUpdate but that type was not returned as a job dependency.
To ensure correct behavior of other systems,
the job or a dependency of it must be returned from the OnUpdate method.

I’m not sure why tho, because in OnUpdate I schedule it like this:

var job = new BoneUpdate()
{
   time = Time.time,
};
var handle = job.Schedule(query, inputDeps);
return handle;

when I schedule it with this instead of my EntityQuery I dont get this error. Can anyone explain why?

But I’m still not allowed to write to the DynamicBuffer

InvalidOperationException: The previously scheduled job BoneSystem:BoneUpdate 
writes to the NativeArray
BoneUpdate.Iterator. You must call JobHandle.Complete() on the job BoneSystem:BoneUpdate,
before you can read from the NativeArray safely.

I can call JobHandle.CompleteAll() with my UpdateBones Job Handle in my character loop but I assume this is waiting on the main thread, right?

protected override JobHandle OnUpdate(JobHandle inputDeps)
{
    _Characters.Clear();
    EntityManager.GetAllUniqueSharedComponentData(_Characters);

    foreach (var character in _Characters)
    {
        if (character.Material == null || character.Mesh == null)
            continue;
        m_Characters.SetFilter(character);
        
        var bonesjob = new WiggleBones() {time = Time.time};
        var handle = bonesjob.Schedule(m_Characters, inputDeps);
        JobHandle.CompleteAll(ref inputDeps, ref handle);
    }
    
    return inputDeps;
}

My understanding is that the BufferFromEntity is a lookup table (not the actual buffer). Marking the BufferFromEntity as readonly shouldn’t effect your ability to modify the buffer.

If you need to modify the buffer and you are using it IJobForEach_BC you should not mark the buffer as readonly.

Can you post the code for the job extending IJobForEach_BC?

Have you tried added query.AddDependency(inputDeps); to your scheduling code?

One problem is that you are trying to write to a buffer that has been marked readonly.

Your entity query should be query = GetEntityQuery(typeof(BoneBufferElement)); because you want to write to the buffer.

Then in your IJobForEach_BC:

public void Execute(DynamicBuffer<BoneBufferElement> buffer, [ReadOnly] ref LocalToWorld localToWorld){/*logic for modifying buffer*/}

Yes that would wait on the main thread.

Thanks for your detailed answer. So what I currently have Job-Wise looks like this:

public struct UpdateBones : IJobForEachWithEntity_EBBC<BoneBufferElement, ChainBufferElement, ChainHead>
{
          public void Execute(Entity entity, int index, DynamicBuffer<BoneBufferElement> bones, [ReadOnly] DynamicBuffer<ChainBufferElement> chain,
                [ReadOnly] ref ChainHead head)
            {

which works fine (the ChainBufferElement is not modified thus marked as ReadOnly, same as the head component)

my Query looks like this

m_Characters = GetEntityQuery(
                ComponentType.ReadOnly<RenderCharacter>(),
                ComponentType.ReadWrite<BoneBufferElement>(),
                ComponentType.ReadOnly<ChainBufferElement>(),
                ComponentType.ReadOnly<ChainHead>()
            );

Oh maybe my mistake is adding the job handle as a dependency for the query. I need to test that tomorrow. The Scheduling code currently looks like this:

 protected override JobHandle OnUpdate(JobHandle inputDeps)
        {
            _Characters.Clear();
            EntityManager.GetAllUniqueSharedComponentData(_Characters);

//            var handles = new NativeArray<JobHandle>(_Characters.Count, Allocator.Temp);
            for (var i = 0; i < _Characters.Count; i++)
            {
                var character = _Characters[i];
                if (character.Material == null || character.Mesh == null)
                    continue;
                m_Characters.SetFilter(character);

                var bonesjob = new UpdateBones()
                {
                    time = Time.time,
                    dt = Time.deltaTime,
                };
                var handle = bonesjob.Schedule(m_Characters, inputDeps);
//                handles[i] = handle;
//                m_Characters.AddDependency(handle);
                JobHandle.CompleteAll(ref inputDeps, ref handle);
            }

//            JobHandle.CompleteAll(handles);
//            handles.Dispose();

            return inputDeps;
        }

I need to do a CompleteAll call right now but maybe it’s just because I added the wrong handle as a dependency.

If I understand ISharedComponentData right and how this loop works it would mean:
If I instantiate e.g. 1000 entities using the same ISharedComponentData (in this case the same RenderCharacter Data) my list of UniqueCharacters (“_Characters”) would only contain 1 element?

I think that is definitely part of the issue.

I’m wondering if it is actually necessary to schedule per character mesh. As long as the material and the character mesh aren’t null you should just be able to do something like:

        OnUpdate(JobHandle inputDeps)
        {
                var bonesjob = new UpdateBones
                {
                    time = Time.time,
                    dt = Time.deltaTime,
                };
                var handle = bonesjob.Schedule(m_Characters, inputDeps);

            return handle;
        }

That should be the case.

so I tried both approaches (adding the correct dependency to the query and also your code and it works IF I call “inputDeps.Complete()” in the GPUCharacterRenderSystem before looping the ISharedComponents.

If I don’t wait for the completion I get

InvalidOperationException:
The previously scheduled job UpdateBonesSystem:UpdateBones
writes to the NativeArray UpdateBones.Iterator.
You must call JobHandle.Complete() on the job UpdateBonesSystem:UpdateBones,
before you can read from the NativeArray safely.
...... (pointing to Update in the GpuCharacterSystem).

I tried adding both JobSystems to the same UpdateGroup and marking the BoneSystem with

[UpdateInGroup(typeof(PresentationSystemGroup))]
[UpdateBefore(typeof(GpuCharacterRenderSystem))]

but I get the same error. I also tried adding the BoneSystem to the SimulationSystemGroup where it actually belongs I think but that doesnt help either.

What I do in GpuCharacterRenderSystem is also loop the characters and then accessing the BoneBufferArray

var bonesBuffer = GetBufferFromEntity<BoneBufferElement>()[character.Entity];
 var bones = bonesBuffer.ToNativeArray();

and GetBufferFromEntity is where the exception is thrown.


Ok I actually found a solution just now but I’m not 100% sure why it helps. I changed the code from

 JobHandle jobA, jobB;
var coords = m_Characters.ToComponentDataArray<AnimationTextureCoordinate>(Allocator.TempJob, out jobA);
var localToWorld = m_Characters.ToComponentDataArray<LocalToWorld>(Allocator.TempJob, out jobB);
               
var bonesBuffer = GetBufferFromEntity<BoneBufferElement>(true)[character.Entity];
var bones = bonesBuffer.ToNativeArray();
JobHandle.CompleteAll(ref jobA, ref jobB);

to

 JobHandle jobA, jobB;
var coords = m_Characters.ToComponentDataArray<AnimationTextureCoordinate>(Allocator.TempJob, out jobA);
var localToWorld = m_Characters.ToComponentDataArray<LocalToWorld>(Allocator.TempJob, out jobB);
JobHandle.CompleteAll(ref jobA, ref jobB);
Profiler.EndSample();
               
var bonesBuffer = GetBufferFromEntity<BoneBufferElement>(true)[character.Entity];
var bones = bonesBuffer.ToNativeArray();

Does this ensure the Simulation/Transform updates are done so the UpdateBoneSystem is obviously done? But shouldnt the Attribute-Like-Scheduling help as well in making sure previous systems are done?

In this case no.

I just got the chance to download the GPUAnimationExamples project from GitHub (sorry about that) last night and after reviewing the entire GpuCharacterRenderSystem, you need to ensure that your UpdateBone system’s jobs complete before GpuCharacterRenderSystem fetches the bones buffer.

The issue is that the GpuCharacterRenderSystem does not schedule its jobs based off of the inputDeps. It just returns them. This means that you must ensure that any jobs that operate on the same data as GpuCharacterRenderSystem (e.g. the bone buffer of an entity) must complete before the GpuCharacterRenderSystem is updated.

If you process the bone update system like:

You need to call JobHandle.CompleteAll() to ensure that you are done manipulating the bones buffer before the GpuCharacterRenderSystem accesses them.

If you return the handle from the bone jobs like:

you need to ensure that your UpdateBone system’s jobs complete before GpuCharacterRenderSystem fetches the bones buffer by calling JobHandle.CompleteAll(ref inputDeps); in the GpuCharacterRenderSystem before entering the foreach loop.

thanks for having a look into the package. Yeah I guess by waiting for the LocalToWorld etc datas I’m implicitly waiting for the input dependencies to be completed.

Anyways I got another question :slight_smile:

Is there a rule for the order in which I define my buffer args in a Job?

For Example this works:

struct UpdateHeadDelta : IJobForEach_BBC<SimulationBufferElement, ChainBufferElement, LocalToWorld>
{
public void Execute(
    DynamicBuffer<SimulationBufferElement> simBuffer,
    [ReadOnly] DynamicBuffer<ChainBufferElement> chain,
    [ReadOnly] ref LocalToWorld world)
{

whereas this throws an error:

struct UpdateHeadDelta : IJobForEach_BBC<ChainBufferElement, SimulationBufferElement, LocalToWorld>
{
    [ReadOnly] public NativeArray<float3> TargetPositions;

    public float deltaTime;
   
    public void Execute(
        [ReadOnly] DynamicBuffer<ChainBufferElement> chain,
        DynamicBuffer<SimulationBufferElement> simBuffer,
        [ReadOnly] ref LocalToWorld world)
    {

the error looks like

NullReferenceException: Object reference not set to an instance of an object
Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle.CheckReadAndThrow (Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle handle) (at <f206f586b3cb43239fcbb553ea9e7c16>:0)
Unity.Entities.BufferAccessor`1[T].get_Item (System.Int32 index) (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities/Iterators/ArchetypeChunkArray.cs:373)
Unity.Entities.JobForEachExtensions+JobStruct_Process_BBC`4[T,T0,T1,T2].ExecuteChunk (Unity.Entities.JobForEachExtensions+JobStruct_Process_BBC`4[T,T0,T1,T2]& jobData, System.IntPtr bufferRangePatchData, System.Int32 begin, System.Int32 end, Unity.Entities.ArchetypeChunk* chunks, System.Int32* entityIndices) (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities/IJobForEach.gen.cs:3107)
Unity.Entities.JobForEachExtensions+JobStruct_Process_BBC`4[T,T0,T1,T2].Execute (Unity.Entities.JobForEachExtensions+JobStruct_Process_BBC`4[T,T0,T1,T2]& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities/IJobForEach.gen.cs:3066)

both use this query:

headQuery = GetEntityQuery(
                ComponentType.ReadOnly<LocalToWorld>(),
                ComponentType.ReadOnly<Fish>(),
                ComponentType.ReadOnly<ChainBufferElement>(),
                ComponentType.ReadWrite<SimulationBufferElement>()
            );

Does the argument order for jobs matter (if a buffer is read-write vs readonly?) If I mark both as read-write (and adjust the query) both jobs work.

I think this is a bug.

I tend to group ReadOnly data with ReadOnly data when I write jobs, but I don’t think (from a design stand point) that a user should have to group components by whether or not they are ReadOnly. I think the error you are seeing might be a bug with applying attributes to DynamicBuffers. I had a similar error when I marked a DynamicBuffer as both ReadOnly and FilterChanged.

1 Like

Ok, that’s what I guessed but good to know, thanks :slight_smile:

No day without new questions. Earlier today I started refactoring the RenderSystem to group together the bone matrices in a NativeArray to reduce draw calls (instead of 100 draw calls per Entity only one (except the buffer gets too big) per ISharedComponentData (RenderCharacter)).

For that purpose I added a Job to set the bone data, the issue is when I have multiple Unique Characters (e.g. a “fish” and a “plant”): then I get values from both types.
Currently the job looks like this:

    struct CreateRenderBuffers : IJobForEachWithEntity_EBC<BoneBufferElement, LocalToWorld>
    {
        [ReadOnly]
        public int BonesBufferStride;
        // we need to do this to be able to write for all entities to the same buffer at once
        // we could potentially avoid this by introducting an "buffer entity" that has dynamic buffers for this
        [NativeDisableParallelForRestriction]
        public NativeArray<float4x4> BoneBuffer;
        [NativeDisableParallelForRestriction]
        public NativeArray<float4x4> ObjectBuffer;

        public void Execute(Entity entity, int index, [ReadOnly] DynamicBuffer<BoneBufferElement> bones, [ReadOnly] ref LocalToWorld world)
        {
            ObjectBuffer[index] = world.Value;

            if (BoneBuffer.Length > 0 && bones.Length > 0)
            {
                var startIndex = index * BonesBufferStride;
//            Debug.Log(index + " / start index " + startIndex + " buffer size " + BoneBuffer.Length + " / " + bones.Length + " end index: " + (bones.Length-1 + startIndex));
                for (var i = 0; i < bones.Length; i++)
                {
                    var bufferIndex = startIndex + i;
                    if (bufferIndex >= BoneBuffer.Length) break;
                    BoneBuffer[bufferIndex] = bones[i].Value;
                }
            }
         
       
        }
    }

and in the foreach(character loop I do:

// merge buffers for one big draw call
                var bonesPerCharacter = character.BonesCount;
                if (bonesPerCharacter <= 0) continue;
                allBones.SafeDispose();
                allBones = new NativeArray<float4x4>(characterCount * bonesPerCharacter, Allocator.TempJob);
                allObjects.SafeDispose();
                allObjects = new NativeArray<float4x4>(characterCount, Allocator.TempJob);

                var bufferJob = new CreateRenderBuffers()
                {
                    BonesBufferStride = bonesPerCharacter,
                    BoneBuffer = allBones,
                    ObjectBuffer = allObjects,
                };
//                inputDeps.Complete();
                var bufferJobHandle = bufferJob.Schedule(bufferQuery, inputDeps);
                JobHandle.CompleteAll(ref jobA, ref bufferJobHandle);
                drawer.Draw(allBones, coords.Reinterpret_Temp<AnimationTextureCoordinate, float3>(),
                    allObjects,
                    character.CastShadows, character.ReceiveShadows);

Is there a way to schedule the job for only entities that have that specific unique type of “RenderCharacter” (ISharedDataComponent) or how could I organize that?

My understanding is that entities with the same SharedDataComponent are store in the same chunk. You should be able to use IJobChunk to process entities that have the same RenderCharacter.

1 Like

Setfilter

1 Like

Oh thanks, that was actually missing