[Example] IJobForEach and IBufferElementData (Unity 2020.1)

Unity Engine
,
1

In the light of current changes, I decided to write up and post very basics example of
IJobProcessComponentDataWithEntity IJobForEach and IBufferElementData
without injection, hoping it will become useful. And maybe can be further improved.

Unity 2020.1a15+ (Entities 0.2.0-preview.18)
Last update 2019 Dec. 10
https://github.com/Antypodish/DOTS_IJobForEach_IBufferElementData

// Unity DOTS-ECS very simple example with IJobForEachWithEntity, IJobForEach_BC and IBufferElementData.

// 2019.12.10

// Requires Unity 2020.1.0a15+

// Tested with
// Burst 1.2.0-preview.10
// Collections 0.2.0-Preview.13
// Entities 0.2.0-preview.18
// Jobs 0.2.1-Preview.3
// Mathematics 1.1.0.preview.1

// New repo name
// ECS_IJobForEach_IBufferElementData
// https://github.com/Antypodish/ECS_IJobForEach_IBufferElementData
// Old repo name
// ECS_IJobProcessComponentDataWithEntity_IBufferElementData
// https://github.com/Antypodish/ECS_IJobProcessComponentDataWithEntity_IBufferElementData

using Unity.Collections ;
using Unity.Entities ;
using Unity.Jobs ;
using Unity.Burst ; // See commented out [BurstCompile] lines, above jobs.

using UnityEngine ;

namespace ECS.Test
{

    struct Instance : IComponentData
    {
        public float f ;
    }

    // For testing in Job_BC
    struct SomeBufferElement : IBufferElementData
    {
        public int i ;
    }

    // For testing in JobWithEntity
    struct SomeFromEntityBufferElement : IBufferElementData
    {
        public int i ;
    }

    public class BufferWithJobSystem : JobComponentSystem
    {
 
        // protected override void OnCreateManager ( int capacity ) // Obsolete
        protected override void OnCreate ( )
        {
            base.OnCreate ( ) ;

            Debug.LogWarning ( "Burst is disabled, to use Debug.Log in jobs." ) ;
            Debug.LogWarning ( "Jobs are executed approx every second." ) ;

            Instance instance = new Instance () ;

            Entity entity = EntityManager.CreateEntity ( typeof (Instance) ) ;
 
            EntityManager.SetComponentData ( entity, instance ) ;
            EntityManager.AddBuffer <SomeBufferElement> ( entity ) ;
 
            DynamicBuffer <SomeBufferElement> someBuffer = EntityManager.GetBuffer <SomeBufferElement> ( entity ) ;

            // Add two elements to dynamic buffer.
            SomeBufferElement someBufferElement = new SomeBufferElement () ;
            someBufferElement.i = 100000 ;
            someBuffer.Add ( someBufferElement ) ;
            someBufferElement.i = 200000 ;
            someBuffer.Add ( someBufferElement ) ;

            EntityManager.Instantiate ( entity ) ; // Clone entity.

   
            entity = EntityManager.CreateEntity ( typeof (Instance) ) ;
 
            EntityManager.SetComponentData ( entity, instance ) ;
            EntityManager.AddBuffer <SomeFromEntityBufferElement> ( entity ) ;

            DynamicBuffer <SomeFromEntityBufferElement> someFromEntityBuffer = EntityManager.GetBuffer <SomeFromEntityBufferElement> ( entity ) ;

            // Add two elements to dynamic buffer.
            SomeFromEntityBufferElement someFromEntityBufferElement = new SomeFromEntityBufferElement () ;
            someFromEntityBufferElement.i = 1000 ;
            someFromEntityBuffer.Add ( someFromEntityBufferElement ) ;
            someFromEntityBufferElement.i = 10 ;
            someFromEntityBuffer.Add ( someFromEntityBufferElement ) ;

            EntityManager.Instantiate ( entity ) ; // Clone entity.

        }
 
        float previoudTime = 0 ;

        protected override JobHandle OnUpdate ( JobHandle inputDeps )
        {
   
            float time = Time.time ;

            // Execute approx every second.
            if ( time > previoudTime + 1 )
            {
                previoudTime = time ;       
            }
            else
            {
                return inputDeps ;
            }

            JobHandle job_withEntity = new Job_WithEntity ()
            {
                time = time,
                someBuffer = GetBufferFromEntity <SomeFromEntityBufferElement> ( false ) // Read and write
 
            }.ScheduleSingle ( this, inputDeps ) ; // Using instead job.Schedule ( this, inputDeps ), for single threaded test and debug.
            // }.Schedule ( this, inputDeps ) ; // Allow execute job in parallel, if there is enough entities.
   
            JobHandle job_BC = new Job_BC ()
            {
                time = time,
       
            }.ScheduleSingle ( this, job_withEntity ) ; // Using instead job.Schedule ( this, inputDeps ), for single threaded test and debug.
            // }.Schedule ( this, jobWithEntity ) ; // Allow execute job in parallel, if there is enough entities.

            return job_BC ;
        }

        // [BurstCompile] // Disbaled burst, as Debug.Log is used.
        [RequireComponentTag ( typeof ( SomeFromEntityBufferElement ) ) ]
        // struct Job: IJobProcessComponentDataWithEntity <Instance> // Obsolete
        struct Job_WithEntity : IJobForEachWithEntity <Instance>
        {
            public float time;

            // Allow buffer read write in parralel jobs
            // Ensure, no two jobs can write to same entity, at the same time.
            // !! "You are somehow completely certain that there is no race condition possible here, because you are absolutely certain that you will not be writing to the same Entity ID multiple times from your parallel for job. (If you do thats a race condition and you can easily crash unity, overwrite memory etc) If you are indeed certain and ready to take the risks.
            // https://discussions.unity.com/t/712852/4
            [NativeDisableParallelForRestriction]
            public BufferFromEntity <SomeFromEntityBufferElement> someBuffer ;

            public void Execute( Entity entity, int index, ref Instance tester )
            {
                tester.f = time ;

                DynamicBuffer <SomeFromEntityBufferElement> dynamicBuffer = someBuffer [entity] ;

                SomeFromEntityBufferElement bufferElement = dynamicBuffer [0] ;
                bufferElement.i ++ ; // Increment.
                dynamicBuffer [0] = bufferElement ; // Set back.
       
                // Console will throw error when using debug and burst is enabled.
                // Comment out Debug, when using burst.
                Debug.Log ( "T: " + tester.f + " IJobForEachWIthEntity " + " #" + index + "; entity: " + entity + "; " + dynamicBuffer [0].i + "; " + dynamicBuffer [1].i ) ;

            }

        }

        // [BurstCompile] // Disbaled burst, as Debug.Log is used.
        struct Job_BC : IJobForEach_BC <SomeBufferElement, Instance>
        {
            public float time;

            // Allow buffer read write in parralel jobs
            // Ensure, no two jobs can write to same entity, at the same time.
            public void Execute( DynamicBuffer <SomeBufferElement> dynamicBuffer, ref Instance tester )
            {
                tester.f = time ;


                SomeBufferElement bufferElement = dynamicBuffer [0] ;
                bufferElement.i ++ ; // Increment.
                dynamicBuffer [0] = bufferElement ; // Set back.
       
                // Console will throw error when using debug and burst is enabled.
                // Comment out Debug, when using burst.
                Debug.Log ( "T: " + tester.f + " IJobForEach_BC (Buffer, Component) " + "; "  + dynamicBuffer [0].i + "; " + dynamicBuffer [1].i ) ;

            }

        }

    }
}

Unity 2019 (Entities 0.0.12-preview 12)
Last update 2019 Jan 3
~~https://github.com/Antypodish/ECS_IJobProcessComponentDataWithEntity_IBufferElementData/blob/master/BufferWithJobSystem.cs~~

// Unity ECS very simple example with IJobProcessComponentDataWithEntity and IBufferElementData.

// Requires official Unity samples
// https://github.com/Unity-Technologies/EntityComponentSystemSamples
// Or own bootstrap.
// Just copy this script to the project
// Based on SimpleRotation / RotationSpeedSystem.cs

// 2018.11.16

// Tested with
// Entities 0.0.12-preview 12
// Burst 0.2.4-preview.37
// IncrementalCompiler 0.0.42-preview.24
// Jobs 0.0.7-Preview.5
// Mathematics 0.0.12.preview.19

using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Burst;
// using Unity.Mathematics;
// using Unity.Transforms;
using UnityEngine;

namespace ECS.Test
{

    struct Instance : IComponentData
    {
        public float f ;
        // public DynamicBuffer <int> db_a ;
    }

    struct SomeBufferElement : IBufferElementData
    {
        public int i ;
    }

    public class BufferWithJobSystem : JobComponentSystem
    {
        [BurstCompile]
        [RequireComponentTag ( typeof (SomeBufferElement) ) ]
        struct Job: IJobProcessComponentDataWithEntity <Instance>
        {
            public float dt;

            // Allow buffer read write in parralel jobs
            // Ensure, no two jobs can write to same entity, at the same time.
            // !! "You are somehow completely certain that there is no race condition possible here, because you are absolutely certain that you will not be writing to the same Entity ID multiple times from your parallel for job. (If you do thats a race condition and you can easily crash unity, overwrite memory etc) If you are indeed certain and ready to take the risks.
            // https://discussions.unity.com/t/712852/4
            [NativeDisableParallelForRestriction]
            public BufferFromEntity <SomeBufferElement> someBufferElement ;

            public void Execute( Entity entity, int index, ref Instance tester )
            {
                tester.f = 10 * dt ;

                DynamicBuffer <SomeBufferElement> someDynamicBuffer = someBufferElement [entity] ;

                SomeBufferElement buffer = someDynamicBuffer [0] ;

                // Uncomment as needed
                // buffer.i = 99 ;

                // someDynamicBuffer [0] = buffer ;

                // Debug Will throw errors in Job system
                // Debug.Log ( "#" + index + "; " + someDynamicBuffer [0].i + "; " + someDynamicBuffer [1].i ) ;

            }
        }

        protected override JobHandle OnUpdate(JobHandle inputDeps)
        {
            var job = new Job () {
                dt = Time.deltaTime,
                someBufferElement = GetBufferFromEntity <SomeBufferElement> (false)
 
                } ;
            return job.Schedule(this, inputDeps) ;
        }

        // protected override void OnCreateManager ( ) // for Entities 0.0.12 preview 20
        protected override void OnCreateManager ( int capacity )
        {
            base.OnCreateManager ( capacity );

            Instance instance = new Instance () ;

            Entity entity = EntityManager.CreateEntity ( typeof (Instance) ) ;
 
            EntityManager.SetComponentData ( entity, instance ) ;
            EntityManager.AddBuffer <SomeBufferElement> ( entity ) ;

            var bufferFromEntity = EntityManager.GetBufferFromEntity <SomeBufferElement> ();
            var buffer = bufferFromEntity [entity];

            SomeBufferElement someBufferElement = new SomeBufferElement () ;
            someBufferElement.i = 6 ;
            buffer.Add ( someBufferElement ) ;
            someBufferElement.i = 7 ;
            buffer.Add ( someBufferElement ) ;
        }
    }
}
19 Likes
GetBufferFromEntity length check.
Samples
Store an array in IComponentData
Need help to understand DynamicBuffer
What's a simple way of using DynamicBuffer in a job now?
How do you get a BufferFromEntity or ComponentDataFromEntity without Inject?
Questions about Buffer
Array in component
How to have components containing arrays in ECS? (743359)
EntityCommandBuffer and Dynamic Buffers
2

Hi Antypodish, thanks for the demo script. I have some questions.

  • // Allow buffer read write in parallel jobs. In void, there is Execute DynamicBuffer is declared. A DynamicBuffer to me is an array that can be dynamically resized. I think you mean RandomAccess Buffer? But it’s no a thing.

Some comments to the code. Perhaps you can explain.Thank you.

protected override void OnCreateManager ( int capacity )
        {

            // ... Not sure what this does in this context.
            base.OnCreateManager ( capacity );
            Instance instance = new Instance () ;

            // ... Create a single entity without world?
            Entity entity = EntityManager.CreateEntity ( typeof (Instance) ) ;
          
            // ... Set the component value of the instance.
            EntityManager.SetComponentData ( entity, instance ) ;

            // ... AddBuffer Structure to the entity?
            EntityManager.AddBuffer <SomeBufferElement> ( entity ) ;

            // ... Get base reference from Entity Arrays?
            var bufferFromEntity = EntityManager.GetBufferFromEntity <SomeBufferElement> ();

            // ... Get offset reference from bufferFromEntity( entity * sizeof(<SomeBufferElement>))
            var buffer = bufferFromEntity [entity];

            // ... Initialize a new SomeBufferElement to handle values.
            SomeBufferElement someBufferElement = new SomeBufferElement () ;

            // ... When adding a new buffer, what happen to the struct Instance : IComponentData
            // ... public float f ? Where it's used or manipulated ?
            someBufferElement.i = 6 ;
            buffer.Add ( someBufferElement ) ;
            someBufferElement.i = 7 ;
            buffer.Add ( someBufferElement ) ;

        }
How to think about setting up large number of entities
3

To be honest this is default created, when typing

protected override void OnCreateManger.

I don’t thing actually ‘base.’ is necessary in this case. It works as well without it.

Yes, that is not a problem. you can create many entities this way.

Self explanatory.

If you want dynamic buffer per entity, where you can extend size as needed, then you need assign created IBufferElementData to entity at some point.

Since we created buffer element and assigned entities to it, we can read it now.
First need get access to all entities data, which have SomeBufferElement.

And now we can access our data with entities, which hold this buffer element, using entity index.

That is irrelevant in this case.
We are not affecting IComponentData when manipulating IBufferElementData.
IComponentData is here just as an example, to better represent creation of entity.
We could equally make empty entity, or make just IComponentData as tag, without value.
I could use instance in job for example, to read / write f value, of given entity.

[quote=“Antypodish, post:1, topic: 721877, username:Antypodish”]
public void Execute( Entity entity, int index, ref Instance tester )
*[/quote]
*
But instance in this case, allows me select all entities with that component.
Example assumes, entity having Instance, has also buffer data.

1 Like
4

You should really add a requirecompenenttag to the job to ensure it only executes for entities with the buffer.

5

I haven’t used it before.

Something like that?

[RequireComponentTag ( typeof (SomeBufferElement ) ) ]
public class BufferWithJobSystem : JobComponentSystem
{ ... }
6

Add it above the IJobProcessComponentData

(in bed otherwise I would have written it up properly)

7

Oh me daft, picked wrong line.
That makes more sense. If correct.

[RequireComponentTag ( typeof (SomeBufferElement ) ) ]
struct Job: IJobProcessComponentDataWithEntity <Instance>

But don’t worry, replay tomorrow, if you can not confirm.
Then I will update scripts.

8

Yep that looks good.

1 Like
9

Do you think that would be possible to access all entites at once from the enitity manager by IBufferElementData and save or load them into a file or serialize? I use meshInstanceRender (mesh and material) and LocalToWorldMatrix with a fixed count of entities only.

I didnt find a way to serialize or save/load the content of an enititymanger at once.
Any idea about?

10

RotationSpeedRotation . Such a project dos not exist in EntityComponentSystemSamples
So I used SimpleRotation or GalacticConquest/Scripts/System.

The example does not compile for any reasons.
Assets\GameCode\Samples.Common\SimpleRotation\BufferWithJobSystem.cs(76,33): error CS0115: ‘BufferWithJobSystem.OnCreateManager(int)’: no suitable method found to override.

11

My Apology. I Haven’t precised well. I have corrected ( RotationSpeedSystem.cs ) and you found right file anyway.

You are probably using later version of Entity preview, where int in OnCreatemanger is removed.
Replace line with

protected override void OnCreateManager ( )

You can iterate through entities on main thread, if you want use EntityManager.
Then store appropriate data, in relevant collection.
Then I think best bet is use SharedComponent, via Hybrid ECS to read data in classic OOP. So you can use standard Unity API as you like.
But there may be better solution?

1 Like
12

Updated, thx

13

Unity have special utility class for serialization/deserialization worlds.
SerializeUtilityHybrid.Serialize / Deserialize
SerializeUtility.SerializeWorld / DeserializeWorld

2 Likes
14

So, is this in anyway similar to ParallelFor? With your code, I check the index, and it’s always 0. Am I missing something? There are two items in the buffer, so shouldn’t I be seeing some batching?

15

Similar in the sense, that you multithread per entity chunk, rather per index. Also you can get entity data straight away.

You should see values in buffer. Are you sure you are looking at right entity? Asking just in case, if you got other systems active.

1 Like
16

Ah, so if I just wanted to make an iteration on an “array” (i.e. buffer) on an entity parallel, this would not be the right solution for me? With ParallelFor, I was able to multi-thread on a NativeArray.

Sorry, I am trying to understand ECS a bit better.

17

It’s thread per chunk, not entity.

1 Like
18

Thank you for correction. I suppose was to vague.

19

To be honest, I never thought on parallel iterating, over the entity buffer array itself. You would be likely having let say 100k entities, for each having buffer array component, with 10k elements each.
Then each array is iterated inside the job, on its own thread, for currently loaded entity data. Therefore, I Process Component Data With Entity.

My understanding is, providing I am correct, normally you should not access same buffer array at the same time, on multiple threads, as I believe accessing same entity data, is illegal operation. This is to avoid race conditions, as far I am concerned.

1 Like
20

That makes sense, but what about the cases in which you are 100% sure you aren’t accessing the same elements? Isn’t that the whole point of the index?

For example, I have an array of shorts which act as my blocks (voxelt errain). In my case, that array is ReadOnly, so there shouldn’t even be an expectations of illegal operations (the data is then added to a different array).