Thousands spawners of thousands entities

Hi. I am learning DOTS now, and I don’t really understand much about it. To understand it much better i am trying to solve one abstract task.

I have a thousands of spawners and each of them need to spawn thousands of identical entities with different velocities on them. What is most performant way to do it?

As I know now - the most performant way to create entities is EntityManager.Instantiate(Entity, NativeArray<Entity>), but i can’t use it in jobs. Proceeding thousands of spawners in main thread is not so fast. And after i somehow create this entities, what type of Job should i use to modify velocity on created entities?

1 Like

Directly from the Docs:

General workflow should be 1) convert everything into entity prefabs 2) instantiate all in a EntityCommandBuffer 3) use IJobProcessComponentDataWithEntity or IJobChunk to set velocities on entities.

You can refer to Samples.HelloCube_06 for reference:

https://github.com/Unity-Technologies/EntityComponentSystemSamples/blob/master/Samples/Assets/HelloECS/HelloCube_06_SpawnFromEntity/HelloSpawnerSystem.cs

using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Mathematics;
using Unity.Transforms;

namespace Samples.HelloCube_06
{
    // JobComponentSystems can run on worker threads.
    // However, creating and removing Entities can only be done on the main thread to prevent race conditions.
    // The system uses an EntityCommandBuffer to defer tasks that can't be done inside the Job.
    public class HelloSpawnerSystem : JobComponentSystem
    {
        // EndSimulationBarrier is used to create a command buffer which will then be played back when that barrier system executes.
        EndSimulationEntityCommandBufferSystem m_EntityCommandBufferSystem;

        protected override void OnCreateManager()
        {
            // Cache the EndSimulationBarrier in a field, so we don't have to create it every frame
            m_EntityCommandBufferSystem = World.GetOrCreateManager<EndSimulationEntityCommandBufferSystem>();
        }

        struct SpawnJob : IJobProcessComponentDataWithEntity<HelloSpawner, LocalToWorld>
        {
            public EntityCommandBuffer CommandBuffer;

            public void Execute(Entity entity, int index, [ReadOnly] ref HelloSpawner spawner,
                [ReadOnly] ref LocalToWorld location)
            {
                for (int x = 0; x < spawner.CountX; x++)
                {
                    for (int y = 0; y < spawner.CountY; y++)
                    {
                        var instance = CommandBuffer.Instantiate(spawner.Prefab);

                        // Place the instantiated in a grid with some noise
                        var position = math.transform(location.Value,
                            new float3(x * 1.3F, noise.cnoise(new float2(x, y) * 0.21F) * 2, y * 1.3F));
                        CommandBuffer.SetComponent(instance, new Translation {Value = position});
                    }
                }

                CommandBuffer.DestroyEntity(entity);
            }
        }

        protected override JobHandle OnUpdate(JobHandle inputDeps)
        {
            //Instead of performing structural changes directly, a Job can add a command to an EntityCommandBuffer to perform such changes on the main thread after the Job has finished.
            //Command buffers allow you to perform any, potentially costly, calculations on a worker thread, while queuing up the actual insertions and deletions for later.

            // Schedule the job that will add Instantiate commands to the EntityCommandBuffer.
            var job = new SpawnJob
            {
                CommandBuffer = m_EntityCommandBufferSystem.CreateCommandBuffer()
            }.ScheduleSingle(this, inputDeps);


            // SpawnJob runs in parallel with no sync point until the barrier system executes.
            // When the barrier system executes we want to complete the SpawnJob and then play back the commands (Creating the entities and placing them).
            // We need to tell the barrier system which job it needs to complete before it can play back the commands.
            m_EntityCommandBufferSystem.AddJobHandleForProducer(job);

            return job;
        }
    }
}

No, don’t use EntityCommandBuffer! It’s much slower than batch api!

For hugh amount of entities use a subscene or a different world to spawn these.
Always use batch api for that. It’s super fast.

So 1000 spawners with 1000 entities with random velocities sounds more like a particle system than a game one, surely this would be better as a GPU system than CPU one.

Or at least I cannot imagine a game that needs 1,000,000 units that would be controllable by humans?

Well I can already think of a couple of solid game ideas just from the phrase “1.000.000 player controllable agents”.

2 Likes

EntityCommandBuffer is much slower, than EntityManager.Instantiate(Entity, NativeArray<Entity>).

Subscenes are predefine set of entities, not created at runtime. And about another world - how do you recomend using it?

GPU is not a variant, because this task is for learn purposes, not for actual gameplay.

I think this steps are reuired:

  1. Convert everything into entity prefabs.
  2. With some Job go through all spawners just to calculate sum of entities to spawn.
  3. In main thread create entities with EntityManager
  4. With other Job go through all created Entites to set velocities.

But here is two problems:

  1. I don’t know how to calculate sum with Job without writing to one Sum variable.
  2. There will be sync point between first and second Job, because I need to create entities in main thread.

Mass online bullet hell anyone?

You can create a Native Array of ints sized to your spawner count, or add a spawn count component to each spawner. Then a parallel job (IJPCDWE or IJPCD respectively) counts the number to spawn for each spawner. Then you run a single IJob or IJPCD.ScheduleSingle to generate the final sum.

But what I think you really want is to use the ExclusiveEntityTransaction API and have a spawning world per thread. This lets you actually generate your entities in parallel. You can use ToComponentDataArray to get the data out of your simulation world into the spawning worlds and then schedule a bunch of jobs using slices. Then you just move the entities from the spawning worlds to the simulation world.

And if you really want to squeeze the system, before you move entities from the spawning worlds, you can kick off another set of spawning worlds and jobs that run on the worker threads while the main thread does the world merging. That means that if you can calculate the spawns for frame n+1, you can truly prevent a main thread bottleneck.

How come I have read that ops on EntityCommandBuffer are more optimized than to use EntityManager directly? It doesn’t apply to entity creation?

EntityCommandBuffer works fine for me for thousands of spawners. But if you are trying to actually create millions of entities in a few frames, then you need another approach.

For loading from another world, you need to use ExclusiveEntityTransaction. Doc says:

https://docs.unity3d.com/Packages/com.unity.entities@0.0/manual/exclusive_entity_transaction.html?q=ExclusiveEntityTransaction

1 Like

That’s true if you are building entities one by one or are doing some sort of conditional spawning as there’s an overhead for each call into EntityManager. But when you only make one call to instantiate 1000s of entities based on a single prefab, well that’s an easy code path for Unity to optimize.

Yes you are right. I’ve always thought that Command buffer playback should have similar performance with the batch api (if not more optimized). I guess it’s not the case (yet)?

1 Like

Oh, new feature I didn’t heard of (ExclusiveEntityTransaction). Idea with world per job is really interesting, I will try it.

Can you also attach link, for quick access doc reference? Thx.

Sure thing. I actually couldn’t find it online at the time since they moved the old doc to Packages Doc, so I copy&pasted from local source. But I just found it now:

https://docs.unity3d.com/Packages/com.unity.entities@0.0/manual/exclusive_entity_transaction.html?q=ExclusiveEntityTransaction

1 Like

Any improvements to this? I would prefer just a EntityCommandBuffer with Instantiation of a native list…? Unity? Wondering if it’s in the works, or we have to use this roundabout method for mass instantiation!

I wrote a tool that does this. It pretty much always beats out ECB for large numbers of entities. https://github.com/Dreaming381/Latios-Framework/blob/v0.3.3/Documentation~/Core/Custom%20Command%20Buffers%20and%20SyncPointPlaybackSystem.md#instantiatecommandbuffer

1 Like

Thank you, checking it out! Really cool work :open_mouth: