Flagging for filtering: Better to add/remove empty component or poll over component data int.

Hello again, I’ve got an architectural question of sorts

With some experience with ECS outside Unity, I’ve seen this pattern: add component to flag an entity so some system runs on it. Sometimes they’re just a bitmask (no data). But I’ve seen them probably on a different memory structuring version of ECS.

I gather from the tutorials about the way I think chunking might work, changing structure of several entites might start some memory reorganization process that’s eventually slower than just polling through hundreds of thousands and and-ing to check some flag against a small integer from component data (Even though I guess some small penalty from branch mispredictions might arise as there is no way to define sort order on them)

I’ve tried it, deferring additions and removals with a barrier concurrent-able buffer, I get framerate spikes on the given barriers (while running the commands, not on the jobs adding them).

Have any of you profiled this? Is there a turning point by amount for when it’s better to add-component or set-data-flag?
Is there another way to handle this that I’m not aware of? When I have more time I’ll do the testing myself but maybe there’s a better way to handle this and I can avoid investing my time on that.

Use case: Only sync/mirror data to classic game objects when really necessary (only when changed)

Thanks!

I’ve done some tests of a similar issue and found that creating an event as a separate entity is similar in performance to iterating over persistent entities where 0.1% of queried entities have that event. And if that even is true in 1+% of cases, then it is definitely more performant to iterate through all entities and check than to have it as a separate new event entity. But take it with a grain of salt, my benchmark was a bit messy.

Haven’t tested adding component vs checking a value in a component. Would like to see the results or test it at some point.

And yes, the barrier’s entityCommandBuffer will playback on the main thread so you’ll get the spikes.

My janky benchmark in pictures:
4006540--345676--upload_2018-12-17_14-28-30.png

4006540--345685--upload_2018-12-17_14-41-48.png

  1. Setting and checking a 1000 components
  2. Adding a 1000 components
  3. Adding 10 components
  4. Adding 100 components

I don’t think there needs to be a bigger scale benchmark since it becomes nonviable for me at less than 1000 components per second. Interestingly a 1000 and a 100 are almost identical.

A bigger benchmark is needed for cases where less than 1% of entities get flagged.

So, a tag component filters might be good for some very occasional events, but in other cases, it is better to have a persistent component and to poll over them.

4 Likes

  1. Adding and removing 1000 entities
  2. Iterating over 1,000,000 entities

  1. Adding and removing 1000 entities
  2. Iterating over 10,000,000 entities

  1. Spawining 100,000,000 entities
struct TestComponent : IComponentData { public int Active; }
struct TestComponent2 : IComponentData { }
struct TestComponent3 : IComponentData { }

public class IterateSystem : JobComponentSystem
{
    override protected void OnCreateManager()
    {
        for (int i = 0; i < 10000000; i++)
        {
            EntityManager.CreateEntity(ComponentType.Create<TestComponent>());
        }
    }

    [BurstCompile]
    struct Job : IJobProcessComponentData<TestComponent>
    {
        public void Execute(ref TestComponent c)
        {
            if (c.Active != 0)
            {
            }
        }
    }

    protected override JobHandle OnUpdate(JobHandle inputDeps)
    {
        var job = new Job { };
        return job.Schedule(this, inputDeps);
    }
}

public class AddComponentSystem : JobComponentSystem
{
    [UpdateAfterAttribute(typeof(AddComponentSystem))]
    class Barrier : BarrierSystem { }
    [Inject] Barrier b;
    [Inject] EndFrameBarrier efb;

    override protected void OnCreateManager()
    {
        for (int i = 0; i < 1000; i++)
        {
            EntityManager.CreateEntity(ComponentType.Create<TestComponent2>());
        }
    }

    struct Job : IJobProcessComponentDataWithEntity<TestComponent2>
    {
        public EntityCommandBuffer.Concurrent ecb;
        public EntityCommandBuffer.Concurrent eecb;
        public void Execute(Entity e, int i, ref TestComponent2 c)
        {
            ecb.AddComponent<TestComponent3>(i, e, new TestComponent3());
            eecb.RemoveComponent<TestComponent3>(i, e);
        }
    }

    protected override JobHandle OnUpdate(JobHandle inputDeps)
    {
        var job = new Job { ecb = b.CreateCommandBuffer().ToConcurrent(), eecb = efb.CreateCommandBuffer().ToConcurrent() };
        return job.Schedule(this, inputDeps);
    }
}
4 Likes

This basically means that on 4 core CPU currently if I have an object that is receiving an event less often than every 10,000 - 20,000 frames, it is probably more efficient to iterate than to flag with a component.

Removing takes twice as long as adding a tag component. So if you change state only one way the numbers are different.

For me, it means that most of the things in my game will have persistent components because the events on them generally happen more often than those 100 - 200 seconds. This also makes performance less spiky.

3 Likes

It was brought to my attention that creating separate entities might be faster than tagging existing.

  1. Well, that’s creating and destroying a 1000 entities. Can’t find batching API. Using archetype.

Also, I checked the difference between tagging an entity with one or 7 components already on it. No difference found. EDIT: There actually is a slight difference between tagging an entity with less or more data.

6 Likes

These might be the most amazing replies I’ve ever got. Thanks for your time and your info!
Also thanks for profiling adding/removing too.

I haven’t thought of entities with just the one component, too bad they don’t work either. I’d probably need more than 1% so I’ll go for polling. Maybe I’ll just need to have a persistent NativeArray, too bad I can’t write trivially in parallel .

Maybe the overhead comes from chunk allocation, even for a single-component archetype?
If I understand correctly, chunk size / int32 component index + entity id = 16kB / 8 = 2048.
That could explain having similar performance up to two thousand entities.
Also found:

        public const int kChunkSize = 16 * 1024 - 256; // allocate a bit less to allow for header overhead
        public const int kMaximumEntitiesPerChunk = kChunkSize / 8;

Reading this I found why it’s a terrible idea to add/remove, because it changes it’s archetype and moves all component data to a different chunk!
https://discussions.unity.com/t/702049/4

:smile:

No problem, I needed this info myself :slight_smile:

That was also a very dirty benchmark, actually my game code, a bunch of overhead.

Yes. Also, I’m gonna add an edit. There actually is a slight difference between tagging an entity with less or more data.

1 Like

By the way… In the light of all the above, Entity pooling makes sense when we need to create a lot of entities each frame or save some performance in general.

Also, I missed something in the benchmark, the system that iterates over millions of entities should also SetComponent<>() for a 1000 entities to be fair.

True! Also try add/rem of SharedComponentData as tag, because it’s saved once per chunk. For tagging it would probably just keep only two sets of chunks and move between them, maybe?
I’ll get on it when I get back.

The important update:

  1. Setting a 1000 components 2 times and Iterating over 10,000,000 components
  2. Tagging entities with data by adding and removing a 1000 tag entities

This is the more fair comparison between tagging and polling.

So even when events occur very rarely 1/10000 the largest cost comes from setting a component, not from polling components. If they occur 1/100 the polling cost is negligible, but setting a component is a cost to keep in mind although it is significantly lower than the tagging cost.

I don’t think tagging with an ISharedComponentData is viable. It still gonna move each entity individually to a new chunk.

2 Likes

I was wondering about this too. Thanks for the analysis. Looks like setting and polling is the way to go.

1 Like

You can greatly improve performance of creating and destroying entities by doing them in a batch instead of creating them 1 by 1.

EntityManager.CreateEntity(EntityArchetype, NativeArray<Entity>)

Can see my experimentation here (and my final solution in the last comment that batches entity event creation): Batch EntityCommandBuffer

Result Summary:

I got more than a 10x speedup over using EntityCommandBuffer CreateEntity/SetComponentData.
Creating, set component data (and destroying previous frame) 100,000 entities in 3.4ms (~300fps)

4 Likes

Wow… Pretty amazing results. Do I even need to tell which is which? They are almost identical.

4010839--346270--upload_2018-12-18_12-40-35.png

Blue - Setting a 1000 components twice on the main, and iterate over a 10,000,000 in a job.
Red - Creating and destroying a 1000 event entities on the main, and use them via ComponentDataFromEntity in a job. Number of existing target entities doesn’t matter, so imagine it’s also 10,000,000

This makes a separate entity based event system very much viable, which is great since it might be a better design. Except the whole create the entity, connect to the target and then get data from its target process requires a lot of extra steps and buffering when you want to use batching. If API gets better at it, this could be a great way though.

Also, need to note that it is only that fast for the entities that are living only one frame because they are deleted by archetype all at once.

EDIT: I forgot burst while using the event entities, and with burst, it almost disappears from the worker thread, so the batched event entities are actually faster in the benchmark.

4010839--346282--upload_2018-12-18_13-14-56.png

The code.

using Unity.Entities;
using Unity.Burst;
using Unity.Jobs;
using Unity.Collections;
using Unity.Transforms;
using Unity.Mathematics;
using UnityEngine;

struct TestIterationComponent : IComponentData { public int Value; }
struct TestEventComponent : IComponentData { public Entity Target; }
struct TestTargetComponent : IComponentData { public int i; }

[UpdateBeforeAttribute(typeof(CreateEntitySystem))]
public class IterateSystem : JobComponentSystem
{
    struct Targets
    {
        public ComponentDataArray<TestIterationComponent> Component;
        public EntityArray Entities;
    }
    [Inject] Targets components;

    override protected void OnCreateManager()
    {
        for (int i = 0; i < 1000000; i++)
        {
            EntityManager.CreateEntity(ComponentType.Create<TestIterationComponent>());
        }
    }

    [BurstCompile]
    struct Job : IJobProcessComponentData<TestIterationComponent>
    {
        public void Execute(ref TestIterationComponent c)
        {
            if (c.Value != 0)
            {
            }
        }
    }

    protected override JobHandle OnUpdate(JobHandle inputDeps)
    {
        for (int i = 0; i < 2000; i++)
        {
            EntityManager.SetComponentData<TestIterationComponent>(components.Entities[i], new TestIterationComponent { Value = 1 });
        }
        var job = new Job { };
        return job.Schedule(this, inputDeps);
    }
}

public class CreateEntitySystem : ComponentSystem
{
    EntityArchetype eventArvhetype;
    struct Targets
    {
        public ComponentDataArray<TestTargetComponent> T;
        public EntityArray Entities;
    }
    [Inject] Targets targets;

    override protected void OnCreateManager()
    {
        eventArvhetype = EntityManager.CreateArchetype(typeof(TestEventComponent));

        var newTargets = new NativeArray<Entity>(1000000, Allocator.Temp);
        EntityManager.CreateEntity(EntityManager.CreateArchetype(typeof(TestTargetComponent)), newTargets);
        for (int i = 0; i < 1000000; i++)
        {
            EntityManager.SetComponentData<TestTargetComponent>(newTargets[i], new TestTargetComponent { i = 1 });
        }
        newTargets.Dispose();
    }

    protected override void OnUpdate()
    {
        var targetEntities = new NativeArray<Entity>(1000, Allocator.Temp);
        for (int i = 0; i < 1000; i++)
        {
            targetEntities[i] = targets.Entities[i];
        }
        var events = new NativeArray<Entity>(1000, Allocator.Temp);
        EntityManager.CreateEntity(eventArvhetype, events);
        for (int i = 0; i < 1000; i++)
        {
            EntityManager.SetComponentData<TestEventComponent>(events[i], new TestEventComponent { Target = targetEntities[i] });
        }
        events.Dispose();
        targetEntities.Dispose();
    }
}

[UpdateAfter(typeof(CreateEntitySystem))]
public class UseEntitiesSystem : JobComponentSystem
{
    [Inject] ComponentDataFromEntity<TestTargetComponent> componentDataFromEntity;
    ComponentGroup g;

    override protected void OnCreateManager()
    {
        g = EntityManager.CreateComponentGroup(typeof(TestEventComponent));
    }

    [BurstCompile]
    struct Job : IJobProcessComponentDataWithEntity<TestEventComponent>
    {
        [ReadOnly] public ComponentDataFromEntity<TestTargetComponent> componentDataFromEntity;

        public void Execute(Entity e, int i, ref TestEventComponent c)
        {
            var d = componentDataFromEntity[c.Target];
            if (d.i == 0)
            {
                d.i = 1;
            }
            else
            {
                d.i = 0;
            }
        }
    }

    protected override JobHandle OnUpdate(JobHandle inputDeps)
    {
        var job = new Job
        {
            componentDataFromEntity = componentDataFromEntity
        };
        return job.Schedule(this, inputDeps);
    }
}

[UpdateAfter(typeof(UseEntitiesSystem))]
public class DestroyEntitySystem : ComponentSystem
{
    ComponentGroup g;

    override protected void OnCreateManager()
    {
        g = EntityManager.CreateComponentGroup(typeof(TestEventComponent));
    }

    protected override void OnUpdate()
    {
        EntityManager.DestroyEntity(g);
    }
}

Now I’m gonna try to find a way to make event entities pretty and usable in the code. For state switching setting components is still better of course. For things like JustSpawned, IncomingDamage, GameOver, the event entities are the best. Especially Damage in this case since you can add as many instances as you wish.

The difference is more in favor of event entities when there are more of them.

  • Setting 10,000 and iterating costs 9ms because we are setting twice
  • Create>Set>Destroy 1.5ms
    Actually doesnt make sense, I would expect 9 and 4.5 ms at least.
2 Likes

Best thread ever! Thank you very much for benchmarks, very insightful. I was wondering about the SharedComponent as well but to use it in a slightly different manner. Usually whenever I need an event that some system will receive I create a separate Entity on that system. Like this:

    public class SpawnEnemiesSystem : JobComponentSystem
    {
        public struct SpawnEnemiesEvent : IComponentData { }

        public           Entity          SpawnEnemiesEventEntity;
        [Inject] private EndFrameBarrier _endFrameBarrier;

        protected override void OnCreateManager()
        {
            SpawnEnemiesEventEntity = EntityManager.CreateEntity(typeof(LevelDataComponent));
        }

        protected override JobHandle OnUpdate(JobHandle inputDeps)
        {
            return new SpawnEnemiesJob
            {
                CommandBuffer = _endFrameBarrier.CreateCommandBuffer().ToConcurrent()
            }.Schedule(this, inputDeps);
        }

        private struct SpawnEnemiesJob : IJobProcessComponentDataWithEntity<SpawnEnemiesEvent, LevelDataComponent>
        {
            public EntityCommandBuffer.Concurrent CommandBuffer;

            public void Execute(Entity                 entity,
                                int                    index,
                                ref SpawnEnemiesEvent  eventComponent,
                                ref LevelDataComponent levelData)
            {
                // DO WHATEVER YOU NEED TO ON THIS EVENT
              
                //Remove event
                CommandBuffer.RemoveComponent<SpawnEnemiesEvent>(index, entity);
            }
        }
    }

If I need to trigger this event I just [Inject] this system into other system, get an entity and add SpawnEnemiesEvent to it. As was mentioned above using SharedComponent causes every entity to use the same Chunk regardless of it’s Archetype. So maybe using SharedComponent not as marker that event occurred but rather as a marker that entities which receive events shouldn’t create new Archetype on adding a Component would work. Something like this:

        public struct EventSharedComponent : ISharedComponentData { }

Now just set this SharedComponent to every entity that will have marker components added to it and and they shouldn’t create new Archetype when the marker component is actually added.

I’ll try to create some benchmarks this week if I manage to find some time

Absolutley incorrect. SCD divide entities by chunks by SCD value inside archetype. For example you have A,B,C - IComponentData and D - ISharedComponentData.
You have 3 archetypes:
[A,D]
[B,D]
[C,D]
in all cases, lets say, D has in field Value with value = 0
And, lets say, 3 chunks inside every archetype:
[A,D] - [C1, C2, C3]
[B,D] - [C1, C2, C3]
[C,D] - [C1, C2, C3]
Then we change D Value to “1” for two entities inside C3 chunk of CD archetype, and we see:
[A,D] - [C1, C2, C3]
[B,D] - [C1, C2, C3]
[C,D] - [C1, C2, C3, C4]
Where in C4 we see thes two entities grouped by other SCD value.

SharedComponentData is not forcing entities into the same chunks. If you have 10 chunks of entities and then you add Same shared component data to all entities and change value of a half of SCDs, you won’t connect chunks into two chunks based on SCD values, you will fragment the chunks into 20 chunks. (given that each chunk has both SCD values)

So if every entity has the same value of a SCD they all stay in the same chunk no matter what other components do they have added to them, correct? As I written above SCD would only be used to force the entites into the same chunk so they don’t create a new one if we add a marker component to any of them. I might be wrong though

No. They will be in chunks corresponding to their archetypes.

As eizenhorn said, no. Every entity archetype would have its own chunk. They will only be in the same chunk if both the value of the scd and the archetype are the same.

Scd causes significantly more chunks and should be avoided where possible.

1 Like