DOTS skill system repo available

Unity Engine
,
1

Edit - repo available here GitHub - WAYN-Games/MGM-Ability

edit - Changed title to better reflect most of the content of the thread.

Hi,

I have an initializaition system that need to get data from an IComponentData class to add/set IComponentData struct based on its content before removing the IComponentData class from the entity.

I can make the code run fine on the main thread my question is, I know I’m not working on conflicting entities (no race condition) and appart from the conversion system, it’s the only system that write to these component, so how can I go about offloading the work to a job, even better if possible have it multithreaded ?

When testing with 20K entities I get from 400 to 900 ms executions wich I’d like to reduce as much as possible, even if it’s an intialization system that should only run once.

I also managed to make this work with c# Tasks but I don’t know about the compatibility of this method over the diferent platforms.

If none of this works, I may also be able to spread the initialization over several frames by limiting the number of entities processed per frame so that it don’t freeze the game.

Any suggestions ?

2 Likes
2

While Burst doesn’t work with managed objects, Jobs does but requires some extra workaround to be able to handle managed objects. Here is a good starting point: Run managed code in Unity’s Job System – COFFEE BRAIN GAMES

3

does that imply I will be making one job per entity ?
If so, I’m worried about the shob scheduling overhead miking it even less efficient…

4

I don’t see why it would imply that as it would work the same as it would with C# Tasks, but using the Job System instead.

5

Sorry I thought the method you proposed would allow me to multithread the process but it just allow to offlaod it to a background thread.

I’m investigation manual chunk iteration with the help of GetComponentObjects to get the IComponentData class.

For now it does not trigger any safetycheck but I don’t do anything either in my job…

If this work it would allow to process each chunk in parallel.

If this work out I’ll post a sample code in this thread.

Also I need to figure aout a way to get a viable index for using in the concurrent ECB i’ll need in hte IJob. :confused:

6

I would strongly advise against any of this. If you want performance. Then using struct IComponentData and use the builtin functionality to make it fast.

Ultimately multithreading is only a small part of getting good performance. Memory layout & codegen is equally important.

Class components exist for handling a couple of entities in the game where you are absolutely sure that performance is not important.

I’d love to know what the specific scenario is that makes doing that impossible for to use struct IComponentData and friends?

1 Like
7

Hi @Joachim_Ante_1 , thanks for your interest in my post,solving the root problem would be great but for now I just worked arround several issues to make sure that my goals was at least doable. (which they seem to be :smile:)

I’ll try to put something together to explain it as well a possible (hat’s gona be a long post :)) and ping you again once it’s done.

8

@Joachim_Ante_1 ,here is the scenario in which I need the IComponentData Class. Basically it boils down to a compatibility issue with subscene and the use of hash code (that could be worked around).

This is explained to the best of my languages ability and hope it makes sense (it’s almost a draft documentation of the thing…)

I’m not yet ready to call that a release so for know let’s consider it as an alpha preview .

What is the goal ?

Make a designer friendly skill system

What is a skill ?

The skill is an ability that produce effects on one or several entities. The effect can for example be from reducing a pool (HP/Mana/Bullet magazine), moving an entity (apply a push back, teleport,…), triggering a message.

Skill have some properties in common (ex : a range, a cost, …) but each have a unique set of effects.

Some skills can however share the same effects.

The effect can be applied either to the target or « caster » of the skill.

As I said an effect can be a wide range of actions. An the action it has on entities can depend on other factor to the entities itself (damage can be reduced by armor based on the damage type,…)

How to convey such effect in DOTS?

To covey the effect on the different entities we can use a flavor of an event system.

We could for instance add a IComponentData to the affected entities where the IComponentData have a the necessary data to apply the effect in a system that iterate aver all entities with that component (and the ones needed for the effect to apply like armor).

The drawback is that this implementation trigger a change in the archetype every time we want to apply an effect.

To work around that we can implement a custom event system with native containers (native stream) and have a system that consume the stream to apply the effect to the necessary entities.

This work great when we know the effect we want to apply when codding the actual game logic, but that’s not designer friendly.

When thinking about it a skill effect should not change at runtime it’s defined by the disigner when building the game and don’t change once the game run (it’s just applied).

So my idea is to make a registry based event system.

What is a registry based event system ?

Like the name should imply it’s an event system that uses a registry of possible events.

The idea is to let the designer define the set of effect a skill should have through the editor with some custom editors. That way we can author the behavior with the handy abstractions of OOP while having a (hopefully) performant runtime.

How does it work ?

This process work in several phases :

The authoring phase

The authoring phase consist in defining the list of effects that a skill should have through the unity editor.

The conversion phase

The conversion phase add this list of effect in an IComponentData class to be consumed by the next phase.

The initialization phase

The initialization phase iterate over all the entities with the IComponentData class added at the conversion phase. This IComponentData class contains the list of effect (IEffect) to be added to the skill, it adds it to a registry (a singleton class containing a dictionary of dictionary). The registry returns a reference to the effect (the reference is composed of a hash of the effect type and a hash of the effect itself). The registration is unique so if I try to register 2 identical effect, there is only one instance in the registry and when trying to add the second one the registry returns the same reference.

The effect reference is then added to a dynamic buffer of effect reference.

Doing that allow the next phase to trigger effect without knowing what it actually is.

The work done by the initialization phase cannot be done at convert time because it would not work for entities having skill in subscene (the registry is a simple singleton so it’s not persisted in the serialized subscene and the effect reference relies on the hash which would be different at runtime because not computed by the same process) That’s why I need the IComponentData Class.

The production phase

During this phase, any system that need to trigger effect grab a native queue from the dispatcher system and enqueue an effect command containing the emitter entity, the target entity and the reference of the effect.

The dispatcher phase

The dispatcher loop through the list of native queue requested by the producer systems and dispatch the effect in a native multi hashmap (command map) based on the effect type id (contained in the effect reference)

The consumer Phase

In the consumer phase a IJob per effect type gets the list of effect to apply from the dispatcher (command map). It take all the necessary component data for the application of the effect and apply it.

That way, the effect can be handle in parallel (careful about the order so that 2 consumer don’t alter the same component).

End notes

For now I don’t have the skill system per say, it’s just the effect that I apply based on the collisions between entities. (each skill will probably be an entity that has a buffer of effect and that is triggered by some system) (skill themselves will likely be attached to a « skill bar » buffer of the player entity)

The effect system POC work in my playground tank game. (implementation in this sample is « full » of bad code)

Now I’m trying to extract the registry based event to a package with proper unit and performance tests, to improve both the « cleanness » of the code an performance.

Tank demo playground :

1 Like
9

Fair enough. So to summarize. You want some sort of a state machine / event system. Right now you think that changing state via AddComponent / RemoveComponent is too expensive, because it moves archetypes.

We have found a similar problem in Dots shooter. But it really depends on the frequency of the addcomponent / removecomponent calls, is a perf issue or not.

That is why we are currently working on adding Enabled state to all components. This way you can enable / disable components from any job safely without changing the structure. Entities.ForEach will automatically do the filtering for you. It will process only the entities that match essentially a disabled component appears as a component that doesn’t exist.

Internally this is done via packed bitfields across multiple entities and SIMD bit mask tricks so it will be quite efficient.

In Dots shooter our plan is then that during conversion you setup all possible components (Enabled or disabled) and at runtime you simply enable / disable components. Our game code team also prefers this approach since it will be clearer to have all possible components already present. So its easier to identify a specific type of object in the world…

That sounds like it should solve your problem…

So the question what do you do for now, until this feature lands.

In DOTS Shooter we are simply pretending the functionality exists and are putting a bool Enabled; in each of the Ability System components. And then we do the filtering manually in Entities.ForEach in the first line of code.
It’s not optimal but it gets us to a place where we can structure our game code how we want it to be and once native support for enabled / disabled components lands we can simply remove all of the .Enabled bools and let ecs take care of the filtering for us.

An alternative is to simply live with the cost of add/remove component even for high frequency changes for the time being, until we land native support for Enabled flags.

34 Likes
10

Thanks for the reply @Joachim_Ante_1 .

That’s a cool concept.

Will it also work with dynamic buffers (or are considered not there the empty ones maybe) ? if so the skill system could be something like this :

The skill is an entity with an IComponentData for each of its effect.

Target gets a DynamicBuffer of IBufferElement per effect type to act as a queue of effect to apply to the entity (this is to allow on target to receive several of the same effect in one tick/frame, most likely possible in a MMORPG ), when adding an element to the buffer we add/activate it.

Then we have one system per DynamicBuffer type (run only on enabled ones) that loop through the buffer and apply the effect to the necessary component data on the entity, at the end, the buffer is cleared and disabled.

The authoring is easy to manage with skill entities (prefab) (target get the buffer on the first occurrence of the effect and then keep it disable). this will most likely work well with live link.

This would still use the ECB to alter the effect buffer content on the target entity in a concurrent way.

Until the feature land (any ETA ?:p), I’ll try it in a similar fashion that you mention in the DOTS sample to see if it works.

11

Agree. This approach works well with authoring & live link.

I expect this to land somewhere between 3weeks to 3 months.
It’s unclear mostly because it just really changes a massive amount of assumptions in the internals of entities that it is hard to say exactly how long it takes to ensure all of them are handled robustly and as expected.

1 Like
12

Is this concept of “enable/disable” components also used to replicate the components attached to a ghost in netcode? So for example you can enable a “Buff” component on a ghost on the server and it gets automatically added to the ghost on the client?

13

I’ve done a couple of effect systems in production and a few more just refining the approach.

What you want is a concept of effects attach to other logical entities. And then you tick the effects. Some will just tick once, some get a lot more involved, the pattern holds up well over almost any genre and handles almost any type of once and over time logic you can dream up.

Then you have what your affects do. This should be reduced down to a specific singular action, like add health/remove health, increase magic recovery, decrease magic recovery, etc… An effect does one thing and one thing only. So effects are NOT player level abstractions. Skills, spells, whatever your higher level abstraction is, will have a list of effects that get created when they trigger.

How you resolve effects is multiple steps. I start with an effect context abstraction. That gets created when you trigger an effect. It has the effect lifetime state info and the entity it’s attached to at a minimum.

On your effect tick interval you first iterate all of your effect context’s and group them by effect type. Then those groups are handled by effect specific systems (not necessarily one system per effect type). You don’t have to do this but it solves a bunch of issues in one go. It makes it easy to have effect specific systems, you created a nice linear list of data for each of them to operate on. Your effect ticking should be abstracted out from what your effects actually do. And you often find you want effects processed in order of the effect type, which in itself requires this type of transformation.

We solve this outside of the context of Unity so I’m thinking off the top of my head what tools would work best in an ECS context. Effect types definitely not a component but an enum. You need to key off of that in a lot of context’s that are more granular.

For effect context’s you could likely use entities just fine. Effects are generally not something you create so many of per frame that it’s going to be an issue. Note that iterating and grouping every tick is important. The groups can potentially change every tick for reasons other then effect context’s coming and going. So your effect context’s need to persist. Ie you can’t for example just use a NativeQueue for your effect context’s.

How exactly you group the entities to act on by effect type could go several ways. I would most likely use DynamicBuffer I think. An entity per effect type each with a DynamicBuffer holding the entities that effect should act on. Practically speaking effects most often modify similar things like stats. And you would generally have core stats grouped in one or two components at most. It’s going to pan out like that for other stuff as well. In any case you never need a system or job per effect type you can consolidate that a great deal. Even if you had a lot of jobs each acting on one effect type it’s easy to early out here. Your entity has an EffectType enum so jobs can early out on that.

14

Agree if you really just trying to model state. Using enums and just processing everything every frame is a perfectly reasonable approach. With burst you can blast through an array of ints so fast doing it for 10k ints is just only barely going to show up if you have the right early outs.

2 Likes
15

@snacktime I feel like what you describe is close to what I’m doing in my current implementation.

My level of abstractions are Skill and effect.

Yes each effect only affect one component data (hp/mp) or have a single action (pushback,…)

That’s the production phase where I create the effect command with the effect that should be applied and the context (emitter/target)

That’s the dispatcher

That are the consumer that consume only one effect type.

I did not get to implement the effect over time or effect over area but for effect over time I think I can span an entity with the timing and ticking logic. And for the effect over area I can either make a special effect that cast a collider and register the effect like any other producer or have each effect cast their own collider and act on all returned entities (probably less performant).

Technically I’m outside of “unity” (or ECS at least) because I rely on native containers, and a registry, not on entities.

Are you talking about creating an effect (authoring) or applying it to the target entity (produce/group/consume)

The produce/group/consume all happen in the same tick through dependencies. (for now the consuming can overlap to the next frame but I can easily constrain it to the end of the simulation group.

Have you any metrics on how many effect can be applied per tick ? I go with a wild estimation of a worst case scenario where in a MMO (like WOW) you have 1000 player all fighting the same both. If by some miracle all the player activate a skill at the same tick, I’ll get 1000 effect (reduce hp) on the mob and likely 1 effect (reduce mp) per player. And that don’t take into account the effect over area or other potential effects… in my system that lead to 2 consumer thread each iterating 1000 times. (I’d like to improve the dispatch logic to be able to handle entities in parallel)

[quote]
So your effect context’s need to persist. Ie you can’t for example just use a NativeQueue for your effect context’s.
[/quote] why would they need to persist ? once the command (context + effect) are consumed I don’t need them. Or are you talking about the registry of all possible effects ?

That’s the dispatcher’s job. And with one system er effect type, I don’t run the job when there is no effect of the type to consume.

All in all, I think that my implementation is conceptually not that bad. I just need to make sure it performs well for a realistic work load (not easy to determine what is realistic with my little experience so tyr to mai for mig unreasonable number :p).

As for the approach I gave regarding the use of the enable/disable component feature, I don’t think it will work. I still end up with some of the pitfall I thought about at the beginning.

Using the skill is an entity with several effect component I author it easily but at coding, I have no idea how much and which effects a skill have. so I need to treat each effect separately. I could have a system that activate the skill component tag and then treat every effect that are linked to an active skill component tag, do the dispatching of the effect and force it to finish before a last system that disable the skill component tag.

Now dispatching the effect to a buffer on the targeted entity, that seems great because I can still have one system per effect type (nice granularity) and I get to handle each entity in parallel safely because the effects are linked to only one entity for each buffer. The issue here is that it does not seem there is a way to write safely to a dynamic buffer in a concurrent manner. So I can’t have several entities applying the same effect to the same target at the same time.

1 Like
16

I’m now trying to go with the skill entity with each component data as an effect managed like I discribed in the previous post :

As I said, the dynamic buffers for the target entiteis can’t be writen to in a thread safe maner (as far as I know).

So I’d like to write to a native container instead.

Since each effect is produced only by one system, I can have a native stream per consumer taht is populated by each corresponding producer. That way there is no issue dispatching effect by type since it’s directly done by the producer.
The writing also happen in parallel (one write index per chunk).

Then when the consumer is swhedule I use an entity for each statement taht match the component data that need to me modified (or taken into acount) by the consumption of the effect. The native stream is converted to a native array before the job so that in the job fore each potential targeted entity , I can iterrate over all effects and apply it when necessary (therest of the time I early out).

From my understanding, this make the best used of th data layout, the consumer relies on Entities.ForEach, the producer uses IJobChunk (Entities.ForEach don’t support generic) and the native stream is converted (copied) to an array so the memory loyout for that should be linear.

I did a first test implementation and it seems to work. I’ll try to perf test this and provide my result in this thread after that.

Here is the implmentation I get for now :
Test Implementation

using NUnit.Framework;

using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;

namespace WaynGroup.MgmEvent.Tests
{

    class EffectSystemTest : DotsTest
    {
        #region Core features

        #region Components
        // Skill tag, Enabled will be replaced by the feature Joachim_Ante talked about
        public struct Skill : IComponentData
        {
            public bool Enabled;
        }

        public struct Target : IComponentData
        {
            public Entity Value;
        }

        #endregion

        #region System
        [DisableAutoCreation]
        public class SkillDeactivationSystem : SystemBase
        {
            protected override void OnUpdate()
            {
                Dependency = Entities.ForEach((ref Skill skill) =>
                {
                    Skill s = skill;
                    s.Enabled = false;
                    skill = s;
                }).ScheduleParallel(Dependency);
            }
        }
        #endregion

        #region Abstractions
        public interface IEffect : IComponentData { }

        public struct ContextualizedEffect<EFFECT> where EFFECT : struct, IEffect
        {
            // Could add the emitter entity to get the needed data to apply the effect on the target
            // Better to have a reference and get the data in the consumer than adding the to the context from the producer
            // That way the producer reamin a simple derivation of the abstract system and the effect memory footprint is reduced
            // It doe not cahgne the cost of getting the data from the emitter because the skill is a nentity in itself so there
            // would be an indeirection in the producer too.
            // public Entity Emitter;
            public Entity Target;
            public EFFECT Effect;
        }

        [UpdateBefore(typeof(SkillDeactivationSystem))]
        public abstract class EffectTriggerSystem<EFFECT, CONSUMER> : SystemBase where EFFECT : struct, IEffect
    where CONSUMER : EffectConsumerSystem<EFFECT>
        {
            private EffectConsumerSystem<EFFECT> ConusmerSystem;
            private EntityQuery Query;

            protected override void OnCreate()
            {
                base.OnCreate();
                ConusmerSystem = World.GetOrCreateSystem<CONSUMER>();
                Query = GetEntityQuery(new EntityQueryDesc()
                {
                    All = new ComponentType[]
                    {
                        ComponentType.ReadOnly<Skill>(),
                        ComponentType.ReadOnly<Target>(),
                        ComponentType.ReadOnly<EFFECT>()
                    }
                });
            }

            struct TriggerJob : IJobChunk
            {
                [ReadOnly] public ArchetypeChunkComponentType<Skill> skillChunk;
                [ReadOnly] public ArchetypeChunkComponentType<Target> targetChunk;
                [ReadOnly] public ArchetypeChunkComponentType<EFFECT> effectChunk;
                public NativeStream.Writer ConsumerWriter;

                public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
                {
                    NativeArray<Skill> skills = chunk.GetNativeArray(skillChunk);
                    NativeArray<Target> targets = chunk.GetNativeArray(targetChunk);
                    NativeArray<EFFECT> effects = chunk.GetNativeArray(effectChunk);
                    ConsumerWriter.BeginForEachIndex(chunkIndex);
                    for (int i = 0; i < chunk.Count; i++)
                    {
                        if (!skills[i].Enabled) continue;
                        ConsumerWriter.Write(new ContextualizedEffect<EFFECT>() { Target = targets[i].Value, Effect = effects[i] });
                    }
                    ConsumerWriter.EndForEachIndex();
                }
            }

            protected override void OnUpdate()
            {
                Dependency = new TriggerJob() // Entities.ForEach don't support generic
                {
                    effectChunk = GetArchetypeChunkComponentType<EFFECT>(true),
                    skillChunk = GetArchetypeChunkComponentType<Skill>(true),
                    targetChunk = GetArchetypeChunkComponentType<Target>(true),
                    ConsumerWriter = ConusmerSystem.GetConsumerWriter(Query.CalculateChunkCount())
                }.Schedule(Query, Dependency);
            }
        }

        [UpdateAfter(typeof(SkillDeactivationSystem))]
        public abstract class EffectConsumerSystem<EFFECT> : SystemBase where EFFECT : struct, IEffect
        {
            protected NativeStream EffectStream;


            protected void DisposeStream()
            {
                if (EffectStream.IsCreated)
                {
                    EffectStream.Dispose(Dependency);
                }
            }

            public NativeStream.Writer GetConsumerWriter(int foreachCount)
            {
                EffectStream = new NativeStream(foreachCount, Allocator.TempJob);
                return EffectStream.AsWriter();
            }

            public NativeArray<ContextualizedEffect<EFFECT>> GetArray()
            {
                NativeArray<ContextualizedEffect<EFFECT>> result = default;

                if (EffectStream.IsCreated)
                {
                    result = EffectStream.ToNativeArray<ContextualizedEffect<EFFECT>>(Allocator.TempJob);
                    EffectStream.Dispose();
                }

                return result;
            }

            protected override void OnDestroy()
            {
                base.OnDestroy();
                DisposeStream();
            }
        }


        #endregion


        #endregion


        #region Game specific

        public struct Health : IComponentData
        {
            public int Value;
        }

        #region Effect 1
        public struct Effect1 : IEffect
        {
            public int Value;
        }
        [DisableAutoCreation] // I like to control everything in tests
        public class Effect1TriggerSystem : EffectTriggerSystem<Effect1, Effect1ConsumerSystem>
        {
        }
        [DisableAutoCreation]
        public class Effect1ConsumerSystem : EffectConsumerSystem<Effect1>
        {
            protected override void OnUpdate()
            {
                NativeArray<ContextualizedEffect<Effect1>> effectArray = GetArray();
                if (!effectArray.IsCreated || effectArray.Length == 0) return;
                Dependency = Entities.WithReadOnly(effectArray).ForEach((ref Entity entity, ref Health health) =>
                {
                    for (int i = 0; i < effectArray.Length; i++)
                    {
                        if (!entity.Equals(effectArray[i].Target)) continue;

                        Effect1 effect = effectArray[i].Effect;
                        Health hp = health;
                        hp.Value -= effect.Value;
                        health = hp;
                    }
                }).ScheduleParallel(Dependency);

                Dependency = effectArray.Dispose(Dependency);
            }
        }
        #endregion

        #region Effect 2
        public struct Effect2 : IEffect
        {
            public int Value;
        }
        [DisableAutoCreation]
        public class Effect2TriggerSystem : EffectTriggerSystem<Effect2, Effect2ConsumerSystem>
        {

        }
        [DisableAutoCreation]
        public class Effect2ConsumerSystem : EffectConsumerSystem<Effect2>
        {
            protected override void OnUpdate()
            {
                NativeArray<ContextualizedEffect<Effect2>> effectArray = GetArray();
                if (!effectArray.IsCreated || effectArray.Length == 0) return;
                Dependency = Entities.WithReadOnly(effectArray).ForEach((ref Entity entity, ref Health health) =>
                {
                    for (int i = 0; i < effectArray.Length; i++)
                    {
                        if (!entity.Equals(effectArray[i].Target)) continue;

                        Effect2 effect = effectArray[i].Effect;
                        Health hp = health;
                        hp.Value -= effect.Value;
                        health = hp;
                    }
                }).ScheduleParallel(Dependency);

                Dependency = effectArray.Dispose(Dependency);
            }
        }


        #endregion

        [DisableAutoCreation] // this system should actually be driven by user input.
        public class SkillActivationSystem : SystemBase
        {
            protected override void OnUpdate()
            {
                Dependency = Entities.ForEach((ref Skill skill) =>
                {
                    Skill s = skill;
                    s.Enabled = true;
                    skill = s;
                }).ScheduleParallel(Dependency);
            }
        }

        #endregion

        [Test]
        public void Test_Skill_System()
        {
            // Arrange
            Entity target = _entityManager.CreateEntity();
            _entityManager.AddComponentData(target, new Health() { Value = 100 });

            Entity entity = _entityManager.CreateEntity();
            _entityManager.AddComponentData(entity, new Skill() { Enabled = false });
            _entityManager.AddComponentData(entity, new Effect1() { Value = 1 });
            _entityManager.AddComponentData(entity, new Effect2() { Value = 2 });
            _entityManager.AddComponentData(entity, new Target() { Value = target });


            Entity entity2 = _entityManager.CreateEntity();
            _entityManager.AddComponentData(entity2, new Skill() { Enabled = false });
            _entityManager.AddComponentData(entity2, new Effect1() { Value = 3 });
            _entityManager.AddComponentData(entity2, new Target() { Value = target });

            Entity entity3 = _entityManager.CreateEntity();
            _entityManager.AddComponentData(entity2, new Skill() { Enabled = false });
            _entityManager.AddComponentData(entity2, new Effect2() { Value = 4 });
            _entityManager.AddComponentData(entity2, new Target() { Value = target });

            _world
                .WithSystem<SkillActivationSystem>()
                .WithSystem<Effect1TriggerSystem>()
                .WithSystem<Effect2TriggerSystem>()
                .WithSystem<SkillDeactivationSystem>()
                .WithSystem<Effect1ConsumerSystem>()
                .WithSystem<Effect2ConsumerSystem>();

            // Act
            _world.UpdateAndCompleteSystem<SkillActivationSystem>();

            // Assert
            Assert.True(_entityManager.GetComponentData<Skill>(entity).Enabled);

            // Act
            _world.UpdateAndCompleteSystem<Effect1TriggerSystem>();
            _world.UpdateAndCompleteSystem<Effect2TriggerSystem>();
            _world.UpdateAndCompleteSystem<SkillDeactivationSystem>();

            // Assert
            Assert.False(_entityManager.GetComponentData<Skill>(entity).Enabled);
            Assert.AreEqual(100, _entityManager.GetComponentData<Health>(target).Value);

            // Act
            _world.UpdateAndCompleteSystem<Effect1ConsumerSystem>();
            // Assert
            Assert.AreEqual(96, _entityManager.GetComponentData<Health>(target).Value);
            // Act
            _world.UpdateAndCompleteSystem<Effect2ConsumerSystem>();
            // Assert
            Assert.AreEqual(90, _entityManager.GetComponentData<Health>(target).Value);

        }

    }

    public abstract class DotsTest
    {

        protected TestWorld _world;
        protected EntityManager _entityManager;

        [SetUp]
        public void SetUp()
        {
            _world = new TestWorld();
            _entityManager = _world.GetEntityManager();
        }


        [TearDown]
        public void TearDown()
        {
            _world.CompleteAllSystems();
            _world.Dispose();
        }

    }
}
17

OK, here are the perf results and the updated implmentation :

Edit 3 : the perf test code is full of mistake (sorry) the setup of the test is wrong I’m always targetting the same entity having many entities that don’t actually contribute anything to the test.

5884673--627029--upload_2020-5-22_13-4-35.png
5884673--627029--upload_2020-5-22_13-4-35.png2056×790 44.6 KB

Edit : Forgot the hardware spec
I7-8650U 1,9HGz [8 Cores]
RAM 16G
GTX 1060

using NUnit.Framework;

using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.PerformanceTesting;

using UnityEngine;

namespace WaynGroup.MgmEvent.Tests
{

    class EffectSystemTest : DotsTest
    {
        private const bool TEST_PERF_ENABLED = true;

        #region Core features

        #region Components
        // Skill tag, Enabled will be replaced by the feature Joachim_Ante talked about
        public struct Skill : IComponentData
        {
            public bool Enabled;
        }

        public struct Target : IComponentData
        {
            public Entity Value;
        }

        #endregion

        #region System
        [DisableAutoCreation]
        public class SkillDeactivationSystem : SystemBase
        {
            protected override void OnUpdate()
            {
                Dependency = Entities.ForEach((ref Skill skill) =>
                {
                    Skill s = skill;
                    s.Enabled = false;
                    skill = s;
                }).WithBurst().ScheduleParallel(Dependency);
            }
        }
        #endregion

        #region Abstractions
        public interface IEffect : IComponentData { }

        public struct ContextualizedEffect<EFFECT> where EFFECT : struct, IEffect
        {
            // Could add the emitter entity to get the needed data to apply the effect on the target
            // Better to have a reference and get the data in the consumer than adding the to the context from the producer
            // That way the producer reamin a simple derivation of the abstract system and the effect memory footprint is reduced
            // It doe not cahgne the cost of getting the data from the emitter because the skill is a nentity in itself so there
            // would be an indeirection in the producer too.
            // public Entity Emitter;
            public Entity Target;
            public EFFECT Effect;
        }

        [UpdateBefore(typeof(SkillDeactivationSystem))]
        public abstract class EffectTriggerSystem<EFFECT, CONSUMER> : SystemBase where EFFECT : struct, IEffect
    where CONSUMER : EffectConsumerSystem
        {
            private EffectConsumerSystem ConusmerSystem;
            private EntityQuery Query;

            protected override void OnCreate()
            {
                base.OnCreate();
                ConusmerSystem = World.GetOrCreateSystem<CONSUMER>();
                Query = GetEntityQuery(new EntityQueryDesc()
                {
                    All = new ComponentType[]
                    {
                        ComponentType.ReadOnly<Skill>(),
                        ComponentType.ReadOnly<Target>(),
                        ComponentType.ReadOnly<EFFECT>()
                    }
                });
            }

            [BurstCompile]
            struct TriggerJob : IJobChunk
            {
                [ReadOnly] public ArchetypeChunkComponentType<Skill> skillChunk;
                [ReadOnly] public ArchetypeChunkComponentType<Target> targetChunk;
                [ReadOnly] public ArchetypeChunkComponentType<EFFECT> effectChunk;
                public NativeStream.Writer ConsumerWriter;

                public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
                {
                    NativeArray<Skill> skills = chunk.GetNativeArray(skillChunk);
                    NativeArray<Target> targets = chunk.GetNativeArray(targetChunk);
                    NativeArray<EFFECT> effects = chunk.GetNativeArray(effectChunk);
                    ConsumerWriter.BeginForEachIndex(chunkIndex);
                    for (int i = 0; i < chunk.Count; i++)
                    {
                        if (!skills[i].Enabled) continue;
                        ConsumerWriter.Write(new ContextualizedEffect<EFFECT>() { Target = targets[i].Value, Effect = effects[i] });
                    }
                    ConsumerWriter.EndForEachIndex();
                }
            }

            protected override void OnUpdate()
            {
                Dependency = new TriggerJob() // Entities.ForEach don't support generic
                {
                    effectChunk = GetArchetypeChunkComponentType<EFFECT>(true),
                    skillChunk = GetArchetypeChunkComponentType<Skill>(true),
                    targetChunk = GetArchetypeChunkComponentType<Target>(true),
                    ConsumerWriter = ConusmerSystem.GetConsumerWriter(Query.CalculateChunkCount())
                }.Schedule(Query, Dependency);
                ConusmerSystem.RegisterProducerDependency(Dependency);
            }
        }

        public abstract class EffectConsumerSystem : SystemBase
        {
            protected NativeStream EffectStream;

            private JobHandle ProducerJobHandle;

            protected void DisposeStream()
            {
                if (EffectStream.IsCreated)
                {
                    EffectStream.Dispose(Dependency);
                }
            }

            public void RegisterProducerDependency(JobHandle jh)
            {
                ProducerJobHandle = jh;
            }

            public NativeStream.Writer GetConsumerWriter(int foreachCount)
            {
                Debug.Log(foreachCount);
                EffectStream = new NativeStream(foreachCount, Allocator.TempJob);
                return EffectStream.AsWriter();
            }


            public NativeStream.Reader GetReader()
            {
                return EffectStream.AsReader();
            }

            protected override void OnDestroy()
            {
                base.OnDestroy();
                DisposeStream();
            }

            protected override void OnUpdate()
            {
                Dependency = JobHandle.CombineDependencies(Dependency, ProducerJobHandle);
            }
        }


        #endregion


        #endregion


        #region Game specific

        public struct Health : IComponentData
        {
            public int Value;
        }

        #region Effect 1
        public struct Effect1 : IEffect
        {
            public int Value;
        }
        [DisableAutoCreation]
        [UpdateBefore(typeof(Effect1ConsumerSystem))]
        [UpdateBefore(typeof(SkillDeactivationSystem))]
        public class Effect1TriggerSystem : EffectTriggerSystem<Effect1, Effect1ConsumerSystem>
        {
        }
        [DisableAutoCreation]
        [UpdateAfter(typeof(Effect1TriggerSystem))]
        public class Effect1ConsumerSystem : EffectConsumerSystem
        {
            protected override void OnUpdate()
            {
                base.OnUpdate();
                NativeStream.Reader effectReader = GetReader();

                Dependency = Entities.WithBurst().WithReadOnly(effectReader).ForEach((ref Entity entity, ref Health health) =>
                {
                    if (effectReader.ComputeItemCount() == 0) return;
                    for (int i = 0; i != effectReader.ForEachCount; i++)
                    {
                        effectReader.BeginForEachIndex(i);
                        int rangeItemCount = effectReader.RemainingItemCount;
                        for (int j = 0; j < rangeItemCount; ++j)
                        {

                            Effect1 effect = effectReader.Read<ContextualizedEffect<Effect1>>().Effect;
                            Health hp = health;
                            hp.Value -= effect.Value;
                            health = hp;
                        }
                        effectReader.EndForEachIndex();
                    }

                }).ScheduleParallel(Dependency);

                DisposeStream();
            }
        }
        #endregion

        #region Effect 2
        public struct Effect2 : IEffect
        {
            public int Value;
        }
        [DisableAutoCreation]
        [UpdateBefore(typeof(Effect2ConsumerSystem))]
        [UpdateBefore(typeof(SkillDeactivationSystem))]
        public class Effect2TriggerSystem : EffectTriggerSystem<Effect2, Effect2ConsumerSystem>
        {

        }
        [DisableAutoCreation]
        [UpdateAfter(typeof(Effect2TriggerSystem))]
        public class Effect2ConsumerSystem : EffectConsumerSystem
        {
            protected override void OnUpdate()
            {
                base.OnUpdate();
                NativeStream.Reader effectReader = GetReader();
                Entities.WithBurst().WithReadOnly(effectReader).ForEach((ref Entity entity, ref Health health) =>
                {
                    for (int i = 0; i != effectReader.ForEachCount; i++)
                    {
                        effectReader.BeginForEachIndex(i);
                        int rangeItemCount = effectReader.RemainingItemCount;
                        for (int j = 0; j < rangeItemCount; ++j)
                        {

                            Effect2 effect = effectReader.Read<ContextualizedEffect<Effect2>>().Effect;
                            Health hp = health;
                            hp.Value -= effect.Value;
                            health = hp;
                        }
                        effectReader.EndForEachIndex();
                    }

                }).ScheduleParallel();

                DisposeStream();
            }
        }


        #endregion

        [DisableAutoCreation] // this system should actually be driven by user input.
        public class SkillActivationSystem : SystemBase
        {
            protected override void OnUpdate()
            {
                Dependency = Entities.WithBurst().ForEach((ref Skill skill) =>
                {
                    Skill s = skill;
                    s.Enabled = true;
                    skill = s;
                }).ScheduleParallel(Dependency);
            }
        }

        #endregion

        [DisableAutoCreation]
        [UpdateAfter(typeof(Effect1ConsumerSystem))]
        [UpdateAfter(typeof(Effect2ConsumerSystem))]
        public class EndOfTestSystem : SystemBase
        {
            protected override void OnUpdate()
            {
                Dependency.Complete();
            }
        }


        [TestCase(1)]
        [TestCase(10)]
        [TestCase(100)]
        [TestCase(1000)]
        [TestCase(10000)]
        [TestCase(100000)]
        [TestCase(200000)]
        [TestCase(300000)]
        [TestCase(400000)]
        [TestCase(500000)]
        [TestCase(600000)]
        [TestCase(700000)]
        [TestCase(800000)]
        [TestCase(900000)]
        [TestCase(1000000)]
        [Performance]
        public void Test_Skill_System(int entityCount)
        {
            // Arrange

            Entity target = _entityManager.CreateEntity();
            _entityManager.AddComponentData(target, new Health() { Value = 100 });

            Entity entity = _entityManager.CreateEntity();
            _entityManager.AddComponentData(entity, new Skill() { Enabled = false });
            _entityManager.AddComponentData(entity, new Effect1() { Value = 1 });
            _entityManager.AddComponentData(entity, new Effect2() { Value = 2 });
            _entityManager.AddComponentData(entity, new Target() { Value = target });


            Entity entity2 = _entityManager.CreateEntity();
            _entityManager.AddComponentData(entity2, new Skill() { Enabled = false });
            _entityManager.AddComponentData(entity2, new Effect1() { Value = 3 });
            _entityManager.AddComponentData(entity2, new Target() { Value = target });

            Entity entity3 = _entityManager.CreateEntity();
            _entityManager.AddComponentData(entity2, new Skill() { Enabled = false });
            _entityManager.AddComponentData(entity2, new Effect2() { Value = 4 });
            _entityManager.AddComponentData(entity2, new Target() { Value = target });

            _world
                .WithSystem<SkillActivationSystem>()
                .WithSystem<Effect1TriggerSystem>()
                .WithSystem<Effect2TriggerSystem>()
                .WithSystem<SkillDeactivationSystem>()
                .WithSystem<Effect1ConsumerSystem>()
                .WithSystem<Effect2ConsumerSystem>()
                .WithSystem<EndOfTestSystem>();


            // Act
            _world.UpdateAndCompleteSystem<SkillActivationSystem>();

            // Assert
            Assert.True(_entityManager.GetComponentData<Skill>(entity).Enabled);

            // Act
            _world.UpdateSystem<Effect1TriggerSystem>();
            _world.UpdateSystem<Effect2TriggerSystem>();
            _world.UpdateAndCompleteSystem<SkillDeactivationSystem>();

            // Assert
            Assert.False(_entityManager.GetComponentData<Skill>(entity).Enabled);
            Assert.AreEqual(100, _entityManager.GetComponentData<Health>(target).Value);

            // Act
            _world.UpdateAndCompleteSystem<Effect1ConsumerSystem>();
            // Assert
            Assert.AreEqual(96, _entityManager.GetComponentData<Health>(target).Value);
            // Act
            _world.UpdateAndCompleteSystem<Effect2ConsumerSystem>();
            // Assert
            Assert.AreEqual(90, _entityManager.GetComponentData<Health>(target).Value);

            /**************************************************
             *
             * PERF TEST
             *
             * **************************************************/
            if (TEST_PERF_ENABLED)
            {
                // Arrange
                NativeArray<Entity> targets = new NativeArray<Entity>(entityCount, Allocator.TempJob);
                for (int i = 0; i < entityCount; i++)
                {
                    Entity tmp = _entityManager.CreateEntity();
                    _entityManager.AddComponentData(target, new Health() { Value = 100 });
                    targets[i] = target;
                }

                for (int i = 0; i < entityCount - 1; i++)
                {
                    Entity tmp = _entityManager.CreateEntity();
                    _entityManager.AddComponentData(tmp, new Skill() { Enabled = false });
                    _entityManager.AddComponentData(tmp, new Effect1() { Value = 1 });
                    _entityManager.AddComponentData(tmp, new Effect2() { Value = 2 });
                    _entityManager.AddComponentData(tmp, new Target() { Value = targets[i % 3] });


                    Entity tmp2 = _entityManager.CreateEntity();
                    _entityManager.AddComponentData(tmp2, new Skill() { Enabled = false });
                    _entityManager.AddComponentData(tmp2, new Effect1() { Value = 3 });
                    _entityManager.AddComponentData(tmp2, new Target() { Value = targets[i % 5] });

                    Entity tmp3 = _entityManager.CreateEntity();
                    _entityManager.AddComponentData(tmp3, new Skill() { Enabled = false });
                    _entityManager.AddComponentData(tmp3, new Effect2() { Value = 4 });
                    _entityManager.AddComponentData(tmp3, new Target() { Value = targets[i % 7] });
                }

                targets.Dispose();

                Measure.Method(() =>
                {
                    _world.UpdateSystem<SkillActivationSystem>();
                    _world.UpdateSystem<Effect1TriggerSystem>();
                    _world.UpdateSystem<Effect2TriggerSystem>();
                    _world.UpdateSystem<SkillDeactivationSystem>();
                    _world.UpdateSystem<Effect1ConsumerSystem>();
                    _world.UpdateSystem<Effect2ConsumerSystem>();
                    _world.CompleteAllSystems();
                }).Run();
            }


        }

    }

    public abstract class DotsTest
    {

        protected TestWorld _world;
        protected EntityManager _entityManager;

        [SetUp]
        public void SetUp()
        {
            _world = new TestWorld();
            _entityManager = _world.GetEntityManager();
        }


        [TearDown]
        public void TearDown()
        {
            _world.CompleteAllSystems();
            _world.Dispose();
        }

    }
}

I am both impressed and disapointed.
It amaze me that this hold up well until 100 ~ 200 K entities but at the same time I’m disaponted that the base perf is still 2-3 ms, even for a single entity.

Edit 2 : the perf entity count are false, I actually have 4 time that number of entity (1 target and 3 “attacker”) with an average of 4 times the entity count of effect per target.

If anyone sees a flaw in either the implementation or the perf test solgic please let me know.

One other thing I don’t get regarding dependancy between systems. I though that with system base, the automatic Dependency management and the UpdateBefore/After tag scheduling dependent job wetween system would be handle “automagically” but it does not seem to e the case.
I still had to manually register the producer as a dependency to the consummer…

[...]

 protected override void OnUpdate()
            {
                Dependency = JobHandle.CombineDependencies(Dependency, ProducerJobHandle);
            }

[...]

      ConusmerSystem.RegisterProducerDependency(Dependency);

[...]

Idealy i’d like to hide the NativeStream.Reader logic to let the consumer only care about the actual effect logic. (custom job ??)

18

  • Your trigger job is not using burst.

  • NativeArray<ContextualizedEffect> effectArray = GetArray(); seems to be doing work on the main thread can this be moved to a burst job before your entities foreach?

  • Lastly for very small entity count, using Entities.Run() on your jobs will be significantly faster. (You still want to burst all your code so that you have bursted code running on the main thread)

  • Debug.Log(foreachCount); Debug.Log is very expensive…

  • Using [DeallocateOnJobCompletion] in an already existing job is faster than Dispose on the array. Optimally you wouldn’t be recreating containers every frame though. You have a system own a NativeList or array or whatever fits and reuse it. So it is only diposed on OnDestroy

  • Are you testing in standalone player? Perf in editor especially on base overhead is bigger than in standalone.

Lastly we are aware that performance when running small entity count is not optimal, right now dots is very well optimized for the scale case but not so much the i have one entity case, scheduling job overhead etc that are being worked on.

2 Likes
19

Hi,

I have some feedback on this. There are cases where it is desirable to have a system run on the main thread based on some condition (if we are running as a server or if the entity count < n).

Would it be possible to add new scheduling methods like ScheduleParallelWhen(bool condition) and ScheduleWhen(bool condition) that would schedule the code through the job system when the condition is met and run it on the main thread when it is not?

2 Likes
20

https://discussions.unity.com/t/786143/1