Generic systems responding to similar data on entities

Unity Engine
,
1

Hi all,

I’ve been working on a few systems lately including an Attribute/Stats system, and one pattern keeps coming up that is bothering me regarding re-usable systems logic processing “lists” of something on entities.

Afaik there is no way to write a system that can query for a variable number of components that all have something in common?

If I would have a query that looks for component A OR B OR C etc. then I’m still not sure how I could iterate over a variable number of components on the entities found? And even then I wouldn’t have an interface to cast them to to process them in a common way.

I feel like I might be missing something…

For this Stats System:

  • Entities can have a variable number of “stats” (ie Health, Food, Armor, Speed etc).
  • Each stat has things like: CurrentValue, MinValue, MaxValue, NormalizedValue, ChangeOverTime.
  • Then each stat has a “modifier stack” to add/subtract or multiply values for Min/Max and ChangeOverTime.
  • The modifier stack is stored in a Dynamic Buffer.
  • You add/remove entries in the modifier stack with “event” entities that a system responds to and edits the buffer.
  • Over the lifetime of an entity, the number of items in the modifier stack changes but the number of stats stays the same.
  • Every time the modifier stack changes we re-calculate the stat values as needed.

I want to have a generic system that can respond to changes in that stack and recalculate all the values as needed.

At a quick glance it feels that Stats should be component data (since the count doesn’t change), and the modifier stack should be a dynamic buffer (since the count does change).

I’ve come up with a few approaches, and don’t really like any of them… but perhaps leaning most towards #3

1) Each stat is a ComponentData (Health, Food etc).

  • Pro: Very easy to read and write current stat values
  • Pro: Easy to write systems that respond to conditions like Health.Value == 0.
  • Pro: Full gain of sequential memory layout.
  • Con: Since we don’t have inheritance or polymorphism, I would need to write a system for each stat that has the same logic. ie HealthStatSystem, FoodStatSystem and so on… And all those systems would have the same logic to bake down the modifier stack from the dynamic buffer.

2) All entity stats stored together in a “Stats bag” Dynamic Buffer.

  • Pro: One system can do all the calculations to bake down the modifier stacks into the stat values.
  • Con: Not as easy to read/write stat values.
  • Con: Harder to write systems that respond to conditions like Health.Value == 0.
  • Con: Memory layout not as ideal (not all Health stats together in sequence)

3) A generic system where each entry in modifier stack can write directly into a separate component on target entity.

  • A rough version of this would be something like: “if (mod.TargetStat == StatType.Health) { healthComponent.Value = bakedValue; }”
  • Pro: We gain all the benefits of memory layout when reading stats
  • Pro: Systems can respond to conditions like Health.Value == 0
  • Con: The stat component references would need to be brought in as GetComponentDataFromEntity()
  • Con: Since the number of stats varies from entity to entity, we would need to do .Exists checks before accessing components from array
  • Con: System needs write access to all stat components
  • Con: “Hard coded” relationship of stat type and what component to set.

Any ideas would be appreciated :slight_smile:

Cheers,
Siggi.

2

Funny seems like you’re going through the same dilemmas as me when I was transforming my Utility AI system to ECS.

The way I settled few months back was to use ComponentDatas for things that are always on every actionable entity ( e.g. health, stamina ), DynamicBuffers for other various optional stats and other things.

I also thought about hacking in some data blobs, like creating a ComponentData with multiple ulongs and then storing /retrieving bytes based on an index, but eventually I gave up since it was way too complicated to manage.

I remember reading that you were also using an UtilityAI system, and if you decoupled it properly you wouldn’t have to worry about how or where your data is stored, since each consideration system would handle the data in its own way.

3

If all your stats and modifiers are of the same format, can you use generic components and systems with option #1? I have no insight on that myself but I’m sure I’ve seen examples of it on these forums.
CDFE conditional’ing is pretty ugly when you have many of them.

4

You could have each an entity for each stat with a Component as you described ( CurrentValue, MinValue, MaxValue, NormalizedValue, ChangeOverTime) and a tag component (Health, Armor, Food, whatever); and in the entity for the character/player/object/thing have a NativeArray (or another type of collection which can give easier access depending on the case) to associate the character with the stats.

for health==0, when any Stat reaches 0 or another important value (min value and max value I assume) it creates an event-entity

Not sure if this is the best way, Im new to ECS and thought about this.

5

Yep I’m also starting work on a Utility AI (Infinite Axis) system for ECS, I just wanted to tackle this system first.

One of the things here that does feel right to me is: Non-optional data = component data, optional/variable size = dynamic buffers.

They aren’t of the same format, and the breaking thing is you can have multiple modifiers even for the same type… and you cannot have multiple components of the same type on an entity… hence why I went with a Dynamic Buffer for that.

If you meant “generic” as in generic classes (Stat), then afaik that wouldn’t make any different since each type would still be separate and no way to process them all together. I’m not even sure if ECS/Burst supports those at all.

Interesting idea and would make the systems processing the stat values smaller (ie one for setting Min/Max, one for normalizing etc), but it would still have the downside of non-linear reading of stats on entities (since you need to read first the stats list on character, then read each entity linked from that).

6

I think it could improve if you have the character-entity dependent systems rely on the event generated by the stat-entities (on change, on max value, on min value, like a pub/sub pattern where the stats publish events and the character-dependent components subscribe to these events. That way the character-entity wouldn’t need to read the stat value directly, instead, each the event-entities generated by the stats would need to have linear access to their subscribers.

Kinda like the INotifyPropertyChanged in the System.ComponentModel library on C#.

P.S.: I think this would decentralise the code too much, might not be a good solution.

Again, I’m not sure if this is the best way, I’m just trying to give some ideas. I would appreciate if you find a nice solution, this looks like an interesting and common problem.

1 Like
7

I was planning on doing change events anyway, and you are right they could be used to catch things like Health==0, but it would not be optimized for linear reading of values… Lets say I want to iterate over enemies to find the one with the lowest health.

I’m starting to lean towards some level of data duplication… ie have the mod stack and the “complex” value type (min/max etc) but then copy the result of the calculation into an easily accessible place (ie multiple specialized entity components, ie HealthStat, FoodStat etc).

8

what I would do would be to set an ID and some bools inside the buffer

the buffer would be something like this

int StatID
float/int value

bool continuous?
int timer

bool Remove?

then I would make every stat a component and make a generic system that I would copy/paste for every stat

then I would go through the systems and give an Stat id for them like… strenght = 1, int = 2, etc

after that I would make them check the buffer for changes to their specific stat using the IDs and mark the entry as “remove” true and run every calculation needed

I would then make a system group that would run AFTER the “stats calculation” group and use a generic system to go through the buffer and remove every entry with “remove = true”

if I need some kind of events I would make 3 groups that would run one after the other

“stat modify group” “Stat After Mod group” “Stat Calculations Finish”

lets say I want to run an event when hp == 0, if I didnt want to mess with the HP script, I Would make ANOTHER script that would check the hp again and check if it’s 0 inside the “stat after mod gorup”

or if i’m willing to mess with the hp system I would simply put a condition inside it or give a component to the entity like “dead” or “hp0” and make a system to run through it, again, inside the second group

===
the reason for not messing with the buffer inside the original systems is so that I can make it read only and run in parallel before going to the main thread to fix the buffer and remove the entries

my reason for going like this is because I know myself and I know I Will want to expand the game later and I dont like messing with old code too much, this way i’m able to make modifications to the core code using something like “events” components and the groups to control the order my new systems will go through

9

There is no reason you cannot have polymorphism inside the ECS systems. On my networking layer I have multiple dynamic buffers on the connection entity that are used by different system protocols. These are either appended to or replaced by the latest DataStreams, depending on the use case for the buffer in question. I have two generic JobComponentSystems for this, one appends, one replaces. Here is an example of the replace system:

public abstract class DataStreamBufferReplaceSystem<T> : JobComponentSystem
    where T : unmanaged, IBufferElementData
{
    public byte Protocol { get; set; }

    private EntityQuery _bufferQuery;

    private ArchetypeChunkBufferType<NetworkProtocolIndex> _networkProtocolIndexTypeRO;
    private ArchetypeChunkBufferType<NetworkProtocolCount> _networkProtocolCountTypeRO;
    private ArchetypeChunkBufferType<NetworkDataStream> _networkDataStreamTypeRO;
    private ArchetypeChunkBufferType<T> _bufferTypeRW;


    protected override void OnCreate()
    {
        _bufferQuery = GetEntityQuery(new EntityQueryDesc
        {
            All = new ComponentType[]
            {
                typeof(NetworkProtocolIndex),
                typeof(NetworkProtocolCount),
                typeof(NetworkDataStream),
                typeof(T)
            }
        });
    }

    private void GatherTypes()
    {
        _networkProtocolIndexTypeRO = GetArchetypeChunkBufferType<NetworkProtocolIndex>(true);
        _networkProtocolCountTypeRO = GetArchetypeChunkBufferType<NetworkProtocolCount>(true);
        _networkDataStreamTypeRO = GetArchetypeChunkBufferType<NetworkDataStream>(true);
        _bufferTypeRW = GetArchetypeChunkBufferType<T>();
    }

    protected override JobHandle OnUpdate(JobHandle inputDeps)
    {
        GatherTypes();

        return new NetworkReplaceBufferJob
        {
            Protocol = Protocol,
            NetworkProtocolIndexTypeRO = _networkProtocolIndexTypeRO,
            NetworkProtocolCountTypeRO = _networkProtocolCountTypeRO,
            NetworkDataStreamTypeRO = _networkDataStreamTypeRO,
            BufferTypeRW = _bufferTypeRW
        }.Schedule(_bufferQuery, inputDeps);
    }

    [BurstCompile]
    private struct NetworkReplaceBufferJob : IJobChunk
    {
        public byte Protocol;
        [ReadOnly] public ArchetypeChunkBufferType<NetworkProtocolIndex> NetworkProtocolIndexTypeRO;
        [ReadOnly] public ArchetypeChunkBufferType<NetworkProtocolCount> NetworkProtocolCountTypeRO;
        [ReadOnly] public ArchetypeChunkBufferType<NetworkDataStream> NetworkDataStreamTypeRO;
        public ArchetypeChunkBufferType<T> BufferTypeRW;


        public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
        {
            var networkDataStreamBuffers = chunk.GetBufferAccessor(NetworkDataStreamTypeRO);
            var networkProtocolIndexBuffers = chunk.GetBufferAccessor(NetworkProtocolIndexTypeRO);
            var networkProtocolCountBuffers = chunk.GetBufferAccessor(NetworkProtocolCountTypeRO);
            var buffers = chunk.GetBufferAccessor(BufferTypeRW);

            for (int i = 0; i < chunk.Count; i++)
            {
                var networkProtocolCount = networkProtocolCountBuffers[i][Protocol].Value;

                if (networkProtocolCount == 0)
                {
                    continue;
                }

                var networkDataStream = networkDataStreamBuffers[i];
                var protocolStartIndex = networkProtocolIndexBuffers[i][Protocol].Value;
                var protocolEndIndex = protocolStartIndex + networkProtocolCount;
                var dataStream = networkDataStream[protocolEndIndex - 1].Value;
                var buffer = buffers[i];
                var length = dataStream.Length - sizeof(byte);
                buffer.ResizeUninitialized(length);

                unsafe
                {
                    UnsafeUtility.MemCpy(buffer.GetUnsafePtr(),
                        dataStream.GetUnsafeReadOnlyPtr() + sizeof(byte),
                        length);
                }
            }
        }
    }
}

The buffers and systems are declared as so:

public struct InputBufferIn : IBufferElementData
{
    public byte Value;
}

public struct StateBufferIn : IBufferElementData
{
    public byte Value;
}

public sealed class ReplaceInputBufferSystem: DataStreamBufferReplaceSystem<InputBufferIn>
{

}

public sealed class ReplaceStateBufferSystem: DataStreamBufferReplaceSystem<StateBufferIn>
{

}

And the specific versions of the generic system are added to the world:

_world.CreateSystem<ReplaceInputBufferSystem>();
_world.CreateSystem<ReplaceStateBufferSystem>();

Everything works pretty much as you would expect. My guess is, based off your original requirements, you could create something like this:

public class StatSystem<T> : JobComponentSystem
    where T : unmanaged, IComponentData, IStat
{
    protected override JobHandle OnUpdate(JobHandle inputDeps)
    {
        throw new System.NotImplementedException();
    }
}

public interface IStat
{
   int Value { get; set; }
   int Min { get; set; }
    int Max { get; set; }
}

public struct Health : IComponentData, IStat
{
    private int Value;
    private int Min;
    private int Max;

    int IStat.Value
    {
        get => Value;
        set => Value = value;
    }

    int IStat.Min
    {
        get => Min;
        set => Min = value;
    }

    int IStat.Max
    {
        get => Max;
        set => Max = value;
    }
}

You can then perform any sort of generic behavior inside your stat system based on your IStat interface. The burst compiler will (to my understanding, though maybe I should confirm this) optimize away the simple property accessors for Health/Min/Max to direct field calls and you should suffer no performance penalty for interacting through the interface.

2 Likes
10

Maybe 5argon can make a blog post investigating burst jobs working through interfaces vs a normal explicit implementation. :roll_eyes:

11

Personally I prefer a generic approach for stats. Value per component is pretty much the opposite of that and IMO not really a good approach for productivity on several levels.

We have a StatType enum and then our various systems can have a uniform interface for dealing with stats. Adding/removing a stat is just adding/removing an enum. It automatically shows up in our various data editors that use stats. We maintain a couple of simple grouping abstractions for what stats attach to what logical entity types.

As @BrendonSmuts points out components can be used in a variety of ways. Just using them as the fields are the public api is not the only approach especially with tools like DynamicBuffer and ComponentDataFromEntity.

For general abstractions for accessing stats, I’ve found that accessing them through methods starting with GetBase/EffectEffective works well and keeps your public api’s more stable. Whether that’s GetEffectiveX or GetEffective(StatType) or whatever depends on how you go in other areas. But stats are just used by so many systems that stability in the api tends to matter a lot. Also you end up with more variations then you might think. Like almost every mmo has a concept of buffs that alter the max. So you need current max, real max, and then the value.

For updating, on the client stat data is readonly. You have a single source for updates, the network. So it’s easy to just update your client side views as that data hits. You have a single place where the data is ever modified.

I don’t really want to drag this off topic so I’ll refrain from talking about server specific flows, but they would necessarily and correctly be different here even if most of the data models were the same.

12

The are a number of advantages for representing stats, or data in general, with an explicit component type.

It makes for better dependency management and parallelization in jobs. Splitting types out means one system can spend its time modifying Health, another modifies Food, another is reading from Amour, etc. If you are merging this data into a single data type you lose the ability to do this.

There is more flexibility in how you can shift the memory layout of your individual stats based on access patterns using this sort of approach.

You can also query against these types in systems that are applicable to only a specific stat. A DeathSystem can filter against entities that have a health component instead of needing to inspect each Statted entity to see whether it contains a health flag.

I think Unity’s implementation of Jobs/ECS needs to have strongly typed data to work best, however only you know what will work optimally for your projects requirements.

1 Like
13

Thanks @BrendonSmuts ! I had no idea you could write generic systems like that, or I just assumed it wouldn’t work.

Going to adapt my approach a bit and explore this :slight_smile: