I found this on reddit and I think it’s interesting as I experience some of the problems that was written. The writer defined a new “ECS” which addresses the problems. While I don’t think Unity will move this way, I find the ideas compelling.
To be fair, I’ve read about Unity’s plans to add a feature where components can be enabled/disabled. This will address some of the pain points mentioned in the article.
We are gonna introduce a couple other concepts that will make usage significantly simpler
But i think most importantly: Not every problem should in fact be solved with ECS.
ECS is amazing at table processing. Animation graphs for example are not a great fit. Hence the new animation system is based on the dataflowgraph package. Which takes data oriented principles to processing graphs. It optimally processes a graph in the exact order while allow parallel processing of independent pieces.
Ultimately data oriented design is about bringing the right solution to the right problem. Not everything has or should be fitted in the same way.
Even when you dont use ECS data for all your processing, what ECS really gives you is a fast common data storage that lets different systems communicate.
Eg.
an animation system still needs to talk to game code changing simulation parameters (Speed / Turn / Shooting etc)
A physics system does collision detection of an BVH and all kinds of internal processing using NativeStream. All of this is specialized code for the specific problem. The way it relates to ECS is that it can efficiently read / write to all the data the game code might modify or that might get streamed in. Eg. which mesh collider, velocity, mass, position etc
There is nothing wrong with having a game code reactive system maintain a BVH tree of all game elements near the player, so that other game code can do fast queries against this optimised structure.
Would’ve been awesome to learn about DataFlowGraph, since there’s second to none documentation on the package’s page https://docs.unity3d.com/Packages/com.unity.dataflowgraph@0.15/manual/index.html
Though I understand that it’s a preview package. I guess we don’t have Animation package docs for the same reason. Just wanted to point out that this package, even if considered low-level, may get handy for some of us and will need docs in the future.
Not every problem should in fact be solved with ECS.
That is of course true, but that shouldn’t stop us from thinking about improvements, especially when those improvements don’t adversely impact all of the good stuff, like having a fast data storage.
FWIW, all of the suggested improvements have been implemented in a framework that storage wise is very similar to Unity DOTS (also archetype based), without giving up on things like being able to iterate over contiguous arrays, vectorize code etc.
One of the things I find appealing about ECS is that it combines high-level abstractions with efficient storage, and I’ll want to use that efficient storage wherever it makes sense. The article addresses what I think are arbitrary limitations in ECS as it is commonly defined. By removing those limitations I increase the number of situations where I can still use this high-level API in combination with the fast storage.
Few notes:
Multiple Component Instances - DynamicBuffers:
Sure, but at that point you’re basically managing your own pools which is less efficient (higher fragmentation vs. having a regular component array), and you’re no longer able to use regular ECS operations.
Runtime Tags - SharedComponents with a unique identifier
What if I want to iterate entities that are organized across different chunks / archetypes? This may be my lack of understanding of Unity ECS, but I thought shared components are bound to a chunk.
System Execution Order - ComponentSystemGroups
This looks good. My goal is to ultimately have a mechanism whereby you can import systems from an asset store, import them into your project and it should “just” work. I think ComponentSystemGroups are a good step in that direction.
@SanderMertens joining the discussion. I agree, DynamicBuffers do have higher chunk fragmentation and majority of the times hurt performance, also the problem comes with loading them up to a cache line when you load the entire buffer into the cache but only use few elements which indeed is wasteful.
I’m not sure what you mean by “managing your own pools”. But I think it is good that there is an explicit difference between component types where only one can exist and where several can exist. It allows for the code to make meaningful assumptions.
It is more like shared components bind entities to chunks. By assigning a bunch of entities with the same shared component, the entities all end up in the same chunks. Then they can all be fetched together. Obviously this isn’t that useful if the entities are changing shared components frequently. But I don’t find that use case to be all that common compared to entities classified at spawn time.
Multiple Component Instances - DynamicBuffers
How though? Say component A and component B needs a separate Timer. You store the Timers in the buffers then assign an index in the components? That’s clunky. My current solution to this is to use a separate entity with Timer then assign this entity to the components.
Another problem is that you can only add one type of Dynamic Buffer. If you have different components that need some kind of container for the same type, you’re left with creating separate entities for the buffers instead then use these entities in the components as reference to the buffers. I may have fixed the data modelling but the nagging question now is “Is this really the right way?”
State machines
Anything that needs a dynamic ordering of execution is not very intuitive in ECS (command pattern, AI behavior). It takes some hoops to implement either by tag components or flag checking. Not to mention the fixed system execution. Say you need to run B then A preferably on the same frame but systems were ordered to run SystemA then SystemB. So B will be executed this frame but A will be executed in the next. This may seem like a non problem but considering that a lot can happen between SystemB and SystemA, it’s a recipe for hard to find bugs. What’s your solution to this? I’ve thought about using OOP for these kinds of problems but my issue is that accessing/modifying data by EntityManager is slow or not suitable to do for every frame.
Why not put the ‘timer’ in each component ? Yes you’ll need to have one system to update the timer of each type of component having a timer but that would be I think the most runtime efficient way of dealing with it. additionnal htoughts regarding timmers, I think it would be better to have a timmer logic that store the next tick time instead of storing aremaining time that you update every frame. It would allow to make use of the DidChange filtering. Also you could probably define a ITimer interface that your ComponentData would implement and taht should allow you to stream line the timer update systems a bit through generics.
That doesn’t scale well though. One, you’ll be adding overhead to every component you want to potentially add a timer to (which also requires foresight about which components need timers, ever, but let’s put that that aside for now).
Secondly, an expiry timer is only one kind of generic “trait” I could apply to a component. What about a trait where I want to add a component after N seconds? What about a generic lerp system, or a system to store the last N values of a component?
IMO, either the component need the timer behavior or it does not and if you have case were the same type of data sometimes need the timer and sometimes don’t, I would argue t’s not the same type of data and therefore would justify 2 components.
For the second point, I would probably go with reactive systems or a system that implement a native stream/queue with a delay mechanism.
Each problem as it’s solution, not everything should be components.
That is fair- though the solutions you propose all perform worse on any metric but one: that we don’t have to change ECS =p That is a perfectly reasonable reason and conclusion, but the premise of my article is to allow for those small tweaks so that we can still do such things in a clean DoD way.
For the timer case I don’t see how it performs worse on every metric. Regarding the memory footprint having the timer in the component itself avoids having to store any reference to a third party structure (buffer or entity). This also avoid having to do the indirection towards the other data structure so less computation.
The fact that they are properly alligned in memory allows for quick loop through with burst.
The only less performant aspect of it I can see is teh convinance of build/use, you’ll need to define more component and more systems instead of having a single system manage all your timing, but even with that in mind I don’t feel that manipulating indirection every time i need to operate on the timer is convinient either…
Granted I did not test both solutions but intuitivly it make sense to think that less work mean more performance and less indirection mean les memory usage, so…
I feel like your view of ECS involves a lot of ‘pointer’ and indirections. I don’t know how it will impact your data layout.
I don’t consider myself an expert in hte matter at all but ECS and DoD are two different things IMO and maybe to have great DoD implies you have to sacrifice some of the conviniance of more abstract approach.
We use this approach for ai behaviors on entities, and it’s worked very well. I wrote a custom job type to automatically execute all ai behaviors which are stored in a buffer like this.
It’s been extremely fast. And the job abstraction also made it intuitive. We also use plenty of state machines. What do you consider clunky about the approach?
Solving this problem has been a major focus of ours since starting to use DOTS. This was kind of a litmus test for us - we had to make sure ECS was capable of performing both behavior patterns A,B,C and C,B,A on the same frame, with strong performance. It absolutely can be done, without resorting to OOP. Seeing the results made DOTS believers out of us.
How did you solve this? I’m all ears. I tried converting our GOAP into ECS and the resulting library is kinda hard to use as I relied to adding/removing tags to order action execution.
I am heartbroken to say that this goes into proprietary tech that I cant share here. I’m truly sorry to have to say that, @davenirline . Maybe one day we will be able to. GOAP should work with our solution, and we have a pattern in mind that’s on our schedule for later in the year. Maybe we’ll run into some of the same issues you did.
I’d probably use a timepoint approach and also store the earliest timepoint in a component so that I only have to iterate over the buffer whenever the next timer expired.
You can have different types of DynamicBuffers attached to a single Entity. Unity already does this. Most entities with hierarchies converted from prefabs have both a LinkedEntityGroup and a Child buffer. Unless you are talking about packing different types into the same buffer? In that case, you may consider unions or some clever use of BlobAssetReferences. But really that is more of an OOP concept anyways.
Your solution performs worse, because even if we forget about the memory overhead for a second, you will have to evaluate every component to see whether the timer should be progressed, regardless of whether the timer is active or not.
TL;DR: the timer component is stored as a regular component array, just like any other component, that you can iterate like a contiguous array. There is no indirection. You iterate the components, update the timer value, and when the timer expires you delete the component the timer is associated with. The one thing that makes this different from a regular component is that I can add it multiple times to an entity.
So to summarize:
You can iterate N timers per entity as a regular component array, which is just as fast as the fastest kind of iteration
You only iterate over timers that are actually active
You don’t incur additional memory overhead besides what is necessary for storing the Timer components
Because of the above, there are no indirections. If it weren’t for the check on whether the timer is expired, this code could easily be vectorized.
OOP and DOD is not opposite thecs the can leave with each other. Even more OOP can be just way to use DOD.
Do any one know that Unity was always be ECS? every MonoBehaviour is actually C and S, GameObject is E
and under the hood every thing work exactly (mostly) like new ECS tech from Unity (Entities).
I want to say that actually nothing changed new tech does not change anything in a way we make code.
They propose new way and it is right way if you want best performance but no one stop you use Entities Package in OOP way. And even then you can have better performance, forgot about pools and GC and just be somewhat happier.
We just do that in our project. added few extension methods for entity like GetComponent, SetComponent, HaveComponent etc to mimic GameObject world and easy translate OOP project to Entities and nothing changed in way we write code. Yes we absolutely dont use DOTS potential I just want to say that
DOTS is not DOD. Plain C# id not OOP.
DOD is how you use tool in your hands OOP is the same.
DOTS is just new tool from Unity. GOOD TOOL
If you say want to store many timers on one Entity.
Old GO tool naive design:
1.Add many Timer Components to GO.
2.Add reference to exact timer to Components that need timer.
3.Every timer component is heavy thing with big
(more than 64 bytes) overhead just for empty MonoBehaiour and it is in random memory and GC and
New Entities Package from Unity naive design:
0.Create ComponentDataArray that will help store regular TimerComponentData struct on Entity many times
1.Add many Timer Components to Entities.
2.Add reference(handle) to exact timer to Components that need timer.
3.Every timer is cheap structure may be just 8 bytes all timers of one Entity can lie near entity no GC and this is
not best performant way it is just OOP way on Entities with all benefits of new Unity tech
I understand know that you want to add some sort of experiation time on any component.
In that case I agree adding the timings data to the compoennt itself would not be a good pattern.
In current ECS implementation you could probably have a system maintining a dictionnary of <Entity,ComponentExpiracy> where ComponentExpiracy contain the time of expiracy (not the remaining lifetime but the time at which the component should be removed so to avoid updating the data every frame) and the component type (or type id) when the expiracy time is reached, issue a remove component ecb command and remove the entry from the dictionnary. This would need additionnal work to make it run in parrallel in a burst job though.
Edit : Having a dynamic buffer of ComponentExpiracy should allow the samebehavior.
Iterating over them should be fast has you can have a bursted system iterate in parrallel over each entity’s dynamic buffer. And the iterating over inactive timer is a non issue as you would remove both the component and timer one the timer is expired.