(Code Architecture) Whats a good way to organise entity data?

At the company I work at, we make small mmo’s, which means we have a ton of systems/code which often interacts with the entities in some way, which are usually the different characters you can play as. The entity might be required to have some system related data associated with it for system for the system to work.

Some examples include:

  • battle systems, whereby the entity needs some health or some stats;
  • camera systems, whereby the entity might need a specific positional and rotational offset for the camera to use (particularly when the entities are different in proportions;
  • Sneak systems, whereby the entity might need to specify some kind of loudness with each step it takes or something.

Right now at my company, we don’t have a common solution for organising where this data is stored. Our solutions include:

  1. Adding the data to the entity data class (non-monobehaviour, usually a scriptable object) or entity controller monobehaviour

  2. Creating Lists or Dictionaries whereby you can retrieve the data using the entity as a key (or an entity type enum)

  3. Adding a system-specific component to the entity that encapsulates the data it wants

I understand there are probably pros and cons to each solution, but is there one that generally offer the best maintainability and readability? Or is this a case specific problem whereby some solutions are better than others in some circumstances.

We need more structure and consistency in our projects because they are becoming increasingly harder to manage as they grow, and I think things could be far less complex.

Thanks in advance for any replies!

If you dont already use it, you may wanna have a look at the new DOTS (ECS, Jobs, Burst) architecture recently introduced to Unity. You basically define Components, which only contain a single piece of information. For example, you would have an HpComponent and a PositionComponent. Entities are then defined by the components they are made of. An Enemy would, for example, contain an HpComponent, as well as a PositionComponent. Systems are then used to execute code, like pathfinding, for all Entities that use a specific component, in this case the PositionComponent. This results in a very efficient memory layout, that can easily be multithreaded using Jobs, which also enables you to make use of the Burst compiler, for huge (!) performance speedup (we are talking in regions of factor 2 - 100 times faster).

Your question was mostly concerned with architecture, not performance, but using DOTS would force you into a very specific way of writing your code (solving your architecture problems), with the added benefit of giving you huge amounts of free performance.

That said, DOTS is not object oriented anymore, thus it will take some time getting used to and (because of that) may not feel very readable. In case your MMOs are for one reason or another not really in need of free performance, or you simply want / need to stay with an object oriented aproach, then we are back to the original question.

Generally speaking “specific solutions being better than others in some circumstances” is always true. However, for most problems commonly solved by programmers, there exist specific programming patters, which can be seen as a state of the art solution for given problems. You may wanna look up programming/design patters to get an idea how specific problems are approached.
That said (if you do not want to use DOTS), i’d go with a strictly object oriented, inheritance based approach for organising your code. You could, for example, define an abstract class for Entities, that contains all the common attributes and functions all Entities must have. Classes like Enemy or MainCharacter will then inherit from the abstract Entity class and add to it, or overwrite some parts of it. Other required functionality can be added through Interfaces. You could, for example define a CharacterCameraSettings interface, which contains, for example, the (unimplemented) GetCameraOffset function. The entire example would work out like this:

public abstract class Entity
{
    internal int currentHP;
}

public interface CharacterCameraSetting
{
    Vector3 GetCameraOffset();
}

public class MainCharacter : Entity, CharacterCameraSetting
{
    public MainCharacter()
    {
        // we can access all the abstract entity data:
        currentHP = 10;
    }

    // We were forced (!) to implement this method because we implement the interface
    public Vector3 GetCameraOffset()
    {
        throw new System.NotImplementedException();
    }
}

You can save lists of Entities if you are only concerned with, for example, HP, but still save your MainCharacter in that list, since it is also an Entity. In the same way, you could have a variable of type CharacterCameraSetting and save an instance of your MainCharacter in it, if all you are concerned with is the contents of the CharacterCameraSetting (in the above example that would be the GetCameraOffset method).

Hope this helps!

Hi Yoreki,

Thank you for the detailed response! It’s funny because when I was trying to search for an answer to my question, ECS videos/links would pop up in google. The usefulness of a more data driven architecture has actually been becoming more apparent to me since I guess you’re not stuck in the OOP paradigm of trying to translate real world stuff into objects when the stuff in our games ends up not acting like stuff in the real world would! If that makes any sense… Of course like you said this can be better managed using interface polymorphism n stuff, but I’ve had some issues with that as well… but that’s probably due to my lack of experience.

Unfortunately because of how new ECS is, I can’t imagine the whole team moving over to it for new projects; I’d have to experiment with it first. But it’s nice knowing that it is a solution. The performance boost would be amazing for us in some cases because we develop our games for mobile.

I’m happy you mentioned the Entity solution with interfaces. I actually tried for the first time to do pretty much exactly that for our current project. We did however run into issues where interfaces started growing too large and some systems ended up needing information from other systems to make game specific decisions which resulted in us just having to fall back to having the system depend on the entity instead of the abstractions (ICameraSettings for example). Of course again, this probably comes to poor implementation and design… but I feel like now we were kind of on the way…

It’s easy to write a system for a game, but when u want to use that same system for multiple different games with slightly different mechanics, or your original game changes in some way… that’s when it starts to become a big mess and a headache.

Thank you again for your advice, this is a difficult problem for me, so it’s great to hear someone else’s opinion!

If you have any more advice, I’d love to hear it.

Often architecture looks a bit more easy on paper than it is in the real world. Remember that our example only was one layer deep, but you can make it as deep as you want. For example, “MainCharacter” was a bad choice on my part, it would rather be a “CameraEntity” type, since it describes "Entity"s with "ICameraSetting"s. These CameraEntitys, instead of the normal Entitiys, could then be implemented by all entities in your game that have some camera related settings.
While this may look very similar to the solution before (and technically it is), this allows you to have lists of CameraEntitiys, which makes sure that the type contains both: Entity related data, as well as Camera related data.
Again, you can make these structures as big as you need. The way to do this always depends on what your problem is.

You also do not necessarily have to make lists of the type of your interface, but could save a list of Entitys and check if they are castable, and if they are access the interface components, like so:

        // I removed the abstract modifier since i'm lazy and couldnt instantiate it otherwise
        Entity ae = new Entity();
        MainCharacter mc = new MainCharacter();

        List<Entity> list = new List<Entity>();
        list.Add(ae);
        list.Add(mc);

        foreach (Entity e in list)
        {
            // Entity-related code here
            var casted = e as CharacterCameraSetting;
            if (casted != null)
            {
                Debug.Log("e is CharacterCameraSetting"); // only gets printed once!
                // CharacterCameraSetting-related code here, using "casted"
            }
        }

If you have a concrete problem you need help with figuring out you can always post it and I, or others, may be able to help you with that specifically.

That’s it with code-specific advices for now, but i have one more thing that may help you. It’s not exactly a lot of fun, but depending on the size of your company and the complexity of the game (which even for small MMOs should be a given), this will make architecture a lot less scary: diagrams. Mostly UML-diagrams.

Class diagrams for example, help you visualizing the contents and relations between classes like so:

That way you have a nice overview and can spot problems before implementing any code. It also allows you to fix problems without thinking about implementation details. Creating these can take a lot (!) of time, especially for a complex project, but it has a huge (!) advantage as well: technically, with a perfect class diagram, you can simply implement the entire architecture in one go, and afterwards only have to think about implementation details (inside functions like “CollectMoney()” in the example).

Flow diagrams can be used to visualize intended workflows, or even algorithms like so:

The same way class diagrams can be helpful for the architecture / structure of your code, these can be useful for the implementation details. I wouldnt make one for every method you plan on implementing, but for complex ones it can definitely help keeping track of when and why stuff is supposed to happen.

And since you are working on MMOs, there are also good diagrams to visualize networking / synchro related code (Sequence diagrams iirc).
TL;DR: it’s definitely worth taking a look at stuff like this as well.

That is definitely true…

Could you clarify what you mean by implemented here? Are you saying that entities like MainCharacter would hold a reference to a CameraEntity? The code you show makes more sense to me.

With regards to the Entity List solution, the issue I have with this, if I understand you correctly, is that using this in a system would make it depend on the entity which could possibly change between games which could break the system… Is that a possibility?

Our team and I have definitely used UML and flow charts before and they are definitely a great tool. Unfortunately we only do it when we start working on a system and never stop to do something rough when we start the game (probably why our data is everywhere!).

Again, thank you for your advice, it’s giving me good insight.

A typo on my end. I meant “inherited”, not “implemented”, sorry. I simply suggested creating more types to prevent the problem that you eventually ran into, where you had a list of type (for example) ICameraSetting, but also required its Entity data. Thus you could simply have a type that inherits from Entity, but implements the ICameraSetting interface and call it CameraEntity, which allows you to have a list of a type (CameraEntity) that can access both Entity data and ICameraSetting functionality. The idea is that you can have as many sub-types as you want to make things easier for you to work with.
The code i showed you was a work-around for not knowing which types “secretly” are more than what you saved them as, if that makes sense. Like, in the example we have our type “MainCharacter”, but save it in a List. As far as the list is concerned, all its elements are of type Entity. So even if we save our MainCharacter object in there, we wont be able to differentiate between that element and other (purely Entity) elements of the list. So if we want to access the CameraSetting functionality of our MainCharacter, we can use the code from the example to only execute code related to camera settings, if that list element of type Entity actually implements the interface we are looking for.

Sorry if i didnt make this clear enough. Hope this helps in understanding what each example was for.
Also, i’m not sure i understand your question with the entity list and other systems. Do you have an example?

And when you say switching between games, do you mean game sessions, or literally two different game projects?
I can see why you would want a shared code basis between projects, but unless it’s absolutely identical for most of your games (like using your own physics calculation library for example), imho there are too many things that need slight adjustments between games.
I can see this causing problems since changes made for one game introduce an overhead (and worse readability and bloated classes) for all other games. If that’s what you are going for tho, then i believe this would actually be a good reason for switching to DOTS, since it would make your architecture less messy. Like, you’d have some of the same systems and components, but can redefine entities on the fly based on that specific games’ needs, without introducing any overhead (since systems work on components, not entities so to speak).
But maybe i’m misunderstanding your intentions here?

All good, but if you have an entity inheriting from the CameraEntity, what happens when u want it to be, say, a SneakEntity too? It made more sense to me when you showed the example of the MainCharacterEntity implementing ICameraSettings because then we can also implement ISneaker (or something).

Different game projects, so for example, if I have a sneak system, I want to use that same sneak system in two different games without the overhead you mentioned. In other words, I don’t want people building dependencies on game specific code into the systems. I want the systems to be ‘Extendable’ like all the gurus say it should be. You know, abide by Open Closed principle and such.

I can understand how switching over to ECS would enforce this architecture of code, however I’m sure it must be achievable using the existing component system. Maybe if I start writing by ECS it will become more apparent to me :,D

Nothing speaks against having tons of interfaces - and in your case this might be a good idea. The MainCharacter example still stands, so to speak. You could still define a type as MainCharacter : Entity, ICamera, ISneak.
However, if you commonly run into the issue that you need, for example, Camera and Entity together, then combining the two into their own type can make sense for your convenience.
It doesnt have to be a type either. C# allows you to inherit interfaces into interfaces. Example for why this may be useful:
Imagine you created 20 or even 100 interfaces. You could define a type as Something : Entity, 1, 2, 3, 6, 12, …, 42, …
but that can potentially get really messy and introduce bad readability. At the same time you may realise that you often use specific interfaces together, so it can make sense to make an interface that simply inherits interfaces 1, 6, and 42. That way you had a type you would commonly use, but still the option to use 1, 6 and 42 on their own.
There are few hard rules for software architecture (you can google for some actual rules tho). So a lot of things you do are simply for your convenience, to improve readability, or the ease of “extendability” of your code.

You are obviously right that you could try to “simulate” ECS using the default component system, however since it’s very specific you’d rewrite most of it, in which case you could simply use it. DOTS does not make use of any GameObjects. It’s an entirely separate system (you can use it as a hybrid with the default Unity workflow tho). In ECS you dont even use any normal objects (classes) anymore. As i said it’s not object oriented anymore, so no objects. “Entities” are the closest you’d get to objects in ECS, but they are still only structs without any overhead. So you could, technically, go ahead and define components as used in ECS, meaning a struct with a single attribute inside. However, you’d still have to save these components as fields of a class, kinda ruining the whole advantage. ECS would automatically run a system that runs on, for example, the PositionComponent, on all entities that have a PositionComponent. Unless i’m overlooking something, i dont think you can simulate this with the default workflow, which means imho that it’d not offer any architectural benefits to do so.

Makes sense i guess. I’d try and design your systems (like the sneak system) as a kind of blackbox. So you’d have a sort of standalone unity asset that comes with all it needs to work. In this case that would be some interface ISneaker, and a script that takes a list of ISneaker as input (or something along those lines). If done perfectly, you could say that this sneak system is its own “Unity asset” - and i would actually export it as one and import it into projects when needed.
As you may have replied next: this potentially results in the problem where two assets rely on the same common group of interfaces (like ICameraSetting or something). For that i’d define another asset containing all the common interfaces, which then gets imported into every project you make, guaranteeing that all “dependencies” are present.
For a system like this i’d highly recomment making detailed UML-diagrams first tho, since for example changing around stuff in the common interfaces later on, may break things and require you to rewrite the depending assets (such as the sneaker asset) as well, which can result in a lot of unnecessary work.

See I just have trouble thinking of a case where I would just have a CameraEntity, unless you’re specifically just using this as an example. But I totally get u with regards to the multiple interface strategy. I think I realise the problem I’m having which I’ll explain at the end.

Oh I just meant maybe trying to replicate some of the structuring that you mentioned it enforces on your code, not so much every detail of the system. Like for example how the one attribute per component kind of makes me think of a small interface which your entity would implement which some system would operate on.

Funny thing, I actually took this approach to the sneak system we use now and it’s probably one of the more stable and flexible systems in the game that I’ve created.

So after mulling over our conversation, I’ve come to the realisation that I haven’t really been thinking of interfaces as a two way street. When I first tried using interfaces, I literally just looked at our entity, and tried to abstract the data/methods into an interface thinking that’s what I was supposed to do. So for example I made an ILevellable interfaces which had a property for the level, and methods for changing the level and an event for when the level changed. I DONT HAVE A SINGLE CLASS IN MY PROJECT THAT EXPLICITLY DEPENDS ON THIS ABSTRACTION. I think I tried to refactor some classes to depend on it instead of the big entity data class, however those classes were often riddled with dependencies to data unrelated to the level code. I think this is probably the big reason that this refactor attempt kinda fell on its head. I probably could have refactored the classes to depend strictly on the data it needed but it would have taken far too long.

The only system I can think of that uses interfaces appropriately is the sneak system.

Anyway I’m sorry I may have no been very coherent, I’m a bit scatter brained when it comes to this code design stuff as you can probably tell, but your advice has been immensely insightful and I appreciate you taking the time to educate me :slight_smile:

Yes, CameraEntity was mostly an example. You should always take what i say as examples or concepts to work with, not as a sort of “definitely do it that way, now!” kind of statement. Keep in mind that my knowledge of your project and situation is limited to the information provided in this thread (some of which i’m probably even forgetting between posts).
That said, i thought CameraEntity might make sense since you once said that you ran into the problem of having lists of your ICamera interface type, but then required the Entity data of those classes as well (or something along those lines).
Still, if a CameraEntity does not make sense, you dont have to use it. Eventually you may run into a situation where something like that makes sense - or you dont and dont need it.

My text was aimed to explain why the first part of this quote is probably not realistically doable. I’m not an ECS expert myself either, since i mostly used Jobs+Burst in my projects so far [***]. If you implement a small example project using ECS, you will probably realise why it’s not easy to use this architecture for object oriented projects.
However, if what you took from it is the second part of the above quote: then go for it. That’s how you are supposed to use interfaces actually - sort of as its own (partial) type, on which executable code can run.

[***]: On that note, i can highly recomment Jobs+Burst, even in object-oriented environments. Meaning, you should probably take a look at it, even if you dont plan on using ECS with it. (C#) Jobs are basically just an “easy” way to get multithreading done, without being able to introduce race conditions or other problems related to multithreading.
Takes a bit of time getting used to it, but it’s safe to use afterwards and (multithreading generally) results in huge speedups. Even more so, since using Jobs for multithreading allows you to compile them using the BurstCompiler, for the aforementioned insane performance gains. Free performance is always great, but it’s especially nice for mobile games, as less time taken computing things on the CPU means less time with a high core clock and load, thus means less battery usage! Needless to say that this is a huge convenience boost for your users. Sadly, while problems such as “has high battery usage” are often noticed in a negative fashion, the opposite is not true. Most people simply wont notice low battery usage, but it’s still there. Which makes me remember an interview i read / heard some time ago. A developer was asked about how he felt that the feature / system he spent a long time developing was not being talked about, to which he just replied something along the lines of: “If nobody talks about this feature that just means it runs without causing anybody problems, which itself is the biggest praise i could get for my system.”. I personally found this very true and inspiring. In the end you (users) only notice implementations when they are off / dont feel good.
But i’m getting off-topic here, so sorry for that.
The entire block was just a thing that came to my mind while writing the above comment, since so far i may have caused the impression that the 3 DOTS technologies can only be used together, which is not the case. So i thought i’d better tell you that you can use Jobs+Burst without ECS. And then it got a bit longer…

A few things that came to my mind while reading your refactoring story:

  • Generally classes (for readability and maintainability reasons) should be a couple hundred lines of code at max. Having more than say 200-300 lines of code often indicates that a class is doing “too much” and that it can/ should be split into multiple smaller classes. That said, from what i heard, in (object oriented) game development you often run into the problem that your main Character/Entity class grows way beyond that. Since i did not personally work on a big NPC or Item system myself yet, i cant really say if this is reasonably avoidable or not. That said, large classes decrease maintainability and thus increase the time spent working on them, which should be prevented whenever possible. I remember a (university) project where my group had a nearly 14.000 lines database management class. It was a living nightmare, i tell you! I later split it into about 40 different classes. Took forever. Which brings me to my next point:
  • Refactoring a project later down the road takes a lot of time and is little fun. So i can definitely see how you ended up creating interfaces that were not actualy required. When planning a new project top-down (best with class diagrams) you can easily prevent this, but when there is some huge class you want to split up, then you are already bound by your knowledge of “how things work now” and end up focussing more on just splitting the code, than to actually create meaningful types.

I’m assuming you plan on doing more refactoring in the future, which possibly was also the reason for this thread. If so, and if we can agree that having systems designed as independant black-boxes (in your situation) is the best solution, then i’d say the best approach may be to (re)design each system as a black box, top-down on paper (using diagrams) first. And when you are done and happy with the result, only then start changing the code to “exract” the old system (or create the new one) and then refactor the code to make use of the new blackboxed system.
I would imagine that your code would end up in a pretty good shape after you are done doing this for all involved systems.

As a final word: dont worry about asking questions or feeling dumb or anything. You are not. This has actually become one of my favorite threads so far. It’s always great to see somebody who wants to actually improve the way they are doing things. Often in forums people just want a quick and possibly dirty solution to some bug they encountered (which also is fine i guess), so this is refreshing.

Hey sorry for my super late reply. I’ve been super busy with work (we’re releasing our game soon) and sick at the same time >.>

I did a bit more research on DOTS and ECS and found it very interesting that even big games like World of Warcraft and Dota 2 are build on top of data oriented systems. Maybe not ECS specifically, but something within that realm, so it must be testament to the pattern for games development. I think there’s definitely a lot of operations in our game that could be done via the job system. Our games are currently using a lot of battery power struggle to run >30 frames on lower end mobile devices (S4 is our test phone) and optimising the CPU performance would definitely help this problem since it’s not exactly a GPU bottleneck. I’m starting on my own project so perhaps I’ll give it a go implementing one of the domains like AI or something on the DOTS.

See our projects seem to move so fast, we don’t necessarily go back and refactor stuff. But I’m definitely starting to feel the tech debt of that neglect as our mmo’s get bigger and bigger. The time it takes to implement a feature truely grows exponentially when u don’t clean up after yourself. Fortunately we’ve never written 10K line classes! The worst class we have in our project right now is a playercontroller type class that was taken from the prototype of a game another team was working on, and it was a nightmare! We could have spent the day cleaning it up, but we we’re stupid and just moved forward :,(

Yes that’s very true.

Actually it’s more so for future projects :,D I don’t want to run into the same problems over and over again.

I have started taking this approach to my new personal project! It’s much easier to see where some systems start to interact with each-other so I can start to planning the data structures :slight_smile:

I’m glad you’re getting joy from this :,D It is a tendency of mine to ask a question about a problem I’m having when I don’t really understand the problem completely or its a bigger, more complex problem that I thought. It sometimes feels like what I’m saying is a bit of mumbo jumbo but I’m glad you understand me!

Most certainly not DOTS since that got invented by and for Unity as far as i know. However, the concepts involved, such as data oriented programming hve been around for a lot longer, and massive studios / publishers have teams worrying only about performance, which often involves keeping efficient memory layout in mind.
So it’s not surprising to see that well-optimized titles made use of it. What’s new with DOTS is the ability for small (indie) teams to make heavy use of multithreading and efficient memory layout, without having to worry about it or employ entire teams to worry about efficiency.

Yeah as i said earlier, it should help immensely with battery or (cpu-bound) framerate problems. Just as an example, the algorithm i optimised for a personal project (marching cubes for procedural world generation) ran singlethreaded in like 180ms on my system. Now i split it into multiple ParallelFor jobs that already ran in 2-10ms each, for a total of ~30ms, reflecting the advantage of switching from single to multithreading. Then with Burst Compilation enabled each of those jobs runs in about 0.2 - 0.5ms, which is a speedup of 10x-20x (and a total execution time of the algorithm in <~3ms). The Burst Compiler really feels like magic sometimes haha.
So if your framerate is bound by heavy cpu tasks, then this will definitely help. And while i’m not entirely sure how this would be reflected in battery usage, the improvements should be night and day as well.

Last but not least: I hope you get better soon. Wish you a nice day!