Basic Data Oriented Design Question

Hey ECS experts would like to make a simple simulation of the EVE Online Universe as a way to learn DOTS, but am having trouble taking the first step with Data Oriented Design. I’ve got about 5500 solar systems that can be traversed by ships and owned by factions. I’d like to eventually have 1000’s of ships (or fleets of ships) mining, hauling, and fighting (in simple abstracted ways). Just starting with the solar system data, in object oriented world, I’d have a Dictionary of Solar System objects like:

Class SolarSystem()
{
    public string name;
    public int systemID;
    public Vector3 Position;
    public int [] ConnectionIDs;
    public float securityStatus;
    public float[] ResourceAbundance;
    public int factionID;
    public int planetCount;
    public int moonCount;
    public int asteroidCount;
    public int parentConstellationID;
    public int parentRegionID;
}

My first hack at breaking this into components and entities would be:

public struct GalacticPosition : IComponentData
{  
    //as opposed to position within a solar system, which probably should be a different coordinate system.
    public float3 Position;
}
public struct SolarSystemConnection : IComponentData
{  public Entity[] connectedSystems;  }
public struct SolarSystemContents : IComponentData
{
    public int planetCount;
    public int moonCount;
    public int asteroidCount;
    public float[] ResourceAbundance;
}
public struct FactionOwnership : IComponentData
{  public Entity faction;  }
public struct SecurityStatus : IComponentData
{  public float securityStatus  }
public struct ParentConstellation : IComponentData
{  public Entity parentConstellation;  }
public struct ParentRegion : IComponentData
{  public Entity parentRegion;  }
public struct ObjName : IComponentData
[  public string name; }

Does this seem like a reasonable conversion? This data would be used in systems like intersteller ship movement, A* pathfinding from one system to another, AI deciding which systems to mine in, AI factions deciding which systems to conquer, etc. The data in these components wouldn’t change (except factionID). Any feedback or constructive criticism would be very welcome! Thank you!

The arrays are something you need to consider the conversion for. Data with dynamic size (collections like arrays, lists, dictionaries, etc.) should be converted to dynamic buffers or possibly normal components with fixed lists (from the Collections package) or blob assets. Data can also be placed in native containers, but these must be created at runtime. They aren’t safe to put on component types that get iterated over in jobs, and are generally expected to just be used in singletons and passed to jobs.

Data Type Category Bakeable Entity Remapping Usable in Components Iterated in Jobs Locality Memory Management
Native containers No No No Outside chunk Manual
Fixed lists Yes No Yes In chunk n/a
Blobs Yes No Yes Outside chunk Automatic when baked
Dynamic buffers Yes Yes Yes In chunk when at/below internal buffer capacity Automatic
1 Like

It’s difficult to judge whether this is a good decomposition of your data, because it depends almost entirely on what your data access patterns will look like. You’ve kind of sketched out some examples of how you want to use the data (ship movement, pathfinding, etc) but it’s kind of vague. What systems are you going to have and what data will they want to access?

For example, right now you have ParentRegion as an IComponentData, which means that a single chunk can contain solar systems from any regions packed together. Does that actually fit your usage, or would you be better off with it as an ISharedComponentData, so that all systems in a chunk are in the same region? Similar question for faction ownership, constellation, etc.

@Spy-Master Thanks for the response! Sounds like for my connectionIDs, which will have fewer than 10 elements and will not change after initialization, the fixed list would be best. Then, the dynamic buffer for things like pathfinding waypoints. The way I’d previous built things, I had a big dictionary of solar systems objects, each of which had an id int as a key (hence the connectionID being an int. In ECS, would it make more sense to have a fixed array of entities (since they won’t undergo structural changes) instead of an int ID value?

@RichardFine Thank for the feedback sir! As someone who’s just a tinkerer, not an engineer, it sounds like the right workflow to first plan out as many systems as I can, then plan out the data decomposition? These were the first systems I’ve been thinking about for very basic functionality:

  • FleetBehaviorUtilitySystem - calculate best course of action using UtilityAI if interstellerMovement tag is disabled. Result enables corresponding behavior tag.
  • FleetBehavior_MoveToTargetSystem - move towards exit gate leading to next pathfinding waypoint
  • FleetBehavior_IdleSystem - wait in random location with solar system
  • FleetBehavior_FleeSystem - move towards system gate leading back to owned territory. Send support request to other owned fleets
  • FleetBehavior_AttackSystem - move to orbit around enemy target. Fire weapons based on range and cooldown
  • InterstellarMovementSystem - move entities with enabled interstellerMovement tag form an origin to a destination system. Disable tag once arrived
  • SetFleetTargetSystem - based on a faction’s conquest target and whether its own solar systems are under attack, ensure all owned fleets have a target system. Setting a new target triggers a pathfinding job.
  • SetFactionConquestTargetSystem - if faction has no target (initialization, after successful conquest, or an owned system has been conquered) evaluate adjacent solar systems based on resources available and ownership status (e.g. owned by enemy faction)

This is just basic combat behavior. Eventually I’d apply similar logic to mining, hauling, etc. Also might be necessary to abstract all the in-solar system behavior unless the user zooms into one. Then I could pause the high level fleet agent and simulate individual ships in that system alone (like what I assume Total War does on the campaign map vs a battle the user fights). Thanks again!

Yes and no. You want a design which is based around the concrete needs of your project, rather than something abstract, so it’s helpful to understand those needs. At the same time, you don’t want to be too ‘waterfall’ about it, especially if you haven’t already prototyped your gameplay; there will doubtless be needs that you can’t discover until you’ve already implemented some of this stuff. So, expect the design to change as you learn more things.

These read to me more like states in a state machine, rather than ECS systems. Try to think about it more in terms of transformations on the data. For example:

  • What is FleetBehavior_IdleSystem actually modifying about a ship?
  • FleetBehaviour_MoveToTargetSystem, FleetBehaviour_FleeSystem and FleetBehaviour_AttackSystem all involve moving a ship towards a target position (I assume within one solar system). Perhaps there should be one system responsible for “update position to be closer to target position;” with the differences between flying to another system, fleeing, and attacking, just being how the target position is selected?
2 Likes

Thanks again for the thoughtful response! So might a better configuration be a SelectTargetSystem, MoveToSystem, and a ShootStuffSystem? Following Eve’s logic, ships can move interstellar, interplanetary, and on the local grid. I was tracking this in different coordinate systems so the user can zoom to different levels of detail. The MoveToSystem could change the exact logic based on an enablable tag I guess?

One complicating factors is I’m trying to build in time compression into this simulation. So each frame could represent anywhere from .02 seconds up to 80 seconds (if we’re shooting for a max simulation speed of one hour of simulation time per second). The higher the time compression the more simplified/abstract the behavior. Since the delta time can be so high, I’m trying to allow execution of multiple states in a single tick (e.g. an entity, near the end of its interstellar transit, only requires 30 seconds to complete the transit. So it can spend the rest of the delta time beginning to execute it’s next state). So if I have a single MoveToSystem, I’m not sure how I’d account for that case where a target is reached part way through the simulated tick. Any thoughts? Thanks again!

Possibly, yeah.

Or alternatively, have InterstellarMoveToSystem, InterplanetaryMoveToSystem, and LocalGridMoveToSystem, where each one operates on a different position component (where each position component is in a different coordinate system).

Hmm, tricky. Do you expect these “mid-tick” state changes to happen often? Do ships change state all the time, or would it happen in bursts? Can an entity go through more than 2 of those states in a single frame?

I’ve not solved this problem before so I can’t tell you the ‘right’ way to do it, but here’s two ideas off the top of my head:

  • ‘Split the tick’: calculate (without changing any data) that you’re going to have a state change 30s into your 80s tick, and then do a 30s tick followed by a 50s tick. Repeat recursively, i.e. potentially split the 50s tick into smaller ticks as well if further state changes are going to happen there. Could be problematic if you expect to have many state changes happening at different points across a single frame as you could end up ticking the simulation many times and either having a very high frametime or having the user see that the simulation ‘slowed down’ from the expected 80s/tick.
  • Give the ship a ‘time credit’: if it finished its movement 30s into an 80s tick, then change it to the new state but also give it 50s of ‘credit’ for its next tick, i.e. when you give the ship an 80s tick the next frame, it moves based on a tick size of (80+50)s. This doesn’t work well if the ship might have more than one state change during a tick, though, and it can make it complicated for the ships to interact with each other around the state changes because a ship which has just changed state will be ‘lagging’ behind its true position until the next frame.

Maybe one of those thoughts inspires something… or maybe someone else has built this kind of thing before and can tell us the right way to do time compression at scale :slight_smile: