I have a question about whats a safe way/best practice of accessing non-ECS auxiliary services within ECS systems. The intention is simply having data which resides in native containers besides entities and their components. It is particularly mentioned in the DOTS Best Practices Guide that this can be useful:
Unfortunately it is not mentioned how this should be done. It may depend on the requirements, I guess. So this are my requirements:
Access by ECS Systems - The auxiliary services needs to be accessed by ECS systems only. The ECS systems may be managed or unmanaged.
Singletons - Each auxiliary service exists only once, so it can share its data to all ECS systems.
Burst - The auxiliary services can be fully burst compiled and can be used within burst compiled unmanaged systems.
Scheduling/Threading - The auxiliary services can be used within scheduled jobs even in parallel.
Safety - The auxiliary services can be accessed safely even when the threaded job using it is running long over multiple frames.
Out of scope here is the following issue:
Thread safety within the auxiliary services
What I especially want to know is how to create, provide and access such auxiliary services from within regular ECS systems. Are there any best practices? What are the approaches you are using in your projects?
Everyone does it differently. Some will build up custom tooling. Some will just adhere to strict conventions. And the more experienced with DOD will do it differently based on the specific service, because they will analyze the data flow requirements and build a solution that minimizes expensive operations.
If you want a more in-depth answer, perhaps you could define your “auxiliary service” in detail?
Define what you’re understanding by aux services.
If its ads / analytics / something external:
As long as its a system that does not stall job chain (see as SimulationGroup) its fine.
Process results when all job systems are done running.
For example, I’ve got a separate AfterSimulationGroup that runs after SimulationGroup but before PresentationGroup. This allows inserting managed (SystemBase) systems afterwards to it to avoid such stalls / force completes.
If its a lookup system of some sorts with a bunch of native containers:
Anything that works - works.
Personally I’m using structs that are obtained via property from managed systems (to reduce headache from SystemHandle refs). Con is manual dependency management when using container systems without entities.
Basically:
Define your containers inside the system to manage lifetime;
You can as well make data struct an IDisposable to handle container lifecycle from itself as : IDisposable.
Define a temp data struct (e.g. with required containers);
Define a property inside container system that generates a new lookup ref data ;
[adds references to the container to the SomeLookupData];
Expose lookup system’s Dependency [e.g. as Deps property] + add “AddWriteDependency” method. Which combines lookup system deps with passed parameter;
When system wants to read from such lookup:
Grab ref to the lookup system; (easier for the SystemBase)
Combine dependency from lookup system with read system dependency;
Optionally - complete it if its main thread system.
Grab from lookup system via property;
Schedule & process required data, its safe to read now.
If system wants to write as well:
Add system dependency via AddWriteDependency to the lookup system for safety for other system to chain properly
My first two usecases/auxiliary services are these, but more, which I don’t have in mind yet, will certainly follow:
Graph of the level used for pathfinding. The data for this aux service is built once at the start of the level and thereafter it is readonly for all ECS systems.
Proximity/Distances between units in the level. The game needs to determine the nearest units to a specific unit to decide the units behavior. Therefore the data has to be updated every simulation step.
So these two are lookup data for other ECS systems.
I used the lookup approach xVergilx described to build some example of it using my second usecase. Used ISystem instead of SystemBase to get the full Burst compilability. Tested at least the flow (Burst enabled) without really using data in the native containers and only dummy data for the queries. Looks like this. If I made some important mistakes I appreciate some feedback:
Example for proximity/distances
public struct ProximityData
{
public NativeArray<int> SomeArray;
public NativeHashMap<int, int> SomeHashMap;
}
[BurstCompile]
public static class ProximityProcessing
{
[BurstCompile]
public static void FindOtherUnitsOrderedByDistance(in ProximityData proximityData, int sourceUnit, out NativeArray<int> proximityResults)
{
//...
proximityResults = default;
}
}
/// <summary>
/// System using proximity data.
/// </summary>
[BurstCompile]
public partial struct ProximityUsingSystem : ISystem
{
private SystemHandle _auxiliaryProximitySystemHandle;
private EntityQuery _someQuery;
/// <inheritdoc />
public void OnCreate(ref SystemState state)
{
this._someQuery = SystemAPI.QueryBuilder()
// ...
.Build();
this._auxiliaryProximitySystemHandle = state.WorldUnmanaged.GetExistingUnmanagedSystem<AuxiliaryProximitySystem>();
}
[BurstCompile]
public void OnUpdate(ref SystemState state)
{
// For an unmanaged ISystem don't know how to get the dependencies without using the system state
ref var auxiliaryProximitySystemState = ref state.WorldUnmanaged.ResolveSystemStateRef(this._auxiliaryProximitySystemHandle);
// The reference to the system struct for getting the data
ref var auxiliaryProximitySystem = ref state.WorldUnmanaged.GetUnsafeSystemRef<AuxiliaryProximitySystem>(this._auxiliaryProximitySystemHandle);
var job = new ProximityUsingJob()
{
ProximityData = auxiliaryProximitySystem.ProximityData
};
state.Dependency = job.Schedule(this._someQuery,
JobHandle.CombineDependencies(state.Dependency, auxiliaryProximitySystemState.Dependency));
auxiliaryProximitySystem.AddWriteDependency(ref auxiliaryProximitySystemState, state.Dependency);
}
[BurstCompile]
private partial struct ProximityUsingJob : IJobEntity
{
[ReadOnly]
public ProximityData ProximityData;
[BurstCompile]
public void Execute(Entity entity /*, more parameters */)
{
int sourceUnit = 17;
// ...
ProximityProcessing.FindOtherUnitsOrderedByDistance(this.ProximityData, sourceUnit, out var proximityResults);
// ...
}
}
}
/// <summary>
/// System for providing proximity data.
/// </summary>
[BurstCompile]
public partial struct AuxiliaryProximitySystem : ISystem
{
private NativeArray<int> _someArray;
private NativeHashMap<int, int> _someHashMap;
private EntityQuery _proximityUnitQuery;
public ProximityData ProximityData =>
new()
{
SomeArray = this._someArray,
SomeHashMap = this._someHashMap,
};
[BurstCompile]
public void AddWriteDependency(ref SystemState state, JobHandle job)
{
state.Dependency = JobHandle.CombineDependencies(state.Dependency, job);
}
public void OnCreate(ref SystemState state)
{
this._proximityUnitQuery = SystemAPI.QueryBuilder()
// ...
.Build();
this._someArray = new(100, Allocator.Persistent);
this._someHashMap = new(100, Allocator.Persistent);
}
public void OnDestroy(ref SystemState state)
{
this._someArray.Dispose();
this._someHashMap.Dispose();
}
[BurstCompile]
public void OnUpdate(ref SystemState state)
{
var job = new UpdateProximityJob()
{
SomeArray = this._someArray,
SomeHashMap = this._someHashMap,
};
state.Dependency = job.Schedule(this._proximityUnitQuery, state.Dependency);
}
[BurstCompile]
private partial struct UpdateProximityJob : IJobEntity
{
public NativeArray<int> SomeArray;
public NativeHashMap<int, int> SomeHashMap;
[BurstCompile]
public void Execute(Entity entity /*, more parameters */)
{
// Update SomeArray and SomeHashMap here
}
}
}
Yeah something like that. Though I prefer SystemBase for this case.
Much less typing when attempting to grab data from the system.
Alternatively what is technically possible - is to put [unsafe] containers inside the actual IComponentData [since they are just pointers]. That way if you query component data, you’ll get automatic dependency management. That is if containers aren’t touched outside of systems processing. Although I haven’t tried that yet with 1.0 and still need to figure best way to do it.
That would definitely be a nice approach therefore. I am having concerns when using pointers in scheduled/threaded jobs, because the data pointed may be reallocated while the job is running. But I really don’t know how the memory management works here. But when you find a working approach let us now.
For (1) I would use a blob asset if it is known at bake time, and just store that in an Entity. Otherwise I would do what I do for (2).
For (2) most people would choose singletons for this. Personally, I have my own pair of tech for this in my framework (Collection Components + Blackboard Entities) which handle dependencies automatically.
Yeah, having something like directly attached persistent native containers inside IComponentData with automatically managed lifecycle & dependencies would be great to have in base Entities.
That is without abusing DynamicBuffers;