Scheduling multiple SystemBase descendants in deterministic order

Thygrrr · July 7, 2020, 2:10pm

Hello, I got a duo of Questions:

how do I properly and deterministically Schedule() or ScheduleParallel() several SystemBase descendants so each one runs when its predecessor is done?

Then, all of them need to finish within their System Group, i.e. before another SystemGroup starts (e.g. PresentationSystemGroup)?

It seems like UpdateBefore and UpdateAfter don’t amount to much in this context. But how do I communicate a dependency across systems?

how do I schedule the same chain of systems multiple times (e.g. if it has to catch up on some work, or multiple work items arrived at once, but the need to be consumed in strict chronological order)

Thygrrr · July 7, 2020, 2:21pm

// only Entites.ForEach and Job.WithCode are handled by code-gen
Dependency = new SomeJob().Schedule(Dependency)
Entities.ForEach(/*some stuff */).Schedule();
Entities.ForEach(/*more stuff*/).Schedule();

So I see stuff recommended like this, that seems like it encourages huge god-systems “MyWorldSimulationsSystem”, with Jobs and Foreach Lambda blocks as the constituents.

I think I’d rather have somewhat finer granularity with individual systems, which still co-depend?

brunocoimbra · July 7, 2020, 2:44pm

For that, I see 2 alternatives:

a. Use UpdateAfter and UpdateBefore to ensure the correct order of your systems and let the Unity dependency system decide which systems should wait for another system to finish (it is based only on the components used, so if you have dependencies to native collections or custom jobs you will need to track those dependencies manually).

b. If you really want to run your systems sequentially (even when there is no dependency between them) then .Schedule() and .SchedulleParallel.() aren’t going to be of much help, so use .Run() instead, or at the beginning of each system’s OnUpdate do Dependency.Complete();

Not sure if following you on that one, if you want to schedule the same chain multiple times on the same frame, I recommend you to move your logic into jobs instead of systems and schedule those jobs in the same chain as you wish. If you just want to ensure that your systems runs in the correct order, see my 1.a. above.

brunocoimbra · July 7, 2020, 2:51pm

Thygrrr:

// only Entites.ForEach and Job.WithCode are handled by code-gen
Dependency = new SomeJob().Schedule(Dependency)
Entities.ForEach(/*some stuff */).Schedule();
Entities.ForEach(/*more stuff*/).Schedule();
So I see stuff recommended like this, that seems like it encourages huge god-systems “MyWorldSimulationsSystem”, with Jobs and Foreach Lambda blocks as the constituents.

I think I’d rather have somewhat finer granularity with individual systems, which still co-depend?

Well, I do recognize that code sample and is totally taken out of context as it was not meant to encourage any kind of god-systems.

It is fairly common to have jobs to share code across different systems, not to mention work that is not dependent on entities nor components thus making no sense to use Entities.ForEach. Also, some systems need to run multiple ForEachs for different reasons (ie. filter data from one query then use that data within another query).

Thygrrr · July 7, 2020, 3:51pm

Thank you for the reply!

a. I don’t understand - simple example case, I have 2 systems, one that writes a component, and another that will read that component and make a decision based on this (e.g. duck or shoot, the wrong action would have drastic consequences)

Is it enough to use the UpdateBefore Attribute?

So the UpdateBefore attribute doesn’t only cause the system to be started to Update before, but also to Complete said Update before?

But that’s the question! How do I get an outside Dependency INTO a system?

Interesting. I’m thinking about a specific type of client-server model; where a substantial portion of the gameplay simulation is deterministic and kept perfectly in sync across both machines, and only the command streams are communicated back and forth. (i.e. player inputs)

I’d build a server that takes commands in, applies them to its own simulation, and then relays them to all clients with a timestamp as to when (or rather, where) to apply it to their simulations.

Given the nature of the Internet, that means the client could receive any number of “Tick” packets, each containing multiple commands. The client may want to process 0, 1, or more of these in any given frame. (“usually” 0, assuming the game runs at 144 fps and the command stream is 15 to 30 datagrams per second). It gets a bit dicier when the client can barely run 40 frames reliably.

(Yes, the simulation on the client may need to freeze when the local Tick buffer runs out. it’s gonna be fine.)

brunocoimbra · July 7, 2020, 5:12pm

Yes!

In the same way as you already do that with the physics system, you need to expose and handle them manually, like:

public class MySystem : SystemBase
{
    public void AddDependency(JobHandle inputDependency)
    {
        Dependency = JobHandle.CombineDependencies(Dependency, inputDependency);
    }

    public JobHandle GetDependency()
    {
        return Dependency;
    }

    protected override void OnUpdate()
    {
        // Do stuff
    }
}

public class MyOtherSystem : SystemBase
{
    private MySystem _mySystem;

    protected override void OnCreate()
    {
        _mySystem = World.GetExistingSystem<MySystem>();
    }

    protected override void OnUpdate()
    {
        // make this system dependent on mySystem manually
        Dependency = JobHandle.CombineDependencies(Dependency, _mySystem.GetDependency());

        // make mySystem dependent on this system
        _mySystem.AddDependency(Dependency);
    }
}

For the last part I can’t really help much, didn’t touch in anything networked with DOTS yet unfortunately.

sngdan · July 7, 2020, 5:34pm

I have not been toying around with DOTS for quite some time but the first YES above is not correct in the standard setup, unless something has changed.

Standard = you order systems [update before, etc] which controls main thread update order and job execution is managed by unities dependency system (not necessarily in the update order)

simple dependency = dependency is managed by the update order. In this case the update order also sets the dependency order. You have to set this in the bootstrap if I remember correctly

brunocoimbra · July 7, 2020, 5:41pm

I think you missed the context here:

So yes, UpdateBefore is enough to make it Complete before another system which relies on the same component it writes too. You can see it in the official docs btw: https://docs.unity3d.com/Packages/com.unity.entities@0.11/manual/ecs_job_dependencies.html

Thygrrr · July 8, 2020, 10:27am

Thanks everybody, this is very helpful.

Thygrrr · July 10, 2020, 8:30pm

… however, it does not seem to be true that UpdateBefore and UpdateAfter take care of dependencies at the end of a system’s scheduled work, and instead only determine the “start” point of a system.

I’m using pretty much exactly this pattern of a random system that has one mathematics random struct per thread.
https://reeseschultz.com/random-number-generation-with-unity-dots/

I have two systems, DummyFiring and ExecuteFire both access the randoms array from the system. ExecuteFire uses
BeginSimulationEntityCommandBufferSystem and EndSimulationEntityCommandBufferSystem, DummyFiring only
EndSimulationEntityCommandBufferSystem.

Both systems use an Entities.Foreach.WithNativeDisableParallelForRestriction to access the randoms array and write back to it.

Class DummyFiring has the attribute [UpdateBefore(typeof(ExecuteFire))]

When I mash the fire button, sooner or later I get this exception, which tells me that the UpdateBefore didn’t do squat and (of course…) the systems are oblivious to all other systems around them:

InvalidOperationException: The previously scheduled job DummyFiring:<>c__DisplayClass_OnUpdate_LambdaJob0 writes to the Unity.Collections.NativeArray`1[Unity.Mathematics.Random] <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.randoms. You are trying to schedule a new job ExecuteFire:<>c__DisplayClass_OnUpdate_LambdaJob0, which writes to the same Unity.Collections.NativeArray`1[Unity.Mathematics.Random] (via <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.randoms). To guarantee safety, you must include DummyFiring:<>c__DisplayClass_OnUpd
Unity.Entities.JobChunkExtensions.ScheduleInternal[T] (T& jobData, Unity.Entities.EntityQuery query, Unity.Jobs.JobHandle dependsOn, Unity.Jobs.LowLevel.Unsafe.ScheduleMode mode, System.Boolean isParallel) (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/IJobChunk.cs:216)
Unity.Entities.JobChunkExtensions.ScheduleSingle[T] (T jobData, Unity.Entities.EntityQuery query, Unity.Jobs.JobHandle dependsOn) (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/IJobChunk.cs:111)
Jovian.Systems.Weapons.ExecuteFire.OnUpdate () (at Assets/Jovian/Systems/Weapons/Firing.cs:38)
Unity.Entities.SystemBase.Update () (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/SystemBase.cs:414)
Unity.Entities.ComponentSystemGroup.UpdateAllSystems () (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/ComponentSystemGroup.cs:445)
UnityEngine.Debug:LogException(Exception)
Unity.Debug:LogException(Exception) (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/Stubs/Unity/Debug.cs:19)
Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/ComponentSystemGroup.cs:450)
Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/ComponentSystemGroup.cs:398)
Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/ComponentSystem.cs:109)
Unity.Entities.DummyDelegateWrapper:TriggerUpdate() (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/ScriptBehaviourUpdateOrder.cs:192)

Both update in SimulationSystemGroup. What am I doing wrong?

brunocoimbra · July 10, 2020, 9:04pm

Unity only track dependencies automatically for Components, as I informed before:

You have a NativeArray in the middle of the process, so you will need to track that dependency manually, like I mentioned too:

brunocoimbra:

public class MySystem : SystemBase
{
    public void AddDependency(JobHandle inputDependency)
    {
        Dependency = JobHandle.CombineDependencies(Dependency, inputDependency);
    }

    public JobHandle GetDependency()
    {
        return Dependency;
    }

    protected override void OnUpdate()
    {
        // Do stuff
    }
}

public class MyOtherSystem : SystemBase
{
    private MySystem _mySystem;

    protected override void OnCreate()
    {
        _mySystem = World.GetExistingSystem<MySystem>();
    }

    protected override void OnUpdate()
    {
        // make this system dependent on mySystem manually
        Dependency = JobHandle.CombineDependencies(Dependency, _mySystem.GetDependency());

        // make mySystem dependent on this system
        _mySystem.AddDependency(Dependency);
    }
}

Thygrrr · July 14, 2020, 9:20pm

I still can’t get it to work.

I have SystemA and SystemB.

I want to always run B after A (they access the same data structures inside collisionworld doing raycasts).
Both must execute between BuildPhysicsWorld and StepPhysicsWorld (meaning they update always after BuildPhysicsWorld has finished, and finish their jobs before StepPhysicsWorld begins its work.)

My dream solution would be a ComponentSystemGroup I could put them into that ensures at least this “sandwiching”.

I created a ChainableSystem base class that’s exactly what “MySystem” does in bruno’s example above this post.

[UpdateInGroup(typeof(ABSystems))]
class SystemB : ChainableSystem
{
    protected override void OnUpdate()
    {
       Dependency = Entities.Foreach(...)
       .Schedule(_physicsWorldSystem.GetOutputDependency());
 
        _stepPhysicsWorldSystem.AddInputDependency(Dependency);
    }
}

//separate file ...

[UpdateInGroup(typeof(ABSystems))]
class SystemA : ChainableSystem
{
    protected override void OnUpdate()
    {
       Dependency = Entities.Foreach(...)
       .Schedule(_physicsWorldSystem.GetOutputDependency());

        _systemB.AddDependency(Dependency);                 
        _stepPhysicsWorldSystem.AddInputDependency(Dependency);
    }
}

It never works. Interestingly or absurdly enough, unity complains that SystemB’s Job is previously scheduled before SystemA. Why?

I also don’t understand the "Dependency = " magic syntax, when I step into the property it just writes the job handle and does no combining. I’ve tried to manually combine into a temporary dependency variable, but the error stays the same:

InvalidOperationException: The previously scheduled job SystemB:<>c__DisplayClass_OnUpdate_LambdaJob0 writes to the Unity.Collections.NativeArray`1[Unity.Physics.RigidBody] <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.cw.m_Bodies. You are trying to schedule a new job SystemB:<>c__DisplayClass_OnUpdate_LambdaJob0, which reads from the same Unity.Collections.NativeArray`1[Unity.Physics.RigidBody] (via <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.cw.m_Bodies). To guarantee safety, you must include CollectTargets:<>c__
Unity.Entities.JobChunkExtensions.ScheduleInternal[T] (T& jobData, Unity.Entities.EntityQuery query, Unity.Jobs.JobHandle dependsOn, Unity.Jobs.LowLevel.Unsafe.ScheduleMode mode, System.Boolean isParallel) (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/IJobChunk.cs:216)
Unity.Entities.JobChunkExtensions.ScheduleSingle[T] (T jobData, Unity.Entities.EntityQuery query, Unity.Jobs.JobHandle dependsOn) (at Library/PackageCache/com.unity.entities@0.11.1-preview.4/Unity.Entities/IJobChunk.cs:111)

Thygrrr · July 14, 2020, 9:46pm

The error goes away if I don’t place my systems in a

 public class ABSystems : ComponentSystemGroup {}

However, now the systems run before BuildPhysicsWorld… (requirement is after, and I literally set both my systems’ dependencies to include the output dependency of BuildPhysicsWorld)

I have other groups that do this just fine. (and actually, TargetingSimulationGroup == ABSystems, and the systems in question are CollectTargets and RayCastTargeting - in fact, they conflict semantically with OrientAimingAxis, so seeing them interleaved like that is a bit vexing, as they should live in TargetingSimulationGroup, which taunts me by sitting right where I want it to execute)

brunocoimbra · July 14, 2020, 10:17pm

you put the [UpdateAfter(typeof(BuildPhysicsWorld))] and [UpdateBefore(typeof(StepPhysicsWorld))] attributes in your group ABSystems?

If you want to execute SystemB after SystemA you should also put [UpdateAfter(typeof(SystemA))] in SystemB (or [UpdateBefore(typeof(SystemB))] in SystemA, or call their Update() manually in the ComponentSystemGroup).

Also, there is no magic involved with the Dependency property, the only “magic” is that you can do: Entities.ForEach(...).Schedule() instead of Dependency = Entities.ForEach(...).Schedule(Dependency);. Every other case you need to manage this property (Dependency) manually as you would with the inputDeps of JobComponentSystem.

Also, remember that the Dependency property is the one containing the system’s dependencies information (as the same suggests), in both your SystemA and SystemB you are forgetting to combine the Dependency with your _physicsWorldSystem.GetOutputDependency().

The following code should work (I don’t know exactly what is happening in your ForEach or what a ChainableSystem is, so if it doesn’t work then provide the full code to help me to help you):

[UpdateAfter(typeof(BuildPhysicsWorld))]
[UpdateBefore(typeof(StepPhysicsWorld))]
class ABSystems : ComponentSystemGroup { }

[UpdateInGroup(typeof(ABSystems))]
class SystemA : ChainableSystem
{
    protected override void OnUpdate()
    {
       Dependency = JobHandle.CombineDependencies(Dependency, _physicsWorldSystem.GetOutputDependency());
       Entities.Foreach(...).Schedule();
        _stepPhysicsWorldSystem.AddInputDependency(Dependency);
    }
}

[UpdateInGroup(typeof(ABSystems))]
[UpdateAfter(typeof(SystemA))]
class SystemB : ChainableSystem
{
    protected override void OnUpdate()
    {
       Dependency = JobHandle.CombineDependencies(Dependency, _physicsWorldSystem.GetOutputDependency());
       Entities.Foreach(...).Schedule();
       _stepPhysicsWorldSystem.AddInputDependency(Dependency);
    }
}

EDIT: Of course this is considering that SystemB’s ForEach uses some of the same components that SystemA uses, which will cause the SystemB’s Dependency to already have SystemA’s Dependency “injected” under the hood.

Thygrrr · July 14, 2020, 10:24pm

Thank you very much. It seems to work for now I’m very grateful.