Large overhead of IJobParallelForTransform.Schedule

Entity Component System (ECS)
, ,
1

I am using Job system to update many gameobjects’ transforms. I noticed that sometimes IJobParallelForTransform.Schedule reaches a overhead of milliseconds. The profiler showed that Schedule was blocked by worker thread. It looks like Schedule() was waiting for the job to finish.

Example codes of my job manager:

void LateUpdate()
{
    handle2.Complete();
    if (resultArray.IsCreated) resultArray.Dispose();
    resultArray = new NativeArray<Data>(dataLength, Allocator.TempJob, NativeArrayOptions.UninitializedMemory);
    var job1 = new Job1 { /*...*/ };
    var job2 = new Job2 { /*...*/ };
    Profiler.BeginSample("schedule");
    handle1 = job1.Schedule(accessArray1);
    handle2 = job2.Schedule(accessArray2, handle1);
    Profiler.EndSample();
}
2

My profiler looks like this:

9188147--1280414--upload_2023-8-2_16-31-54.png
9188147--1280414--upload_2023-8-2_16-31-54.png749×641 29.5 KB

The Blue block is at Main Thread and Green Blocks are at the worker threads.

3

If its IJobParallelForTransform - its most likely a TAA marshalling cost. Bigger TAA length - higher cost.
There’s nothing can be done about it. You won’t receive much benefits if job is really simple.

What you want to do with data transformations TAA is something like this:
Sync to Entities → Run loads of complex TRS data mutations → Sync back to Transform
Or just running complex simulation in a job and sync’ing back.

This way marshalling cost will be lower than overall cost of data mutation.

One thing I haven’t tested with Entities 1.0+ is burst compiling IJobParallelForTransform jobs.
If its possible, you could try moving scheduling logic to the ISystem & burst compiling OnUpdate.
That should reduce cost of scheduling.

4

Hi @VergilUa , Thanks for your reply. Actually I am not using ECS. This is just for GameObjects and the number of transforms is 100-300. And I assume the “marshalling” you mentioned is the TransformAccessArray.Sort. I sometimes see this in profiler after modifying the TransformAccessArray but the time cost it introduces is negligible.
Overall my jobs run pretty fast, at about 0.08ms:
9188477--1280510--upload_2023-8-2_19-0-44.png
As shown above, the Schedule() wasn’t blocked by jobs and there aren’t much idle betwen jobs. However there are some frames when my main thread was blocked for more than 1 ms, and there is unreasonably long idle between jobs, as shown in my first reply.

5

A performance analysis of my job manager over 300 frames:
9188489--1280513--upload_2023-8-2_19-6-34.png
Most frames it only costs microseconds. There is one frame it reaches 0.5 ms and 1 frame it reaches 0.97 ms

6

For low quantity of transforms it sure shouldn’t cost that much.

Try profiling with Deep Profile on. See what’s actually going on.
It could be .Complete stalling job system. Could be something else.

7

Hi @VergilUa , I checked deep profile and looks like JobsUtility.ScheduleParallelForTransform_Injected() contributes most to the overhead:
9190427--1280867--upload_2023-8-3_11-32-5.png

8

“the cost of actually waking up worker threads can be expensive”, written in Unity - Scripting API: Unity.Jobs.JobHandle.ScheduleBatchedJobs (unity3d.com)

Maybe the ScheduleParallelForTransform_Injected() is waiting for worker threads to wake up and to kick off jobs?

Is there anyway of checking if worker threads are awake? Then I can use a coroutine to call Schedule() only when worker threads are awake.

9

That only applies if you perform that method call. Otherwise jobs are scheduled but not ran instantly (unless you call .Complete right away). So it shouldn’t matter for this case.

Its highly likely native side does some work beforehand.
Hard to tell without the sorces, but basically, its managed to native transfer cost.