Scheduling jobs from NOT a MainThread

Unity Engine
,
1

Hi,

I have got a large code base of jobified and Burst compiled physics simulation, which is scheduled from the main thread.

However, now I need to support a use case where I have got a native thread spawned and owned by haptics hardware API, which should schedule all the calculations in sync. It runs at more or less constant rate of 1kHz, which gives less than 1ms to do the physics calculation and send the resulting force-feedback vector to the haptic device (so called “hapitc frame”).

@Joachim_Ante_1 , @xoofx I would like very much to reuse my current jobified code, which does the job very well and took me a lot of time to develop. I don’t want to schedule/complete jobs from multiple threads, just be able to do this from non-mainthread. I don’t need advanced profiler or debugger support.

I think this is not that unusual use case as it is quite common while working with hardware, for example, with automotive sensors. Currently, such use cases prevent the use of JobSystem and Burst, which are great tools

Thank you

1 Like
2

So there is a native haptic thread in C++ plugin?
And it calls into C# from that C++ code?

And from that C# callback on a custom C++ thread, you want to be able to schedule a C# job?
Do you only want to schedule it or do you also want to call JobHandle.Complete() ?

3

Hi @Joachim_Ante_1 and thanks a lot for a quick response!

Yes, exactly. I would like to schedule jobs as well as complete them from C# callback(s) on a custom C++ thread.

4

Its theretically enable to do that for us. Unfortunately the safety system which is always on would break if we do that because it is a deterministic system that does all checks on the main thread before the job is scheduled etc.

So its a good chunk of work and we are not planning to do that any time soon unless there is a significant demand for it.

That said, burst has an API to compile a delegate directly that you might be able to use to get better C# perf. And you could this way run existing job code directly on that C# callback thread by making your own scheduling method that just runs in on that C# callback thread.

2 Likes
5

I see @Joachim_Ante_1 . Having a fully deterministic simulation would be great indeed. However, for, now, I am breaking determinism anyways by using [NativeDisableUnsafePtrRestriction] to call native code or [NativeDisableParallelForRestriction] to atomically write to other indices during constraints solving, which is typical in physics simulation.

Taking these under consideration, would it be possible to make a “non-deterministic” job scheduling, which would omit all the main-thread safety checks, in a similar spirit how the aforementioned attributes work?

Regarding Burst would it be possible to get a small code snippet to manually compile and invoke the job? The Burst documentation does not mention this.

Summarizing, job scheduling is more important to me as I have got a quite complex jobified physics solver, which already schedules hundreds of jobs from different sub-systems each frame (collisions, deformation, heat propagation, logic). Burst is undoubtedly great piece of software but, in the worst case, with some extra work, I could port the performance-critical algorithms back to native C++, just the way I did it for last couple of years…

Thanks

6

I just want to point out that the most successful commercial physics engine out there is in fact deterministic and this is a perfectly solvable problem… Can’t share any code right now. But making deterministic physics is very achievable without performance loss. And thats our intention for what we want to include in the Unity ECS physics solution.

Check out the Unity.Burst namespace. There is a compile delegate function there somewhere. Sorry for the brevity. Too much stuff to do right now preparing for Unite…

2 Likes
7

+1 on this feature very much. If safety system is biggest issue, maybe you could do it without it at first(unsafe) and we would use it at own risk, I mean we’re not all children playing in sandbox :slight_smile:

8

The issue is really that its deeply integrated and cutting a codepath to turn it off in a subset of the jobs depending on where they are scheduled is actual work. If we do that work i’d rather do it right and make it safe.

I know we are not children… In our community we are professionals & enthusiasts with an incredible amount of skills all across…

But guess what. I think I am somewhat knowledgable in the field of multithreading code. And i can’t possibly imagine writing the amount of jobified code I am writing for features on top of ECS without a safety system. We had so some many freaking examples in our C++ code of just plain wrong jobified code. This is freaking hard to get right. Not having a safety system and writing large amounts of jobified code is quite simply not a responsible thing to do. On the flip side i know for sure of bazillion little issues the safety system found for me that would simply be shipped and the one who cries at the end is the game developer because of a shit metacritic rating because the executable crashes in very hard to reproduce cases…

So in your choice of words… A safety system for jobs is in fact an adult thing to do. Made by and for Adults, who have been there and know why it’s valuable in the long run.

7 Likes
9

Disagree. We are more a childs, than other, because we make games :smile: If you lost child inside - you can’t create games with soul :stuck_out_tongue:

7 Likes
10

Good point :slight_smile:

11

You should see my face when I get a new toy to play around (like ECS, Jobs, new .NET version, etc)… I drop all my matchboxes immediately and jump in the sandbox with drawn toy shovel and bucket, head first.

4 Likes
12

Hi @Joachim_Ante_1 ,
I have been developing a workaround for a while. I have modified my physics solver to be scheduled at 1kHz from the haptics native thread. The solver loop runs a bunch of math heavy functions, which I am porting from Jobs/Burst back to native C/C++. However, it would be great not to double the work and later maintain quite complex algorithms in both HPC# and C/C++.

I have tried calling IJob.Execute() from my native callback and it worked fine. Unfortunately, it is not Burst accelerated. The IJobExtensions.Run(T), although the docs say “Perform the job’s Execute method immediately on the same thread.”, gives exception saying that it has to be called from the MainThread.

I can Burst compile a delegate but I have got no idea how to combine all this with IJob

public delegate void ExecuteJob();
(...)
ExecuteJob exec = BurstCompiler.CompileDelegate<ExecuteJob>(what should go here???);

So going back to your quoted answer from few weeks ago, could you please provide some code snippet of my own scheduling method? I would appreciate this very much.

Thanks!

13

Hi @Joachim_Ante_1 ,

to quickly recap, I have got a heavily jobified surgical simulation system implemented using HPC (JobSystem + Burst, no ECS). It works great with VR controllers input and I am very happy with it. Please have a quick look:

However, now I have got customers, who would like to use my simulator with some more serious, haptic-enabled devices, such as Phantom Omni pictured below. In order to do this, I need to be able to run my simulation physics update at more or less constant rate of 1000Hz to generate realistic and smooth force-feedback.

Unfortunately, it is currently not possible, as Jobs can only be scheduled from Unity’s main-thread. I have started porting a lot of performance critical code back to C/C++ to overcome this issue but it is a lot of code and maintaining two math-heavy versions of the same code is rather painful.

I just wanted to ask whether something has changed over the last year in the topic of giving developers possibility to overcome the safety-system and be able to use JobSystem and Burst-compiled jobs from non-main thread?

Thank you

837×558 78.1 KB

4 Likes
Burst 1.5.0-pre.5 - function pointers not working
14

It is not clear to me why running the simulation at a 1000Hz rate for haptic feedback necessitates a controlling thread.

Assuming your rendering happens at 60 FPS you could schedule 15 chained sets of jobs that update the simulation. IJobParallelForDeferred is necessary to do this assuming that some of batch sizes aren’t known at schedule time anymore. This is how Unity.Physics works. There is no sync point on the main thread.

Truly… The concept of a thread is absolutely not necessary for what you are doing it simply creates unnecessary complexities. It’s just a different way of looking at the problem.

15

Hi @Joachim_Ante_1 , thanks for a quick reply and apologies for delay on my side.

The human haptic perception operates at a far higher rate than our visual system (around 30 times faster).
It is commonly accepted that the required refresh rate to provide realistic force feedback to be at least 1,000 Hz.*

I have already tried what you have described (scheduling 15 jobs inside the graphics frame). This makes sens assuming that I could have the wait period between each subsequent job launch/completion of roughly 1ms. However, in practice, this does not work that well with the hardware, the force feedback is not that smooth and I can feel a small jitter on the device.

To solve this I would need to rather precisely match the hardware timing (its called haptic frame).
Ideally, I would like to to launch burst jobs from the native thread, which is controlled by the haptic device drivers.

I think such use cases are quite common when dealing with timing-dependent hardware. For example, I had similar issues in my previous job in the automotive sector while working with car sensors. If I didn’t “tick” them in a given time-frame they were going into error mode.

I am fully aware that Unity is a game engine and not real-time operating system but having a possibility to use such a great tools as JobSystem and Burst for such cases, without the hassle of native code implementations, would be great and enable many more interesting applications in academia and industry.

Thanks!

* “The Role of Haptics in Medical Training Simulators: A Survey of the State of the Art”, T. Coles & N. John, 2010

16

I recommend sharing the code and using BurstCompiler.CompileFunctionPointer for the time being. It probably requires 19.3, it has a lot of limitations

  • safety system gone
  • Any exception will currently crash (Will be fixed)
  • Possible aliasing bugs at the moment (Will be fixed)

That said, it would allow you to share code. I would recommend using normal system / C# job style approach inside of Unity for best development iteration and then have a wrapper around it with BurstCompiler.CompileFunctionPointer essentially executing the jobs yourself from the dedicated hardware thread. It should even be possible to execute the FunctionPointer you get back directly from C++ without any extra interop.

3 Likes
Burst 1.1.1 new release
17

Thanks again for taking time to help me out and quick response.

Yes, that would be already very helpful. I could use some other job scheduler to launch the Burst compiled jobs.

I have found some more info on BurstCompiler.CompileFunctionPointer but, given it’s description in the 1.1.1 release notes and some example Burst 1.1.1 new release , I am a bit confused how can I achieve this:

Here is some code sample from my repo with JobSystem/Burst examples examples
https://github.com/korzen/Unity3D-JobsSystemAndBurstSamples

I would be very grateful for some snippet on how to use BurstCompiler.CompileFunctionPointer to launch this job from outside the JobSystem

[BurstCompile]
struct OceanJob : IJob
{
    public NativeArray<Vector3> vertices;

    public float time, rippleStrength, scale;

    public void Execute()
    {
        for (int i = 0; i < vertices.Length; i++)
        {
            float3 vertex = vertices[i];
            float2 offset = new float2(vertex.x, vertex.z);
            float ripple = math.sin(time + math.length(offset));
            vertex.y = ripple * rippleStrength;
            vertices[i] = vertex;
        }
    }
}

Thank you

18

See the attached cs file for example code.

4889015–472445–BurstDelegateTest.cs (1.72 KB)

1 Like
19

Thank you Joachim! Have a good weekend

1 Like
20

To be honest, if you trying to control/monitor hardware at such high frequency, you should be using relevant hardware, which does high frequency controls. I wouldn’t be calling Desktop as good tool for this nature. It has lots of own internal overheads and risk of instability at such high rate.

Unity should be used more to do visualization HMI, rather than direct hardware control.
I am just not convinced upon using chosen approach, as for surgical precision solution.