Error when trying to run sentis in another thread

Unity Engine
,
1

Hi,
I’d setup a small Sentis project just for testing, with tinystories model.

The thing is, when I try to run the model in another thread, using Task.Run, I get an error saying MaxGraphicsBufferSize can only be run in the main thread.
There’s no way to run Sentis in another thread? Running it in the main thread is a big issue, since this is heavy processing >.>

2

Sentis Execute only schedule work to the gpu and the job system. Unfortunately these need to happen on the main thread. The kernel math are however happening on separate threads
It is some work on main thread but we’ve constantly been lowering the cost of scheduling so I’d suggest to be on the latest release to have the best performance there.
Overall it shouldn’t be more than a few ms.
If you observe heavy main thread usage, do check that you are not doing a hard download of your result.
See~Samples\Read output asynchronously for a good example on how to do that

3

This is not working though: var cpuCopyTensor = await outputTensor.ReadbackAndCloneAsync();
There is no such method ReadbackAndCloneAsync() in sentis 2.1.0
and the polling version complicates the code.

4

ReadbackAndCloneAsync is available starting unity version 2023.2 or newer.
If you are not on that version, stick to polling, or call the non async version ReadbackAndClone for a blocking call.

1 Like
6

thanks. I will try that

1 Like
7

Thanks for the information, it did work on newer Unity version 2023.2.

Do you have any suggestions on how to implement async model loading as well? I just want to create an awaitable version of the following snippet. In case you already have this implemented somewhere in the examples, could you please point me to it?

Model model = ModelLoader.Load(m_modelAsset);
m_worker = new Worker(model, BackendType.GPUCompute);
8

You should just be able to use Task.Run to run synchronous code asynchronously, in your example

await Task.Run(() =>
{
    model = ModelLoader.Load(m_modelAsset);
});