Unity Sentis

Unity Engine
, ,
1

We have stopped working on the Barracuda package but are continuing it’s mission with the Sentis package. Refer to the Sentis documentation for more technical details, and see our AI announcement for more information.

1 Like
2

Will sentis allow similar results?

3

yes

1 Like
4

Not sure where I should put this, but there’s an error in your documentation.

In here: Run a model | Sentis | 1.0.0-exp.6

It says that the script runs the model one layer per frame. However, that’s not true. It loops in the middle of the Update call so it blocks the thread. You can fix it by changing the Update method to

void Update()
{

    if (!started)
    {
        modelEnumerator = worker.StartManualSchedule(inputTensor);
        started = true;
    }

    // Iterate running the model once per frame
    // In each iteration of the do-while loop, use the IEnumerator object to run the next layer of the model
    if (hasMoreModelToRun)
    {
        hasMoreModelToRun = modelEnumerator.MoveNext();
        Debug.Log(worker.scheduleProgress);
    }
    else
    {
        // Get the output tensor
        var outputTensor = worker.PeekOutput() as TensorFloat;
        outputTensor.PrepareCacheForAccess(blocking: true);
        float[] outputArray = outputTensor.ToReadOnlyArray();
        // turn off the component once the tensor is obtained
        enabled = false;
    }
}

EDIT: reported it as a problem to the documentation, but keeping it here so people can have a workaround.

1 Like
5

How does this work with MLAgents?

6

jasonboukheir3 thanks we’ll take a look and update it!

1 Like
7

What are the requirements for getting closed beta access? I’ve signed up as soon as it was announced and I’d love to use it in order to bring this to reality. I’ve been using Barracuda but struggling to get some of the newer object detection models running.

8

Does anyone by any chance able to test Sentis with Webgl and compare the performance with Barracuda?

9

Hi Aurimasp, thanks to the team for this great piece of work! May I ask how long it usually takes to get approval for using the Sentis package? Now I’m working on a project that needs more flexible tensor operations, which can’t seem to be satisfied by the current Barracuda package.

10

Please sign-up on Unity
We are slowly granting more access to users as time goes by

11

I did, still waiting:))

12

Thanks for replying! I also signed up already, so I’ll keep waiting then.

13

@ we shipped an update to Sentis today to address this issue.

For anyone that is looking to get into the Sentis beta, you can DM me, or email ai-beta@unity3d.com after signing up on Unity.

4 Likes
14

Hi there! I’ve been pestering the signup form for about two or three weeks now trying to get access to Sentis. I got in for Muse (which I really don’t care about) but nothing from Sentis. I’ve actually spent the last two months trying to get an ONNX runtime working. Because Unity’s mono/C# version is so old, most modern NuGet libraries don’t work for this. They work fine in a console app, though. I tried literally all of them from Microsoft.AI.MachineLearning and various OnnxRuntime packages, to pulling apart packages and manually dropping .dll’s into plugin folders and trying to satisfy their dependencies. The last resort was building out my own ONNX runtime server as a separate process talking to it over named pipes, which was a huge pain. We’re doing some amazing stuff on our project (a first person MMO with positional audio, doing speech-to-text, then regenerating the voice on the far side of the network with TTS). But without Sentis, this will be incredibly painful.

1 Like
15

Sentis is in Open Beta now, you should be able to install the package by following the steps in this thread.

16

You don’t need to register anymore for Sentis. It is available via the package manager.

1 Like
17

thats a spam post you replied to if you check carefully :wink:

1 Like
18

How does this work?

19

Hi guys, we can’t download the MNIST model anymore, the link seems broken and I can’t find the model anywhere else: Understand the Sentis workflow | Sentis | 1.3.0-pre.3
https://github.com/onnx/models/tree/main/vision/classification/mnist

20

Thanks for reporting the issue, I’ll update the link for the next Sentis version. You can download the model from here: https://github.com/onnx/models/tree/main/archive/vision/classification/mnist

1 Like