Sentis Inference

Unity Engine
,
1

Hello everyone,

I’m new to the field of AI, I have an ONNX model derived from YOLOv8, and I would like to perform real-time inference.

In addition to this, I have the labels for my model, and I’m interested in obtaining the identification score, class, and bounding box.

I’m a bit lost on how to proceed. Could anyone guide me on the steps to integrate this model into Unity and perform real-time inference? Any help or resources would be greatly appreciated.

Thank you in advance for your time and assistance.

Capture d'écran 2023-12-11 175300
Capture d'écran 2023-12-11 175300913×292 17.8 KB

1 Like
2

The Sentis package comes with like 10 pretty good and narrow examples, you can import them individually by selecting Sentis in the package manager (Window → Package Manager) and going to the samples tab.

Also the manual is pretty good, just make sure you’re on the most recent version (google usually returns an outdated one)
Get started | Sentis | 1.3.0-pre.2 (unity3d.com)

1 Like
3

Thank you so much for your helpful response! I’ve previously managed to successfully load the model as per your instructions. Additionally, I’ve created a tensor from an image without any issues.

However, after completing these steps, I’m facing a new challenge. Although I’ve gone through the manual and the included examples, I couldn’t find specific information on how to retrieve the scores and bounding boxes of recognized elements.

Since I’ve already succeeded in loading the model, could you provide additional guidance or direct me to specific code examples for retrieving scores and bounding boxes?

I appreciate your continued assistance!

Best regards,

4

Here is a blog that may be of interest to you:

and here’s the code on GitHub: GitHub - cj-mills/unity-barracuda-inference-yolox: This Unity package extends the functionality of the barracuda-inference-base package to perform object detection using YOLOX models. (all his other blog posts are super interesting too)

It was made for Unity Barracuda, so it will not work out of the box with Sentis but especially the part on processing the output should be pretty similar.

4 Likes
5

Hai, i am trying to object detection in ar foundation + baracuda yolo v8 model . I had some issue in camera feed i get every 20 th frame and put inferece my device slightly laged and fraction of seconds freezed how to solve it .

7

Could you please share your code?

1 Like
8

@Thohtdelta001 Could you please share some code…I am also trying to do the same but with Yolo v7 version. I am bit new and struggling a lot

9

Hi there is also a YOLOv7 and YOLOv8n model with example code now on our Hugging Face page.

10

Hi everyone, i contribute to this discussion because i am also having a problem on how to show the results of the inference in unity (sentis). i have successfully uploaded the model and serialized it (yourmodel.onnx —> yourmodel.sentis) i have got the code for using the webcam and creating tensor from sentis team.However, there are no results being interpreted even in the debug console, anyhelp!

11

Hi @Abdou0303 which model are you using? Have you checked out our documentation?