Can someone please help me integrate an object detection model with Sentis?

I am currently working on an AR iOS app. I want to use this object detection model in real time to detect stairs. How can I do so with sentis? I heard that there aren’t any official sample code for object detection. Can someone please help me please? Here is the object detection YOLO model basic structure:

Hi check out our latest model examples on Hugging Face . YOLO is a good example for object detection and Blaze Face has a good example using the camera.

1 Like

I just came across a article(in chinese) with a similar input-output to the model you are using. Maybe you can try his parsing method. (But he uses Barracuda :upside_down_face:)

Thank you @conclean I appreciate it :slight_smile:

@PaulBUnity is Blaze face example using sentis? Can you please share the link to repo . All i can see is DepthEstimation , BoardGame and DigitRecognition
Regards,

Here are all the sentis models on Hugging Face. Not all of them have project files. But there are instructions how to put them in your own project.

Thanks @PaulBUnity , I modified the Yolo8n.cs code and able to run it successfully on windows and it can detect the object (front camera)
After that I tried build it for Android , which is successful as well but when i am running the Android App it is showing all white and no camera preview running.
I have also provided the camera permission to the app in the settings but still the same.
Can you please let me know which extra setting i have to enable to make it working.
Thanks to let me know on this.
Best Regards,
Saurabh

If you have not tried yet use const BackendType backend = BackendType.CPU; instead of GPUCompute on mobile platform. I had the same issue with another model running without problem in the editor but producing weird results on android phone.

Hello @MoritzCermann ,
Thanks for the suggestion. I tried this as well but again the all white texture and no camera preview. I was wondering if there is any other setting? Also is it taking the front camera or back camera and how to change it?
Any pointers towards it will be much appreciated.
Thanks a lot for reading this !
Regards,
S

do you have multiple webcam?
if you do you have to select them by name

@alexandreribard_unity yes 2 cams in a phone ( one front and one back) .
fyi i tried building DepthEstimation for android and it builds and works fine. Except that in DepthEstimation the webcam texture is on plane while in my case it is on rawimage.
Any clue?
Regards,
S

you have to select the right one by name

1 Like