(Use Case- Prototype) Handwritten Digits

AI Beta Sentis
1

Hi,

I am an XR research & developer and this is what I was able to prototype using the Sentis.

In this prototype, I am basically drawing digits in VR, which is then passed to the MNIST - Handwritten Digit Recognition model and the output is displayed as text.

image
image1442×859 31.8 KB

2

Hey there, looks awesome! We have actually been thinking about this exact use case recently and how it could be great for a full project sample for Sentis (genericized for use beyond VR). Were there any gotchas that you ran into along the way, anything you would do differently, or anything that Sentis could be further optimized for in this use case?

3

Happy to hear that :slight_smile:
Nope, I didn’t face any glitches. The example in the documentation helped set it up.
Experience wise, I did not see any frame rate drops. So I did not see it’s performance, and so I did not think about optimizing.
Probably this weekend I’ll check it out. Are there any key data you’d like me to watch out for? or record?

4

Thanks for letting us know. Well since you asked :smiling_face_with_three_hearts: …

We would love to know:

  1. What VR headset hardware and OS combination are you deploying to?
  2. What is the highest frame rate (or just the actual rate now) you can run at with no lost frames?
  3. Is this for a specific game? If so, can you share which one? It’s ok if you don’t want to share in this forum

Thanks!

1 Like
5

Alright, finally got some time to work on this again.
I have created this report for you :smiley:

I have answered those questions in that report and you might be interested in the ‘Performance Check’ section.

To summarize:

  • Performance wise, it good and almost similar on all 3 platforms i.e Unity editor, PC build and APK build
  • Prediction wise, it works excellent in Unity editor, little bad in PC build and completely wrong in the APK build.

I have added the videos in the report for your refence.

Hope this helps :slight_smile:

1 Like
6

What hardware are you using for the on device prediction?
It’s not normal that the inference is wrong, would you be able to submit a bug report?

7

Hey @ashraypai_unity , could you provide a little more info on the device you ran the prediction on?

8

This is the hardware specs of Quest2

image
image808×209 21.3 KB

Yeah sure, I’ll submit a bug report.

9

Hey,
So I ran it on my laptop which is i5 10th gen, RTX 3050 and 16 GB RAM.
I also ran it on my Quert2 Headset and you see the details in the image I attached above :slight_smile:

10

Thank you for the info!

11

FYI we have this in our backlog to fix in a future release, known as issue 76.

1 Like