Just want to learn about the interests among the community and gather some initial feedback about a potential package for building webgl based augmented reality app, mainly for image tracking and face tracking.
MindAR is a javascript library built for browsers. I’m thinking about creating a unity package to port the library into unity. The usage will be similar to Vuforia or ARFoundation Image Target. Correct me if I’m wrong, I don’t think there are existing unity libraries that allow us to build web based AR app that can be run in browsers.
However, I don’t have much idea on how many unity developers out there interested in web based AR. Please comments and let me know if it sounds interesting to you. Thanks!
Hello Kim, i’m interested in Web AR, so having Mind AR integrated into Unity would be really incredible. Thank you and i’m looking forward to any updates.
I think this is hard to say. Forum is too easily be overflown in new thread and hard to find anything we needed
Currently now I try to integrate mindar into unity by myself because webxr in safari ios is not ready yet, and webxr mozilla viewer did not support image tracking. But to be honest this is very risky because webxr with image tracking might become standard in every browser anytime in near future. Which is overlapped with mindar
@hiukim-mindar That would be very much helpful, Kim. @ did you succeeded in the integration? can you share any experience or update about it? Thank you very much.
@htetwaiyanaung_unity Sorry for the late because I am not that name. And no I have stop working on mindar integration because I think I could wait for ios implementing webxr in safari, as it was a glimpse of it trying to support in ios 15.4
The situation was called for me to really do mindar in unity again. And with a bit let down about accuracy it really possible to just use mindar in unity webgl
First thing first. There is a trick in unity webgl that, we can just override how webcam work in unity webgl build by simply copy {your Unity Installed Location}\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\lib\WebCam.js into your project Plugins folder and change it to .jslib. Unity will just override every function with the same name and use it as that internally
This allow me to just hook the event of start and stop camera (at the function named JS_WebCamVideo_Start and JS_WebCamVideo_Stop). In this WebCam.js file also utilize variable named activeWebCams that store video element. This is the key to sync mindar with webcam in unity webgl
When you hook the starting event of unity webcam and use the video object to create mindar controller properly and you already compile image target into file .mind. It then start giving you the detection and matrix of image target you want to detect
Finally it now only just interop that you get 16 float number and index of image when mindar detect anything. You then need to have some way to send this data into unity to process into matrix4x4 and calculate it as you would wanted. I use dyncall and marshalling and malloc but you can use sendMessage or anything you prefer
I am not sure how I convey this process properly, you can also take a look at this repo.
First of all, this is great @Thaina . Thank you for integrating mindAR into unity, this will change WebGL AR in unity.
I tried your repo project and changed the targets.mind with my own. I successfully build and hosted it on my server.
I tried it on my xiaomi mi9T android phone but I’m only getting around 5-10fps only, I tried it also on iPhone 13 pro max the experience is smooth but the detection is not working when you first load it, I need to close safari and reopen it to make it work. Hoping for the next updated version of your work.
Has anyone tried to make Face Tracking work? I’m kind of working on it but I’m going kind of blind stumbling through the scripts, working my way based on what @Thaina said but I doubt I’ll be able to work something out.
@kimvasquez17 quez17 @Thaina Hello how are you, I am new in this world of projects and videogames. And I would like to know how you implemented a not so technical step by step. It’s that I need to make a web ar and I see that it works for you but when I mount the project I get errors and I don’t know how to fix them. I would like you to explain to me please
“https://github.com/Thaina/MindARUnity” Download from here and run you can check the error The error occurs inside “mindar-image.prod.js” I find it hard to debug