Hi all!
The app I am making requires me to make scans of a whole floor of a house, with the goal of going all the way to scanning all floors of a house with an iphone using AR kit currently through AR foundation meshing. From what I can tell, the default mode for meshing is a sort of temporary meshing mode. It signals to ‘remove’ any meshes not directly near the camera, and all meshes in the case of tracking lost.
I am attempting to circumvent this by creating my own mesh object with the mesh manager generated meshes, anchoring them, and not deleting them when they are ‘removed’, however this causes some overlapping, duplicate meshes to be created, even though I am maintaining their trackableIds and updating the old meshes when one with the same trackableId is created.
Does anyone know of a way to set AR foundation or otherwise harness AR kit to maintain a full understanding of the meshed area seen in this session? I might eventually setup my own background thread to try to do this processing and remove duplicate mesh sections, but I would much rather take advantage of any sort of standard approach, as I find it hard to believe I am the only one who doesn’t want transient meshing.
Thank you for your help!
Kevin
I’m not sure the phone gyroscopes are good enough to do something like this… without beacons I would imagine they would drift a lot.
For example if you are scanning a wall in one room, then walk out to the hallway and into the adjacent room, then scan the other side of that wall, I’m pretty sure the drift of most consumer-grade phones would not guarantee those two wall sides to be coplanar or even the right distance apart.
It feels like something that actually knows volume and space and can reconcile drift and jitter would be needed, and there would be a constant renormalizing / annealing / reconciling process that would need to happen to keep things truly where they are in space.
You’re not wrong there… I’m sure these real estate house scanner apps can do it… maybe there’s some pacakge up on github that you can work with… I just don’t know what terms to look for.
1 Like
There is not a standard approach for solving this. While there are some persistent features in AR Foundation, meshing is not one of them. (Example of one we do support: persistent anchors for Meta devices: Persistent anchors | AR Foundation | 6.0.3)
This really comes down to the functionality that are provided by ARKit, etc themselves. For example, the persistent anchors mentioned above are powered by the fact that it is natively supported by Meta devices.
You could certainly take the mesh data that is being generated by the meshing subsystem and save it for future use, but there’s not a built-in way to feed this mesh data back into the subsystem or prevent duplicates from being discovered.
1 Like
I think your best bet here is to leverage Apple’s AR World Map feature (we have a sample for this).
That said, a house is a very large space to localize. ARKit is not really designed to handle spaces this large in my opinion, and you might find this to be preventatively difficult due to platform limitations.
But here’s how I would approach the problem.
First you would scan the space. Apple Room Plan would probably give you better results here than saving the ARKit mesh. (Unity does not currently have an Apple Room Plan integration, so this would be on you to integrate.) Another option would be to look at Area Mode captures introduced in iOS 18: Discover area mode for Object Capture - WWDC24 - Videos - Apple Developer
Then, you would do is build multiple AR World Maps (my guess is that one per room would be most efficient), and set up your scene to load the pre-scanned mesh whenever the AR World Map successfully relocalizes.
Importantly, if you are relying on loading a pre-scanned mesh, then you likely shouldn’t also be running real-time meshing. This is simply going to result in duplicate meshes as you’ve experienced.
As a final note, I’d just say that working with meshes the size of an entire floor or even an entire house will be very challenging. I would encourage you to try to limit the scope of your app to one room at a time if possible, then stream between different AR World Maps and different room meshes as the user moves from room to room.
Your mileage may vary, but hopefully some useful insight here for you.