How to Implement Hand Tracking?

[mtschoen]

Hey there! We have another thread going about the gesture detection feature. The short answer is: yes the system works, but no, the built-in gestures do not work reliably. This comes down to how visionOS and Meta treat occluded joints. Where visionOS reports the joint as not tracked immediately as it goes out of view, Meta will provide an estimate for a little while before actually reporting the joint as not tracked. This means that a lot of our pre-defined gestures are relying on joints that will stay tracked on Meta but won’t be tracked on visionOS in most cases. We’re working on a plan to bridge the gap, but in the meantime you may have some luck authoring your own gestures for visionOS.