Native API wrappers for skeleton

In the com.unity.xr.visionos@1.1.6\Runtime\NativeApi\SkeletonJoint.cs, there is an api
ar_skeleton_joint_get_anchor_from_joint_transform.

I guess it correspond anchorFromJointTransform | Apple Developer Documentation

Can we add support parentfromjointtransform?

As visionOS’ ARKit always set some joints as unTracked, we need keep the unTracked joints pose relative to parent, so we can make a prediction for full hand pose.

I had submitted an idead on the roadmap.

Hi there! You are correct, we have a DLLInterop method for calling ar_skeleton_joint_get_anchor_from_joint_transform, which is the C equivalent of the API you linked. There’s nothing special about this C# code living in our package. You are able to copy/paste this method wherever you want and call it from your code, or modify it to link to ar_skeleton_joint_get_parent_from_joint_transform or any other ARKit C APIs. You’ll still need access to the pointer for the OS_ar_skeleton_joint, which we don’t expose so ultimately you’re better off making a local copy of com.unity.xr.visionos in your project’s Packages/ folder to modify our code.

However, I don’t think that any of this will actually get you what you want.

When you say “always sets as untracked,” are you referring to the unsupported Palm joint? Or do you mean you’re seeing joints reporting NotTracked when they are not visible? If so, I don’t think parentFromJointTransform will help you. Some other users have reported issues with the XR Interaction Toolkit hand gestures feature caused by the same problem. The behavior on Quest and other OpenXR platforms is to still report estimated joints as tracked, but it looks like visionOS is more conservative and will report joints as being not tracked, even though you can still get an updated pose from ARKit with an estimate of where the joint should be based on the rest of the hand.

We’re working out the best way to solve this “for real,” but in the meantime you should be able to modify VisionOSHandProvider locally to just always report the joints as being tracked. Simply replace var jointIsTracked = true; with var jointIsTracked = true;. This is on line 147 for the latest version (1.1.6) of com.unity.xr.visionos. Again, to modify the package you’ll need to copy it from Library/PackageCache/com.unity.xr.visionos@1.1.6 to Packages/com.unity.xr.visionos. That will allow you to make changes that don’t get overwritten by the package manager.

Please let us know if this solves the issue for you. Good luck!

2 Likes

Hi, thanks for your reply.

Yes, I mean, i’am seeing joints reporting NotTracked when they are not visible.

According to your suggestion, I modify the package succeed, and it solves the issue for me. :wink:

This also worked out for us, thank you very much!