Object to Follow Hand Joint

XR Hands is working well on device. I am trying to have an object follow one of the hands. Set up is really similar to Polyspatial Mixed Reality example. I’ve tried to get the position and rotation of the palm joint and set the object to the same position and rotation. For some reason the object will not follow. Feel like I’m missing something obvious. Also, is XR interaction toolkit a better way to go about this? Thanks.

using UnityEngine;
using UnityEngine.XR.Hands;
using UnityEngine.XR.Management;


namespace PolySpatial.Samples
{
    public class objectFollow: MonoBehaviour
    {
        [SerializeField]
        GameObject objectToFollow;

        [SerializeField]
        Transform m_PolySpatialCameraTransform;

        XRHandSubsystem m_HandSubsystem;
        XRHandJoint palm;
       
        void Start()
        {
            GetHandSubsystem();
        }

        void Update()
        {

            if (!CheckHandSubsystem())
                return;

            var updateSuccessFlags = m_HandSubsystem.TryUpdateHands(XRHandSubsystem.UpdateType.Dynamic);

            if ((updateSuccessFlags & XRHandSubsystem.UpdateSuccessFlags.RightHandRootPose) != 0)
            {
                palm = m_HandSubsystem.rightHand.GetJoint(XRHandJointID.Palm);
                FollowPalm(palm);

            }
        }

        void GetHandSubsystem()
        {
            var xrGeneralSettings = XRGeneralSettings.Instance;
            if (xrGeneralSettings == null)
            {
                Debug.LogError("XR general settings not set");
            }

            var manager = xrGeneralSettings.Manager;
            if (manager != null)
            {
                var loader = manager.activeLoader;
                if (loader != null)
                {
                    m_HandSubsystem = loader.GetLoadedSubsystem<XRHandSubsystem>();
                    if (!CheckHandSubsystem())
                        return;

                    m_HandSubsystem.Start();
                }
            }
        }

        bool CheckHandSubsystem()
        {
            if (m_HandSubsystem == null)
            {
                Debug.LogError("Could not find Hand Subsystem");

                enabled = false;
                return false;
            }

            return true;
        }

        void FollowPalm(XRHandJoint palmJoint)
        {
            
                  if (palmJoint.trackingState != XRHandJointTrackingState.None && palmJoint.TryGetPose(out Pose palmPose))
                  {
                     
                      Vector3 palmPosition = m_PolySpatialCameraTransform.InverseTransformPoint(palmPose.position);
                      objectToFollow.transform.position = m_PolySpatialCameraTransform.TransformPoint(palmPosition); 
                      objectToFollow.transform.rotation = palmPose.rotation;

                      Debug.Log("Updated Object Position: " + objectToFollow.transform.position);
                      Debug.Log("Updated Object Rotation: " + objectToFollow.transform.rotation.eulerAngles);
                  }
        }
    }
}

There’s no Palm joint in ARKit hands, Hand Skeleton Docs. Try using middle finger metacarpal.

the mixed reality examples contains a joint tracker script

but one issue is that somehow the rotation isn’t always updated

1 Like

You the man Dan, thanks for helping me on X too!

1 Like

Good to know, thanks!

Hi, I download the sample code and run the demo visionOSTemplate-1.1.4, but I can not see the hand or the joints in real device Vision Pro, and log shows XR general settings not set.

Do you have any idea how can solve this.

XR plugin is checked and it’s OKay, and com.unity.polyspatial, com.unity.polyspatial.visionos, com.unity.polyspatial.xr. are incoded.


 var xrGeneralSettings = XRGeneralSettings.Instance;
            if (xrGeneralSettings == null)
            {
                Debug.LogError("XR general settings not set");
            }

Did you install com.unity.xr.visionos? XRHandSubsystem is in there.