[VisionOS][XRI] Interaction customization and DOF constraints

Unity Engine
, ,
1

Hi all,

To test the capabilities of the XRI Toolkit and the VisionOS sample, I set up a scene in which is possible to constrain interactions on a model in certain ways.

For the Camera, I’m using the XRI_SimpleRig from the sample. The object is configured starting from the XRI_Cube prefab from the VisionOS sample, then I did the following:

  • Removed the MeshFilter, MeshRenderer and BoxCollider, from the root of the prefab
  • Added my model as a child and added a BoxCollider resized to match the bounds of the model
  • Modified the RotationAxisLockGrabTransformer from the Starter Assets (adding the script below) and replaced the XRGeneralGrabTransformer on the root with my modified component
public class RotationAxisLockGrabTransformer : XRGeneralGrabTransformer
    {
        [Tooltip("Defines which rotation axes are allowed when an object is grabbed. Axes not selected will maintain their initial rotation.")]
        public ManipulationAxes PermittedRotationAxes = ManipulationAxes.All;

        /// <inheritdoc />
        protected override RegistrationMode registrationMode => RegistrationMode.SingleAndMultiple;

        Vector3 m_InitialEulerRotation;

        /// <inheritdoc />
        public override void OnLink(XRGrabInteractable grabInteractable)
        {
            base.OnLink(grabInteractable);
            m_InitialEulerRotation = grabInteractable.transform.rotation.eulerAngles;
        }

        /// <inheritdoc />
        public override void Process(XRGrabInteractable grabInteractable, XRInteractionUpdateOrder.UpdatePhase updatePhase, ref Pose targetPose, ref Vector3 localScale)
        {
            base.Process(grabInteractable, updatePhase, ref targetPose, ref localScale);
            
            var newRotationEuler = targetPose.rotation.eulerAngles;

            if ((PermittedRotationAxes & ManipulationAxes.X) == 0)
                newRotationEuler.x = m_InitialEulerRotation.x;

            if ((PermittedRotationAxes & ManipulationAxes.Y) == 0)
                newRotationEuler.y = m_InitialEulerRotation.y;

            if ((PermittedRotationAxes & ManipulationAxes.Z) == 0)
                newRotationEuler.z = m_InitialEulerRotation.z;

            targetPose.rotation = Quaternion.Euler(newRotationEuler);
        }
    }

You can see this test in this video:

The video has 5 parts:

  1. No constraint, XRGrabInteractable tracks everything
  2. XRGrabInteractable only tracks Position
  3. XRGrabInteractable only tracks Rotation
  4. XRGrabInteractable tracks everything, RotationAxisLockGrabTransformer. permittedDisplacementAxes set to All, PermittedRotationAxes set to Y
  5. XRGrabInteractable tracks only Position and Rotation, RotationAxisLockGrabTransformer. permittedDisplacementAxes set to X and Z, PermittedRotationAxes set to Y

In general, I noticed that interactions not always behave consistently, especially when interacting with an object from afar or when switching between far and near input.

For example, translation sometimes behaves like a sling, with objects accelerating much faster than the hand. This causes objects to either get closer or further away from the user instead of keeping a constant distance from the hand.

Rotation sometimes happens around pivots that at times seem completely arbitrary. This is especially visible in the last part of the video, where I had a lot of trouble controlling the rotation of the object because it was rotating on a pivot far from both the model and my hand.

Scale is also sometimes very unstable and difficult to control, either jumping from one extreme to the other or not doing much at all.

In some instances, the objects also jump as soon as the interaction starts, either snapping to a certain rotation or teleporting.

Also, again regarding rotation, I noticed that the rotation of the Interactable always follows the rotation of the hand, but in most of the apps on AVP that’s not how rotation usually works. Most of the time, rotation is performed by selecting an object and rotating it by moving your hand left and right. Much like what happens for example in MRTK when interacting with the manipulators on the bounds of an object.

So finally, my questions are: are all the behaviours observed in my testing by design? Do you have any tips on how to improve this?

Also note that I had to modify the RotationAxisLockGrabTransformer because the original component did not work and was completely ignored.