Jittering/desync of objects following camera's position and rotation

Hello! I am trying to have an object move and rotate relative to the position and rotation of the user. For reference, I am only able to test built to the Simulator, so maybe the behaviour is different on the real thing. When I set gameobjects as the child of the main camera, they do follow the position and rotation but during any movement the objects appear to over-compensate for the motion and go further (ie. when walking forward the child objects will move several extra units forward and then rubberband back in place once the user is still) which results in a very jittery look. I have tried several methods to solve this, including using a separate object with a Tracked Pose Driver as well as just updating the position/rotation in LateUpdate of a script, but all seem to have the same results. I feel like I must be missing something silly, any help is much appreciated!

Can you provide some more information related to your setup and what type of app your are building?

  • Fully Immersive (VR) vs Mixed Reality (bounded or unbounded)
  • What is your camera / XR rig setup in the scene

Setup is mixed reality, unbounded. I’m using PolySpatial 1.0.3. I have an XR Rig object with the XR Origin, and it has the AR session set. Its child is a Camera Offset object, and that has the Main Camera as its child. All of these start at positioned at (0, 0, 0). The XR Origin tracking mode is set to Not Specified. The Main Camera has a tracked pose driver attached set to Generic XR Device, Center Eye - HMD Reference, Rotation and Position, Update and Before Render. I’ve continued fiddling with it for the past few days and I’m still seeing the same behaviour.

AVPJitter (3)

Here’s a showcase of the issue I’m talking about, you can see that the world space canvas and object seems to lag behind the camera when turning, but when moving through space it jumps ahead and rubberbands back. I’m not sure if this behaviour is exclusive to the Simulator or not. If this is how it is supposed to track then are there any work arounds to have a more accurate, smooth tracking for such objects?

I tried to repo this on device and things seem to be moving 1:1. I am setting the cube position based on the camera position + an offset and adjusting it’s rotation locally to match the camera rotation (see script below). (Sorry I realize the gif isn’t super legible). I think the simulator might have some expanded movement or something. In general I would recommend doing some LERPing or smooth movement but I think the jitter might just be a simulator quirk.


public class FollowWithOffset : MonoBehaviour
    Vector3 m_StartingOffset;

    Transform m_CubeTransform;

    Transform m_CameraTransform;

    void Start()
        m_StartingOffset = m_CubeTransform.position;

    void Update()
        m_CubeTransform.position = m_CameraTransform.position + m_StartingOffset;
        m_CubeTransform.localRotation = m_CameraTransform.localRotation;