Is getting rid of rigidbody jitter possible?

I have an XR Grab Interactable with it’s mode set to velocity tracking.

My movement is rigidbody based and I handle all the forces and moving in FixedUpdate, however whatever the player holds still jitters when they move.

Smooth position and smooth rotation helped a little, but the jitter is still very noticeable and they make my object track way slower and it lags behind.

Turning on interpolation for first just the objects, then just the player, then both, fixed the jitter however it made it so the object would lag behind and then snap back into the place it’s supposed to be, so actually all it really did was make a slower jitter.

What is going wrong here?
My movement code is this

using UnityEngine;
using UnityEngine.Events;

public class PlayerMovement : MonoBehaviour
{
    [Header("Movement")]
    public float moveSpeed = 6;
    public float movementMultiplier = 10, airMultiplier;
    public float groundDrag = 6, airDrag = 2;
    public float jumpForce = 5;
    public PhysicRig rig;
    public LayerMask groundMask;
    public float gravityMultiplier = 3;
    public bool wallrunning; // used by player soundscape
    public float coyoteTime = 0.1f;
    public UnityEvent OnJump;
    public float ledgeDetectionDistance = 0.5f;
    public float ledgeAssistForce = 10f;
    public float ledgeDebounce;

    [Header("Input")]
    public Controller leftController, rightController;

    Vector3 moveDirection;
    float horizontalMovement, verticalMovement;
    Rigidbody rb;
    Transform cam;
    public bool isGrounded; // accessed by player soundscape
    RaycastHit slopeHit;
    Vector3 slopeMoveDirection;
    float coyoteTimer;
    float jumpDebounce = 1f;
    float debounceTimer;

    private bool OnSlope()
    {
        if ( Physics.Raycast(cam.position, Vector3.down, out slopeHit, rig.currentHeight + 0.5f) )
        {
            return slopeHit.normal != Vector3.up;
        }
                 
        return false;
    }

    private void Start()
    {
        rb = GetComponent<Rigidbody>();   
        cam = Camera.main.transform;
    }

    void MyInput()
    {
        horizontalMovement = leftController.thumbstick.x;
        verticalMovement = leftController.thumbstick.y;

        Vector3 camForward = new Vector3(cam.forward.x, 0, cam.forward.z);
        Vector3 camRight = new Vector3(cam.right.x, 0, cam.right.z);

        moveDirection = camForward * verticalMovement + camRight * horizontalMovement;
    }

    void ControlDrag()
    {
        rb.drag = isGrounded ? groundDrag : airDrag;
    }

    private void Update()
    {
        MyInput();
        ControlDrag();

        isGrounded = Physics.Raycast(cam.position, Vector3.down, rig.currentHeight + 0.3f, groundMask);

        coyoteTimer -= Time.deltaTime;
        debounceTimer -= Time.deltaTime;

        if ( isGrounded )
        {
            coyoteTimer = coyoteTime;

            if ( IsHangingOnLedge() && debounceTimer < 0 )
            {
                AssistClimb();
                debounceTimer = ledgeDebounce;
            }
        }

        if ( rightController.secondaryButton  )
        {
            gravityMultiplier = 0;

            if ( coyoteTimer > 0 && debounceTimer < 0 )
            {
                debounceTimer = jumpDebounce;
                Jump();
            }
        }

        else
        {
            gravityMultiplier = 5;
        }

        slopeMoveDirection = Vector3.ProjectOnPlane(moveDirection, slopeHit.normal);
    }

    bool IsHangingOnLedge()
    {
        RaycastHit hit;
        Vector3 forwardOffset = new Vector3(cam.forward.x, 0, cam.forward.z) * 0.5f;
        Vector3 rayStart = cam.position + forwardOffset + Vector3.up * ledgeDetectionDistance;

        if (Physics.Raycast(rayStart, Vector3.down, out hit, ledgeDetectionDistance, groundMask))
        {
            return true;
        }

        return false;
    }

    void AssistClimb()
    {
        rb.AddForce(Vector3.up * ledgeAssistForce, ForceMode.Impulse);
    }

    void Jump()
    {
        rb.velocity = new Vector3(rb.velocity.x, 0, rb.velocity.z);
        rb.AddForce(Vector3.up * jumpForce, ForceMode.Impulse);

        OnJump.Invoke();
    }

    private void FixedUpdate()
    {
        MovePlayer();
    }

    void MovePlayer()
    {
        if ( !isGrounded )
        {
            rb.AddForce(Vector3.down * gravityMultiplier, ForceMode.Force);
        }

        if ( wallrunning )
        {
            return;
        }

        bool onSlope = OnSlope();

        if ( isGrounded && !onSlope )
        {
             rb.AddForce(moveDirection.normalized * moveSpeed * movementMultiplier, ForceMode.Acceleration);
        }

        else if ( isGrounded && onSlope )
        {
            rb.AddForce(slopeMoveDirection.normalized * moveSpeed * movementMultiplier, ForceMode.Acceleration);
        }

        else if ( !isGrounded )
        {
            rb.AddForce(moveDirection.normalized * moveSpeed * movementMultiplier * airMultiplier, ForceMode.Acceleration);
        }
    }
}

Changing the fixed timestep also did not help

Set to Collision to Continuous

to me it sounds like it works as expected because you are moving while holding, maybe this system is mostly intended to stationary players.

imagine your player as a the rigid arm of a crane and the object being connected to this crane by an invisible elastic rope. so when moving it yoyo around as you have described.

theoretically any physics of the object should not work when moving to simulate being carried around by something else. but then you do want physics if you want to simulate the object in had hitting some other obstacle in scene like a tree or something

maybe this comment can help you

one solution could be to remove the object from the simulation and attach it to the hand bone so it will always move as a whole with the player skeleton. then when you are ready to get rid of the object you move it back into the simulation. maybe have two objects so hide the one that is not needed while showing the other. just so you don’t mess the setup. maybe this system allows this, i didn’t check it

Already done it still jitters.

Well technically the problem isn’t that it’s lagging behind, it’s that it’s jumping back and forth between two positions.

After further testing though, I’ve seen that on a low bitrate it jitters a bunch, but at a high bitrate it doesn’t do that. So I don’t actually think it’s Unity’s fault I think it’s just my bad wifi.

Nevermind… I was completely wrong. The true cause of it is when the game view is not maximized it jitters, and when it is maximized, it does not. I’m gonna do a new post because this seems like a bug.