Thinking about this some more, your main problem is that FixedUpdate occurs independently of Update and LateUpdate. On any frame, Update and LateUpdate are called, followed by rendering. Before Update, however, FixedUpdate will get called zero or more times. After each FixedUpdate, the physics system advances too.
So if the render frame rate is high, Update may be called several times without an intervening FixedUpdate. If the render frame rate is low, FixedUpdate gets called many times before the next Update call. If you do your tracking in the Update or LateUpdate calls then you’re running out of step with the physics, and - crucially - the deltaTime in your Update is not the same as the amount of physics time which has passed. This is most likely what causes your jittering.
If you want to leave your code in Update, running it once per rendered frame with a variable timestep, then thats OK but you need to use the total FixedUpdate deltaTime for the current frame - i.e. count it up in FixedUpdate, use it in Update (for things that need to be in step with the physics), and zero it at the end of Update.
Otherwise, if you want your code to run with a fixed timestep but after the physics updates, you can put it in both FixedUpdate and Update, but only execute it if it’s not the first time in the frame. i.e. set a bool to “true” in Update or LateUpdate, and use that to skip the first execution of your post-physics-fixed-update code in the next frame. If the frame has no FixedUpdates then this will skip the only execution of your code for that frame, which is what you want too.
Example 1:
private float totalFixedDeltaTime;
void FixedUpdate()
{
totalFixedDeltaTime += Time.deltaTime;
}
void Update()
{
holding.transform.rotation = Quaternion.Slerp (holding.transform.rotation, transform.rotation, totalFixedDeltaTime / (holdingInfo.myWeight/100)); //holding rotation lag
holding.transform.position = Vector3.Lerp(holding.transform.position, transform.TransformPoint(0,holdingInfo.myCamDisY,holdingInfo.myCamDisX), totalFixedDeltaTime / (holdingInfo.myWeight/150));
totalFixedDeltaTime = 0;
}
Example 2:
bool firstPostPhysicsFixedUpdate;
void PostPhysicsFixedUpdate()
{
if (firstPostPhysicsFixedUpdate)
{
firstPostPhysicsFixedUpdate = false;
return;
}
holding.transform.rotation = Quaternion.Slerp (holding.transform.rotation, transform.rotation, Time.fixedDeltaTime / (holdingInfo.myWeight/100)); //holding rotation lag
holding.transform.position = Vector3.Lerp(holding.transform.position, transform.TransformPoint(0,holdingInfo.myCamDisY,holdingInfo.myCamDisX), Time.fixedDeltaTime / (holdingInfo.myWeight/150));
}
void FixedUpdate()
{
PostPhysicsFixedUpdate();
...
}
void Update()
{
PostPhysicsFixedUpdate();
firstPostPhysicsFixedUpdate = true;
...
}