There are many things that can cause misprediction.
In general, RULE OF THUMB: any state you modify or you use inside the prediction loop should be a replicated field, unless it is a readonly constant or you absolutely know it is a dependent variable it is always refreshed because of other replicated field changes.
And this because of the rollback and resimulate logic.
When it comes to physics, we already replicate PhysicsVelocity (that it is the only thing necessary for physics to work) so for these components you don’t need to care. But if you have other component used by physics simulation, I would strongly suggest to replicate them.
If you just use physics velocity to move objects then no worries.
One thing you can do, while not clarify the whole picture, is to actually start logging the value of the transform for both client and server for a given tick. On the client in particular because of partial ticks, you will resimulate the tick multiple time (from the last backup) so you would end up with a log that looks like this:
[Client] 998 …
[Client] 999 …
[Client] 1000.ServerTickFraction : Transform X: Physics Y Input: Tick, x,y,z …
[Client] 1000.ServerTickFraction : Transform X: Physics Y Input: Tick, x,y,z …
[Client] 1000.1: Transform X: Physics Y ← FULL TICK Input: Tick, x,y,z …
[Server] 998.1: Transform Y: Physics Y Input: Tick, x,y,z …
I would suggest to put logs before and after the prediction loop so you get the start and end value after all the computation.
Now you can compare for tick 998 what the value calculated by the server was and what input it used. That would help immediately understanding what discrepancy there are in the computation.
That being said there is another source of Jitter that is caused by physics interpolation (in case it is used) if your camera follow that entity. The movement you will see it is substantially a jittery back and forth of the transform.
In that case, and almost in all cases, it is more correct to always use the LocaltToWorld matrix and get both the Translation and Rotation to follow from there. That would remove any jittery motion seen from the camera point of view.
Another potential jitter (for physics) is introduced by quantization. To understand if that is the case, either write two custom variant for transform and physics velocity that remove the quantization (Quantization=0 in the the GhostField) or change the default implementation in the package to remove quantization.
If that remove the jitter you are experiencing, the issue it is due to how we predict physics client side in comparison to what server does.
In particular, because client simulate multiple ticks when re-predict (and the same goes for partial ticks as well), it is 'incorrectly" using the last computed state for simulate tick N+1. However, when it receive the state from the server for tick N, it will receive that quantized. So the starting point of the simulation, while not extremely different, it is still not exactly the same, and may result in a different computation. This is especially visible in presence of ramps in certain cases.
This just to mention a couple of sources. Sure, please file/open a bug report for this, so we can use your project and understand what it is going on.
You can open the bug report directly from the unity editor (help/Rerpot Bug) and add a zip with the repro project.