I am starting to think that there is something broken with NetCode + PhysicsVelocity
I think the issue above is related to the issue below
DOTS Multiplayer discussion page-6#post-6665689
For some reason NetCode kind of loses it with updating PhysicsVelocity. @timjohansson