Title may be confusing but basically the damping is not always applied. Sometimes the camera goes back to the original state with a smooth movement while other times it just pop back in place.
Here a video showing the issue and also some screen of my cinemachine and collider component
