Mathf.SmoothDamp's smoothTime faster at 0 than negative?

I have trouble with my FPS leaning function.
When my camera was leaning to the side a collider with a rigidbody should detect collision.

but for some reason when I was leaning to the right side it looked like the collider detected the wall to late and I was looking through the wall. but when I was leaning to the left side it stops almost instantly.

after… now 2 or 3 hours I found out that it was not the collider but it was the SmoothDamp function that causes the problem.
info: When the Collider hit a wall it tells the SmoothDamp that the target position is the current position.

after I unhided the smoothTime of the Mathf.SmoothDamp function in the script it was obvious.

When I lean to the right side the smoothTime is a positiv number and when the collider hits a wall it takes sometimes more than 1 second till the smoothTime is 0.
When I lean to the left the smoothTime is a negative number and when I collide with a wall the smoothTime is instantly at 0.

I maybe find a work around but is this normal? or is this a Bug in Unity?

smoothTime Approximately the time it will take to reach the target. A smaller value will reach the target faster.

so your number that is smaller than 0 which is the smallest amount of time it could take.

Is taking less time to reach the target than your other number which is greater.

A greater value taking more time.

Sounds like its working as intended