Changing transform.localscale doesn't take affect immediately? The next RayCast isn't precise

I’m changing object’s scale before a raycast to implement some avoidance logic for the AI. One part I couldn’t get done is how to add more space margin around the objects(=players).
The logic I’m using is to increase the object’s size so it’s collider is also increases, then cast rays to see where the AI can go, then decrease the size back.
Debug prints the updated scale before the raycast, but I can see, it had no effect. When I comment out the scale revert code which come after of course the raycast, I can see the effect.

I’m working in the Update function. What is wrong?

I took some testing in a separate project.
Tested transform.position change → doesn’t take affect immediately
Tested gameobject’s SetActive(false) → takes affect immediately