Character Controller collision with Rigidbody problem

Unity Engine
1

My Player uses the Character Controller to move around, and the Tree has a Rigidbody with a box collider.
When I wait for the Tree to fall to the ground Player cannot move it (which is intended).

But as the Tree falls and you bump into it, the Tree gets pushed rather quickly and feels like it weighs only 1 gram, any thoughts on why this is happening?

2

After some tweaking I managed to fix the problem by disabling players collision detection while tree is falling.