OnTriggerEnter detecting Collision but collision not happening

I am creating a cube on mouseDrag.

Starting point of Cube = First MouseDown say POINT1.

End Point or Length of the Cube is decided on MouseDrag say POINT2.

I have attached rigidbody and box collider to the cube.
I want to use this cube as a bat. The first POINT1 is fixed and POINT2 is moving.

Ball is moving in the screen. I want to hit the ball with this bat.

If I move the bat POINT2 slow and hit the ball then ball is deflected as per physics rules.
But If I move the bat POINT2 fast, ball is not getting deflected.

In OnCollisionEnter Method I am able to see collision of BAT and ball but ball is not getting deflected

The Screen shot of what I am trying to do is in below youtube link

Any help is appreciated.

Try switching the RigidBody property Collision Detection to “Continuous Dynamic” on the ball.

Ref: Unity Doc