Raycasting using a Sphere(ball) and a cube(door)?

How can I make raycasting work all the time.The door opens when I am at 5 meters from it, but not from all the angles, I must move the ball in order to open the door.I control the ball from the arrow keys, but the door doesn’t open every time, for example I am facing the door but it doesn’t open and I have to go backwords to make the door open.

You shouldn’t use raycasting for this. It’s much simpler (and requires less computation) to use an AABB bounding box around the door, then just check when your object is inside that AABB. You’ll always have trouble with the ball or any object that requires a raycast (ball in particular since it’s likely rolling it’s pivot around all over the place).