Detemine wich side of a cube was hit

I do a raycast an use the point attribute from RaycastHit.point to call a function

private string HitSide(Vector3 point)
{
    string side = "none";
    if (Physics.Raycast(point, Vector3.down, 1.0f))
        side = "top";
    else if (Physics.Raycast(point, Vector3.up, 1.0f))
        side = "bottom";
    else if (Physics.Raycast(point, Vector3.forward, 1.0f))
        side = "front";
    else if (Physics.Raycast(point, Vector3.left, 1.0f))
        side = "right";
    else if (Physics.Raycast(point, Vector3.right, 1.0f))
        side = "left";

    return side;
}

unfortunately it returns always "top". I think the approach is wrong. Has someone a idea how to determine wich side of a cube was hit?

I always go about these things the simplest way I can, I guess it's because I'm from an artist background.

I would personally set six empty game objects as children of the cube, have no collider on the cube and create six colliders that are aligned to each side of the cube. Then you can do whatever you want when each side of the cube is hit.

I imagine that would be cheaper on the CPU than raycasting?