Raycast push? How?

Hey guys I'm trying to make a script that when both the fire1 input is down and a raycast is hitting an object, it pushes it away from the player, like as if it got hit by a bullet. Obviously the best way to do this would be to assign the object a rigidbody and have it so that it applies a force, but how do i make the direction of applied force be AWAY from the player?

direction = endpoint - startpoint


force = (pushedObject.position - player.position) * scalarMultiplier