how to push an object

I have an object that I push when I touch it and I press E, but it only pushes if I am walking forward, I would like it to work also if I am standing still but leaning against it
public class KickDoor : MonoBehaviour
{
public float KickForce = 1;

private void OnControllerColliderHit(ControllerColliderHit hit)
{
    Rigidbody _rigg = hit.collider.attachedRigidbody;

    if(_rigg != null && (Input.GetKey(KeyCode.E)))
    {
        Vector3 forceDirection = hit.gameObject.transform.position - transform.position;
        forceDirection.y = 0;
        forceDirection.Normalize();

        _rigg.AddForceAtPosition(forceDirection * KickForce, transform.position, ForceMode.Impulse);
    }
}

}

This because your code is using your position to move the object.

What would be more recommended, is telling said object to move along a certain distance(also direction) when you give said command. If you just want to press E once and have the animation do it’s thing, then set that within the object that once user gives the “ok”, to move in a specified direction, at a certain speed, and to stop when reached the full distance.

Now, if you ask on how to do that? It would take a good bit of typing, but I’m down if you truly need it.