Best way to determine which "face" was hit for a box collider

I’m writing some code where a player interacts with a given object (in this case a fence to vault over) and want to determine if my raycast is hitting the front/back of it or the sides/top.

Currently this is my attempt:

	public static bool IsFrontOrBack( Collider collider, Vector3 hit )
	{
		Vector3 localHit = collider.transform.InverseTransformPoint( hit );
		return Mathf.Abs( localHit.z ) >= collider.bounds.size.z / 2.0f;
	}

It appears to work but is certainly less than ideal - for one it is possible (if unlikely) that you could see a high enough z value using a raycast from the side (furthest corners). Is this as good as it gets or does anyone have something cleaner kicking around?

Use ContactPoint.point
in OnCollisionEnter, OnCollisionStay OR OnCollisionExit

This is a ray cast use case so no collisions unfortunately.