Physics.Raycast Not Working All Of The Time

Hi everyone,
I am making mountain biking game where wooden ramps randomly spawn across the terrain, but I have an issue. My code finds smooth regions of ground to place ramps on, so it is possible that several ramps are created very close together.
To fix this, I implemented a short raycast from the centre of each new ramp to the left of it to check if a ramp is already there; if this is the case, the previous ramp is deleted. The problem is that the raycast only occasionally works.
Here is the code:

Transform raycastOrigin = ramp.transform.GetChild(1);
Ray ray = new Ray(raycastOrigin.position, -raycastOrigin.forward);

Debug.DrawRay(raycastOrigin.position, -raycastOrigin.forward * 75, Color.red, 100);
if (Physics.Raycast(ray, out RaycastHit hit, 75, LayerMask.GetMask("TerrainObjects")))
{

    Debug.Log("Removed Duplicate Ramp");
    Destroy(hit.collider.gameObject);
}

Where raycastOrigin is the transform of an empty gameobject attached to the ramp prefab at its geometrical origin, and TerrainObjects is the layer that all ramps are part of. Although it sometimes works, there are occasions where the following happens:


The ray clearly ignores the ramp in front, meaning it does not get destroyed.
If anyone can tell me why this happens, it would be greatly appreciated.

I assume you only do the check in Start()? The image above would support the ramp on the right being created first? Sorry to state the obvious, but for me it is best to confirm this first!