Keeping an raycasting AI within a patrol area

I have an AI that uses three Raycasts to detect objects in front of it, picture below for visual reference. When the Raycasts detect a collider, the AI turns to avoid it. Works great, except here’s the problem. I need to “fence in” the AI with something with colliders (planes for example) but the player can’t walk through anything with colliders on it. Turning the plane’s colliders into OnTrigger doesn’t do the trick, and I’m out of ideas.

	void CheckSurround()
	{
		Ray forwardRay = new Ray(this.transform.position + _colliderData.radius*transform.right, this.transform.forward);
		RaycastHit hitObj;
		//eventually replace the number with currentSpeed once the Ai can turn
		float distanceCheck = (currentSpeed)*predictTime;
			if (distanceCheck != 0 && Physics.Raycast(forwardRay, out hitObj, distanceCheck))
			{
				//if there's a problem with braking, check the && hitObj.collider.CompareTag("tree") code and make sure it applies to braking
				Brake (Vector3.Distance (this.transform.position, Vector3.MoveTowards(hitObj.point, this.transform.position,2f)/distanceCheck));
				//Check direction
				//Check to the right
				Ray rightRay = new Ray (this.transform.position + transform.right, this.transform.forward + (this.transform.right * visionWidth));
				RaycastHit rightCheck;
				if (Physics.Raycast(rightRay,out rightCheck, distanceCheck) == false && hitObj.collider.CompareTag("tree"))
				{
					Turn(1f);
					return;
				}
				//Check to the left
				Ray leftRay = new Ray (this.transform.position + _colliderData.radius*-transform.right, this.transform.forward + (this.transform.right * -visionWidth));
				RaycastHit leftCheck;
				if (Physics.Raycast(leftRay,out leftCheck, distanceCheck) == false && hitObj.collider.CompareTag("tree"))
				{
					Turn(-1f);
					return;
				}
			
				if (rightCheck.distance > leftCheck.distance)
				{
					Turn (1f);
				}
				else
				{
					Turn (-1f);
				}
			}
			else
			{
				Accel();
				BrakeTurn();
			}
		}

10230-airaycast.jpg

Maybe you can try making invisible gameObjects. Then, you add them to their own layer. Finally you would use this code.

if(hit.transform.gameObject.layer == fence)

with hit being the variable you store the Raycast information in. It should work because it does not use colliders.