How to use Physics.OverlapSphere with layer mask?

I faced some troubles about Physics.OverlapSphere(). I want my Physics.OverlapSphere() only detect the gameobject in the particular layer mask, if not it will detect everything including those gameobject that don’t have rigidbody or collider.

I know can use layermask to fix this, but why it didn’t detect the object I want??
here is the code…

neighbourObject = Physics.OverlapSphere(transform.position, sightRadius, 5);

ps: currently learning flocking, want to detect the neighbour around the boid, but failed…

please help and thank you.

I think the problem is the layer mask;

int layerId = 8;
int layerMask = 1 << layerId;
Collider[] colArr = Physics.OverlapSphere(transform.position, radius, ayerMask) ;

If you are using this for flocking overlap sphere may not be such a good idea if you are using it all the time. It is a costly operation.