Mouse Click

Hello there,
I have my targeting basic system:

if (Input.GetMouseButtonDown(0)){
		if (Physics.Raycast(Camera.main.ScreenPointToRay(Input. mousePosition), hit, 100)){
			target = hit.collider.transform;
			isTargetIsAttackable = target.GetComponent(NPC);
		}

But there is one big problem on this. I rotate my camera with mouse click (click and drag). Whenever I click to rotate a camera, it sets a new target, a floor for example. Can I somehow lock my target, that when player will drag and rotate camera, target will stay the same?

So I did it as robertbu said.

Record the camera rotation when the mouse goes down. Do the targeting when the mouse goes up, but only if the delta rotation of the camera is below some threshold.

Thank you.