click to move problem ( some clicks are ignored )

Hello , I have an issue trying to make a moba like game , some clicks seems to be ignored randomly , and its not related to the clicking speed ( or at least it doesnt ignore every fast clicks but I don’t really know how the fixedUpdate works).
Anyone has an idea about what can cause this ?
Here is my code :
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.AI;
public class navmesh : MonoBehaviour {

NavMeshAgent agent;
RaycastHit hit;
// Use this for initialization
void Start () {
	agent = GetComponent<NavMeshAgent> ();		
}

// Update is called once per frame
void FixedUpdate () {
	if (Input.GetButtonDown ("Fire2")) {			
		if (Physics.Raycast (Camera.main.ScreenPointToRay (Input.mousePosition), out hit, 100)) {
			agent.destination = hit.point;
		}
	}
	if (agent.remainingDistance < 1) {
		agent.velocity.Set (0, 0, 0);
	}
}

}

Thank you!

Any input needs to be handled in Update() instead of FixedUpdate(). FixedUpdate isn’t typically fired as often as Update is so you can miss some inputs.