For others hitting the latter “Couldn’t destroy game object because no network view is attached to it.”, it seems another possible cause is using Network.Destroy() instead of NetworkServer.destroy()
You should not really be using NetworkView.Find, instead try explicitly supplying the referenced network view.
As a rule of thumb, any of the .Find or .GetComponent functions are searches and therefore should be avoided at all times unless absolutely necessary. Searching through collections is almost always more expensive than just pointing to the correct object (assuming you know how to find it yourself / can add it as a reference to the script explicitly).
Not only will .Find and .GetComponent functions cause a slow down, but they are also notoriously unreliable and will do EXACTLY what they say, for instance if the function says “returns first object of type found” then it will do exactly that, even if the object is miles away from the actual object your looking for, simply because they are the same type etc.
Generally controlling your own references will cause less errors, and will keep your memory footprint down at runtime.
EDIT:
Also instead of destroying the object, set it to disable and then set its disable method to destroy it, something like :
if (goID != null) {
goID.observed.gameObject.SetActive(false);
}
and then in the script attached to that gameobject
void OnDisable()
{
Destroy(this.gameObject);
}
This will cause the object to disable itself, and then destroy itself automatically when disabled.