You can unregister any delegate without getting an error, whether it is named or not or registered or not. (Why, I don't know, possibly for efficiency reasons or simply because it is not required by the C# standard).
By holding a reference to the delegate, you can successfully unregister it:
using UnityEngine;
public class LogItInAwake : MonoBehaviour
{
delegate void EventHandler();
EventHandler LogIt;
void Awake()
{
EventHandler x = () => Debug.Log("It");
LogIt += x;
LogIt -= x;
LogIt();
}
}