The way I fixed this is by drawing my gizmos in an editor script, and replaced the UI with Unity Handles text. Now my debugging can all happen inside the Editor folder.
I did not know this could be done, but here is another thread where I ask how to do this: