I want the user to be able to select a character and my idea was, if I have a base character prefab, could I swap it out with a variant prefab of the selected character?
There are some objects of the prefab that are publicly referenced in other scripts, so my angle here is if the variants have the same objects (with altered properties), could I switch the prefabs without having to get reference to all the objects again?
I would recommend creating a separate script that contains references to each variant, and some way to access those from your character creator script. You may want to look into Scriptable Objects, but I think just creating a sort of library of variants and accessing them directly would work.
Each prefab will have its inter-references, like a Nav Agent or Collider, etc, self contained so you wont have to worry about reassigning them at runtime.
As for public references, like for example the character referencing a game master script, you may want to reorganize your code so that the characters are referenced by the external scripts and not the other way around. That way when you swap prefabs, your game master could automatically assign itself to that character. Does that make sense?