For starters with a toggle you should define it as:
#pragma multi_compile __ ON
Or even better:
#pragma multi_compile __ myFeature
“on” is implied by toggling it on and off. The “__” defines this as a toggle state without having to define multiple keywords (there’s a global limit of 256 keywords I believe). It will compile a version with and without your feature enabled.
Will I be able to toggle the #if ON calculations at runtime, or is this only for compile time?
Yes* you can toggle at runtime. Unity will compile all variants then just switch variants at runtime.
Do I actually gain any performance by doing this, if the shader has a bunch of characters with it applied to them?
Yes since you in theory are removing a large chunk of the shader code from certain variants. BUT you will be using more memory to store all the variants so it’s a trade off. Don’t have to worry if only a couplf of #ifdefs but remember that multi_compile compiles ALL variants which could easily be 10s of 1000s of shaders.
Does it work with material property blocks?
Nope AFAIK there’s no way to dynamically create/remove shaderLab properties. BUT shaderLab properties may not necessarily affect your final shader. You can include hundreds of properties and if they’re not used in the final vertex/pixel output they will be ignored by the compiler. Bonus fact: the shader compiler will also try and pack float properties into float4 properties. That said you CAN use #ifdef to throw out shader global variables. All compiler directive can only be used between CGPROGRAM and ENDCG as they’re HLSL directives not Unity ShaderLab. See here for more info: Preprocessor Directives (HLSL) - Win32 apps | Microsoft Learn
Bonus fact:
Go here and read about concatenation in the remarks section 
PS: Sorry this is HLSL info, I’m sure it’s a similar story for GL platforms.
If I have a bunch of material instances running the shader, will it compile them twice for each character?
Not if you’re using multi_compile, each shader variant will be pre-compiled.
*You might also consider shader_feature instead of multi_compile. They work exactly the same way except one key difference:
multi_compile will compile ALL variants.
Pros:
Can switch to any variant at runtime since you’re guaranteed all variants will be included in your build. This would be a good option for a smaller number of variants that are commonly switched between at runtime
Cons:
Takes up more memory. If you have a very complicated shader you could take up hundreds of MB of video memory.
shader_feature will only compile USED variants.
Pros:
Less memory. can have a lot of #ifdef’s and not worry about variant count.
Cons:
Only variants used by materials at build time can be used at runtime. You can however manage this with ShaderVariantCollection Unity - Scripting API: ShaderVariantCollection
A good reference is to download the standard shader source and look at how it’s put together.