How to exclude textures from QualitySettings.masterTextureLimit?

Whenever I try to change QualitySettings.masterTextureLimit it messes up a font texture, and makes it unreadable. Is there any way to exclude the font texture from the setting?

Turn off mip maps