By the way, is it possible To just set Text Mesh Pro to fallback to system fonts whenever an unavailable language is used, the same way Unity regular text does?
It is currently possible to create font assets from system fonts but the “automatic fallback to a system font” is not currently available but most certainly something that I want to provide in some form but…
I am not a fan of the “Fire & Forget” approach where the system magically falls back to some system font as this hides from the developer what is actually going on behind the scene. Ie. that whatever font you selected is missing some characters which you might only discover after shipping your product.
You also have the case where in your dev environment (Windows for example) whatever font ends up being used works fine from a visual and metric point of view with the primary font you selected but on some platform because some other system font is used, you end up with some other font that does work well with your primary. Again something you uncover after shipping. Ie. something you would have instantly notice in the Editor when the missing glyph character is displayed where you would be able to do something about it.
So the challenge is …do I do magic stuff in the background that hides the fact your primary font is missing coverage for some languages which I agree is convenient but could be very costly to uncover after shipping.
1 Like
I guess you could do both. Like in my case fallback would be great - simple and easy to vet because I go through all my screens in my built game anyways on my iOS devices. But I get what you mean about offering a more complex but safer version. I guess the real issue is there’s no standard yet so we have to compromise somewhere - complexity for safety or simplicity for unknowns. Maybe one day Earth will have a unified language we can all speak - that’ll probably make game dev, and all development - a lot easier;)
I installed NotoCondensedMedium. Im getting white boxes instead of Chinese characters, but I am sure I am missing a step. European and Russian text shows up fine with this
This is a screenshot of what I did. Do I need to do something special? Like check something to allow for Asian characters?
The reason why the glyphs don’t show up is simply because this NotoSerif-Condensed font does not contain the CJK characters.
The following NotoSerifSC-Regular for Chinese Simplified (SC) should work. There is another version for Chinese Traditional (TC). Here is a link to the combined NotoSerifCJK.
In the end, it comes down to picking the right font with the appropriate glyph coverage for the given language.
Should developers have to worry and learn about this?
Let me answer with another question.
Should developers have to worry and learn about Garbage Collection?
As game developers, mostly indies, we have to learn about all of these topics. Like efficient modeling, light baking, efficient coding, etc. We don’t need to becomes masters but we certainly need to learn enough to become proficient.
P.S. I use tools like High-Logic Font Creator or similar tools to look at font files to see exactly what characters and features they contain.
Often users rely on some font viewer which most of the time does glyph substitution leading them to believe they contain character they actually do not contain.
1 Like
There’s a lot of great advice here - thanks!
I used the NotoSerifCJK - thank you for the link:)
Im not seeing Chinese characters in my TMP text. Is there something in my screenshot that’s incorrect?
I think maybe its the Character sequence or something? I feel like im just missing a step or something.
BTW this is the game I’ll be using it in. It’s out on iOS and there are players around the world, and many of them are asking for translations. Players have been really nice helping me to translate words.
How are you creating the Font Asset?
Looks like you are still using the Font Asset Creator which is fine to create static font assets when you know exactly what characters you need. However, until you have a list of all the known characters used in the project, broken out per language or groups of language, just create and use a dynamic primary font asset created from this NotoSerifCJK as per the following steps.
(1) Import the NotoSerifCJK font file in the project. You have already done this step.
(2) Select the font file and press CTRL-SHIFT-F12. The new dynamic font asset should be created.
(3) Select the newly created dynamic font asset and enable Multi Atlas Texture in the Generation Settings
(4) Assign this font asset to the relevant text objects.
Then start typing away and all the characters should show up.
In the Editor while you are developing the project, all changes made to this font asset will be persistent. This font asset will grow as needed and might end up using multiple atlas textures which is fine while you are developing the project. As you get closer to release and have a list of all the known characters used in the project broken out per language or language groups, you will be regenerating / creating new font assets to now use a combination of static and dynamic font assets but while you are developing, just run in full dynamic mode.
This is a short summary of the new information and revised workflow that I will be covering in the update Font Asset Creation video.
2 Likes
Here is a video that I created tonight to help provide better insight on the font asset creation process and workflow.
P.S. I am still working on a more polished video. This is the quick / rough version to help you out.
3 Likes
It worked:)
I can’t believe you made an entire 20 minute on the fly video like that. It was very clear and helpful, and Im super impressed at how you covered everything - you have a mind like a steel trap Stephan.
Whats nice is that my game loads translations from a csv file at runtime. It looks like all I have to do is run the game, change the language and open up each screen with text once to fill up the atlas. It works really well.
I think especially if you have a known set of text and high quality assurance goals this is the way to go.
For my case though I am continually adding in new languages - I added Thai last night - and I think its too much complexity for me as an individual to find a font that covers Thai and more as well. The regular unity text is handling all the languages and I may have more coming. I think for the future if TMP could fall back if the font isn’t included that would solve my particular case issue. But I have to say the TMP is very tempting because of the simple ways to style it and the low draw calls.
I think for now I need to stick to Unity UI text because of my specific case but I do recommend people watch your tutorial because now I feel like I understand very clearly how dynamic Text mesh pro works whereas before I was very unclear. You really did an amazing job with that and I can see you’re as passionate about helping us out as we are about making games. So thank you very much.
One last question, for the outline component, when you say its bad for performance, is that only because it creates more vertices/geometry? or is there also some kind of scripting/update performance hit happening every frame. Im asking because I can afford to have it on the text in each screen, theres; only 6 to 20 texts on each screen and only one screen canvas is enabled at a time. But if there is some kind of update that is continually making the outline component hamper performance even when the canvas is disabled then I need to think about that.
Forgot something very important… The legacy text system and UI Text does not support proper line breaking for Chinese, Japanese and Korean text which TMP does.
Again, do not be fooled into believing UI Text will be able to magically handle this in the background. Its ability to display Thai is related to Windows as an operating system including over 200 fonts where at least one of them includes support for Thai. This may not be the case on your target platforms / devices. Then you have to understand that the Thai font used when testing on Windows is likely to be different per platform and devices. The font that the legacy text system ends up picking is somewhat random in the sense that it is likely to pick the first font it encounter (alphabetically that contains the character you seek).
Then you have the limitation of the legacy system using a single atlas texture where the atlas will need to contain a copy of each character and for each point size. This quickly fills up the atlas texture where in no time it ends up being 4096 x 4096. This issue is strictly related to the amount of text and point sizes being used. If all the text is the same point size and the number of characters displayed at any given time is small, then this will not be an issue.
In terms of adding support for new languages each day. It comes down to picking a font for the given language of the day. In the case of Thai, this task is trivial given the NotoSans and NotoSerif font family includes font for pretty much all languages, the process consists of (1) Downloading the Thai NotoSerif font (2) Unzipping the NotoSerifThai-Regular.ttf font in your project. (3) Create a dynamic font asset using the new font file by simply selecting it and using CTRL-SHIFT-F12. (4) Lastly assign it as a fallback to your Master / Primary NotoSerif font asset that contains all the Latin characters.
On the next day you would repeat the process for the new language of the day. After a bunch of days, the Fallback list of your Master / Primary NotoSerif font asset would contain a reference to each of those language specific dynamic font assets.
Using the above approach enables you to continue development and adding support for any given language. It allows you in the Editor to make sure all of your text is displayed correctly and guarantees everything will look the same on all platforms and devices since you picked and control the entire process. During development, you only use dynamic font assets which you will later switch to a combination of static with dynamic fallbacks. (Will create a new video to cover this specifically)
The underlying geometry will be 5 times larger which impacts rendering with the addition of extra processing as every time the text changes, this Outline component needs to make 4 copies and offset those. In the end, expect about 5X performance difference between UI Text with Outline component vs. SDF text with an Outline. Adding the shadow component will further impact all of this whereas adding a Shadow with SDF doesn’t change anything.
The main drawback with the above is having to include these font files in the project which will increase build size. There are ways to get around this by using AssetBundles or Addressables. Font assets can also be created at runtime using system fonts but I still have work to do on that before I am satisfied with it.
… now working on the video.
1 Like
You make a really good case for Text mesh pro:) I’m really torn.
Im going to try it again with the Thai fallback you mentioned. If the created font assets aren’t too large mb wise and it works maybe I will stick with TMP because it looks nice and as you said it makes sure that everything will display as is. Thanks so much for all the information about this. I am sure it can’t be easy but hopefully this will help more people.
In my project for now I’m going to keep both Unity UI text and TextMesh Pro text until I figure this out.
You were very helpful explaining the system fonts: I checked to make sure and both Android and IOS support the system fonts im using with UI text including Thai. For anyone targeting modern mobile id say it is pretty safe, plus I think for example most people who speak Thai will have Thai as a system font installed on their device. All my text is static so there’s no typing anywhere and its all the same point size so that’s ok, and I don’t use any shadows on my text, just the outline.
One other issue I noticed though is with Unity UI text I can use any font + the system fallbacks so I have more control. With the TMP I think this impossible too? I guess I have to use my non-asian font as before, then create fallbacks just for cjk and Thai fonts in the Noto fonts (or any cjk/thai font). Like basically we can have as many fallbacks as needed, memory permitting.
Small note: Im on Mac and it’s CONTROL+SHIFT+FN+12 to generate the dynamic font asset.
PS that’s cool about the TMP line breaking
I will have the new video available in the next 30 minutes which will cover what I described above.
1 Like
I think I get what you were talking about before
If I want to use
- Itim.ttf gogle font for all European languages
- Noto for everything else
I create a static font asset for Itim
I create 2 fallback dynamic assets for CJK and Thai, using the only cjk (no euro) and Thai (no euro)
I fill up the dynamic CJK/Thai assets with my game text by playing it at runtime and opening each screen with that languages text (tried this, works)
Then when I run it, it will display Itim for euro and fallbacks for the others.
Ill try it:)
During development, it can be dynamic to keep things simple but before releasing your project you would want to regenerate it as static where it would include all the Latin characters used in the project.
Correct. These additional dynamic font assets, should be assigned as fallback to your Primary / Master font asset created from “Itim.ttf”.
Then when you need to add support for an additional language you repeat the process you did for you CJK and Thai.
1 Like
Here is the other video …
2 Likes
Thank you - im watching it now:)
Is this the right font? Its something called a .ttc font and its 24 mb. Is this the correct files size (it seems really big). The Thai font is only like 500kb. Is it just because CJK has so many letters in the font?
Wow. Stephan your tutorial was fascinating and really well spoken. Do you create all these without writing down what you’re going to say first? I don’t know how you did this and these seem like finished polished tutorial to me, not on the fly temporary ones. And when you are writing in character codes for omegas by memory - ! You really know fonts.
Text Mesh Pro with fallback languages is definitely more complex to learn. But once you watch this it all becomes really clear. I have it working in CJK and Thai and im glad you took the time to explain local and global fallbacks and search order. I think it was important for me us to know that because it opens up a lot of possibilities.
I have some questions:
To fill up my dynamic atlas, Instead of running my game, switching toeach the 4 asian langauges and opening up each option screen, can i just export my 4 language texts from my spread sheet to a text file and import that into my atlas? Like feed unity a text file with all my CJK and Thai letters and use that. It would save time
The outline on my CJK and Thai letters is thicker than the European text. Im guessing this is font specific. Can I change the outline to be thinner on just my CJK and Thai characters? Or do I have to experiment with differ fonts? Like maybe if I use a CJK light font instead the outline will more closely match my Itim font outlines.
Will it save memory to remove default sprite asset from tmp settings if i dont use emojis? Do you think its worth it?
Do you think is it better /more efficient to use separate Chinese, Korean and Japanese fonts as fallbacks? instead of one big CJK font that contains all 3?
Should I leave my European item main font as static? And after I fill up my fallback dynamic atlases, can I switch the dropdown too static before I build it. or just leave it dynamic?
Small maybe bug - when i added the CJK fallback the direction switched right to left. I found and unchecked the RTL checkmark and fixed this, but was unsure if something was going on here. Looks ok though.
can i check the font SDF atlas size without building the game and looking at the build log? I think my only remaining concern is that the atlas sizes will be too heavy memory wise and crash my mobile game.
Can i replace liberation sans from The TMP SETTINGS asset safely? Willl this break anything in the editor? Like does Unity editor UI use Liberaton or something?
why not use static for my CJK and Thai fonts? because it’s too heavy memory wise? Like would stain mean it imports ALL the characters even if I don’t use some?
Does the font sdf atlas size affect performance and memory or just build size?
Im going to test this out and check how the memory performs but if it all checks out ill stick with Text Mesh Pro:)
I think you did absolutely great work here.
One of the options available in the Font Asset Creator to define what characters will be included in a font asset is the Characters from File as seen below.
This option allows you to specific a .txt file that contains the characters you wish to include in this static font asset. This is the option that many studio use to create their static font asset as they get further into the development cycle and being switching from running full dynamic to a combination of static font assets that contain all the known characters for a given language or groups of languages combined with dynamic font assets to handle user input for those same languages or groups.
So until you get further along (as well) in the development process, I would continue to run full dynamic with multi atlas texture enabled and would not try to fill those dynamic font assets as they will fill up gradually as you continue to work on the project.
Now if you really wanted to fill them up with characters, you could (1) use the Font Asset Creator with the “Characters from File” option and manually specifying the Sampling Point Size and Padding to maintain a consistent Sampling Point to Padding Ratio. Font Assets created in the Font Asset Creator are static by default but this can be changed in the Font Asset Inspector Generation Settings. Keep in mind that if these are full, there is no point to switching them to dynamic.
This is where the Sampling Point Size to Padding Ratio becomes important. In order for the Outline, Underlay and other visual treatment to remain consistent across your primary and fallbacks, they have to use the same Sampling Point Size to Padding Ratio. They can use different Sampling Point Size and padding but the ratio must be the same. For instance, if the primary is using 90 sampling point size with padding of 9 for a ratio of 10%, the fallbacks could be using a sampling point size of 60 with padding of 6 for the same 10%. This would ensure visual consistency while also saving texture space since the dynamic fallback is using a smaller point size. Most of the time since the characters contained in these dynamic font asset comes from user input which is usually smaller on screen, it is totally fine to reduce the sampling point size.
The default sprite asset is pretty small but you can safely replace it or remove it.
It all depends on your deployment strategy along with the number of known characters you will end with for those languages. For instance, if you are able to fit all the known characters for these 3 languages in the same atlas texture at good sampling point size / quality which is no larger than 2048 x 2048, then that is desirable. If you could fit each language in their own 1024 x 1024 combined with a dynamic fallback again at good sampling point size and quality that is even better. Keeping those separate might be easier to maintain. But this part is very subjective and sort of up to you.
You can switch from static to dynamic as needed. However, if the atlas texture is full, it doesn’t make much sense to switch to dynamic since the atlas is full anyway. However, as I said above, I would run full dynamic until it is time to make the switch and you have all languages and text / known characters nailed down.
It is likely that you inadvertently clicked that RTL option as font assets on their own have no impact on this. But enabling RTL will reverse the text direction.
The atlas size will remain exactly what you specific in the Generation Settings which by default is 1024 x 1024. Multi atlas texture will create additional one as needed but you can easily see this in the inspector.
Make sure you don’t use a sampling point size that is greater than 90 with padding of 9 as larger than this is rarely needed for Latin text. CJK characters can still look very good at 40 sampling point size with padding of 4 to maintain this 10% ratio.
Yes. Just make sure you replace it and don’t leave the field empty. A Default Font Asset is needed.
You will at some point and for each language or group of language end up with a Static Primary font asset that contains all the known characters for that language or groups of languages. You will pair this static font asset with a dynamic font asset that is using the same Sampling Point Size to Padding ratio as fallback to handle user input for those languages. This will be the same for Latin or CJK. Static Primary coupled with Dynamic Fallback where at ship time, this fallback will be empty with its atlas at size 0 x 0.
On the graphic size, it is more efficient to read from a smaller texture than larger texture. This varies greatly by platform / device. I would not use anything larger than 2048 x 2048. The smaller the better. What will determine this is ultimately the number of known characters for the given language or group.
Thank you.
P.S. I will probably create another video to cover the above.
1 Like
OK thanks to your help I have almost finished using Text Mesh pro for my CJK and Thai translations - thank you:)
Sometimes I notice square boxes as I open up screens - this is usually fixed by changing(lowering) the point size and padding. Im guessing its an artifact of using dynamic assets as I fill up the project.
Ive exported a FallbackCJK.txt and a FallbackThai.txt files that contain all the CJK letters and Thai letters I use. I set up my translations in Google Sheets so its easy for me to add new text as needed and to export these sheets, as well as my master csv right into my unity project: https://bit.ly/3bnknkg
Im still a bit unclear on how to import the Fallback.txt files I created. Right now I have my Static Itim font main TMP asset, with fallbacks to Dynamic CJK and Thai Noto san serif fonts (I switched from serif to sand serif because it think I looks cleaner o screen)
Whats the process to import my text files into the CJK and Thai assets?
Do I first switch them to static, because if I’m importing all the used characters I don’t need dynamic, correct? And then to I open the FONT CREATOR and or other I guess I click UPDATE FONT ASSET at the top of my newly static font assets, and in the font creator panel import the link to my text file, regenerate and that’s it? Then ill have a static font atlas with all my characters (and I can play around with the point size and padding keeping the 10% - thanks for that tip works great - in mind)
This is likely related to Shader Compilation as a result of the following being enabled.
Disable Async Shader Compilation and these squares will disappear.
When you need to update any of your static font assets, you should use the Font Asset Creator with the “Characters from File” option using you exported FallbackCJK.txt. Be sure to use the same ratio of Sampling Point Size to Padding.
Repeat the process for Thai.
You will create and assign an empty dynamic fallback your static CJK and Thai font assets to handle any potential characters you might have overlooked or that may come from user input,
Assuming the static ltim is your primary / master, you will then assign the static CJK and static Thai to it as fallback.
P.S. Like I mentioned a few times already, you don’t need to worry about creating those static font assets until you are getting closer to release. During development you can run with each of those on dynamic mode with multi texture enabled. Then, at some point, you can look to convert to the static setup described above.
1 Like