TextMeshPro - Out of Memory crash when loading incompatible font file

Unity Engine
1

TextMeshPro (1.5.1 in Unity 2018.4.27f1) causes a hard crash on iOS when attempting to load an incompatible OpenType font (i.e. AppleColorEmoji-160px.ttc).

  1. Do we have any functionality (in TextMeshPro, Unity Font class or elsewhere) to know that a font file is incompatible with TextMeshPRO and not attempt to load?

  2. (longer term) Can we please add a feature in TextMeshPro where it can fail to load a FontAsset, instead of crashing?

Relevant code:
Font osFont = new Font(name); // “name” returned by Font.GetPathsToOSFonts()
TMPro.TMP_FontAsset fontAsset = TMPro.TMP_FontAsset.CreateFontAsset(osFont); // crash happens before this call returns, if name points to incompatible font i.e. AppleColorEmoji-160px.ttc

2

An error should be issued when trying to load an Invalid or potential incompatible font file.

I will take a look at this particular font file to figure out what might be happening here and follow up once I have more information.

3

In the Editor, I am getting a warning when trying to create a font asset from this font file via the Create context menu.

I am also getting a warning when using the Font Asset Creator as well.

I am also getting a warning when trying to create a font asset using the CreateFontAsset() function.

Note: That I was only able to find the “Apple Color Emoji.ttc” file and not the -160px.ttc you are referencing. So perhaps this file behaves differently.

Can you provide a link to the .ttc file you are using?

4

My apologies, the crash seems to be caused by out of memory because of a leak of some kind, not because of parsing that particular file.

I would still like some help with the memory leak issue. In the Editor’s profiler, the growing memory usage is not attributed to any assets or textures or anything - it just goes under “Unity”.

I have confirmed the following simplified code is causing the leak. I also confirmed the leak is because of the TMP asset, not the Font which is deallocated fully with the GameObject.Destroy call.

Font osFont = new Font(fontName);
TMPro.TMP_FontAsset fontAsset = TMPro.TMP_FontAsset.CreateFontAsset(osFont);
…
…
GameObject.Destroy(fontAsset);
GameObject.Destroy(osFont);

Is GameObject.Destroy the right method to fully unload a TMP Font Asset?

5

I had a chance to take a closer look at this and will need to make some changes.

First, since this font file contains bitmap strikes (as opposed to scalable outlines), the FontEngine is able to load the font face but returns an error related to this invalid size. The first change, I will make is to issue a more relevant error to indicate the invalid size.

Second, although we did get an error, the face was still loaded and cached by the FontEngine in anticipation of trying to access glyphs from it. As such, it will remain in memory until the FontEngine releases it when it is destroyed.

In addition to the first change described above, I will also automatically dispose of a face when the FontEngine is unable to scale it to the requested size.

In the meantime, the CreateFontAsset() should already be returning null with similar font files. However, since in this case the face was already loaded, you will need to manually call FontEngine.DestroyFontEngine(); to release all cache font files.

As such, I would suggest adding a null check and then manually calling the above function. Let me know if you manually calling works FontEngine.DestroyFontEngine(); as expected on your end?

I will also consider adding some new API functionality to allow releasing specific fonts from the FontEngine cache.

Note: FontEngine changes require new versions of the Unity Editor.

6

Adding UnityEngine.TextCore.LowLevel.FontEngine.DestroyFontEngine(); after the GameObject.Destroy calls definitely fixes the leak. Thank you.

Is the FontEngine mainly freetype output? What is the performance implication of having the font engine destroyed periodically during runtime? Are static and/or dynamic font textures affected? (used by TextMeshPro or the regular UnityUI text).

7

Is there any chance FontEngine.DestroyFontEngine behaves differently on iOS? While in the Editor the memory leak is fixed, on device I get the exact same out of memory crash as if the DestroyFontEngine call was never added.

8

The function should behave the same on all platforms.

Can you provide me with a link to the font file you are using for testing?

9

I will provide the large list of fonts when I have a chance. In the meantime, may I ask another question:

What is the hardcoded list of fonts Unity is looking through per platform, to find missing glyphs in the regular UGUI system? (This hardcoded list is referenced here:
Unity - Manual: Font assets)

That system seems to efficiently find glyphs from system fonts installed on the end user’s computer, and it takes no developer effort to have that work. Trying to implement a similar system with TextMeshPRO on the other hand is proving very difficult. Perhaps if I was using the same font list Unity uses for the regular text system, the task would become easier.

10

I believe the issue you are running into is specific to that emoji font which is special in the sense that it contains bitmap strikes and is likely very large in size.

I will have to look up that list again. I did post it in the past but the forum search feature is being difficult today.

Having said that, can you provide more insight on how you are implementing this?

Are trying to load / look thru every potential system fonts?

11

This is the function:

    // Every dynamic (unknown at compile time) text the game needs to display, i.e. chat or news,
    // goes through here. This function will try to find the system font appropriate to display
    // any characters that are not currently available through the TextMeshPro fonts loaded so far.
    public static void ExamineDynamicTextAndLoadAdditionalFontsIfNeeded(string dynamicText)
    {
        uint[] missingchars;
        if (!Main.Instance.MainFontAsset.HasCharacters(dynamicText, out missingchars, searchFallbacks: true))
        {
            uint[] newMissingChars;

            for (int i = 0; i < DynamicFonts.Count; ++i)
            {
                var fontAsset = DynamicFonts[i];
                if (fontAsset.TryAddCharacters(missingchars, out newMissingChars))
                {
                    missingchars = newMissingChars;

                    if (newMissingChars == null || newMissingChars.Length == 0)
                        return;
                }
            }

            foreach (string fontName in AllowedFallbackFontNames)
            {
                if (string.IsNullOrEmpty(fontName))
                    continue;

                Font osFont = new Font(fontName);

                if (osFont == null)
                {
                    //ClientLog.logMessage("Failed to load font " + fontName);
                    continue;
                }

                TMPro.TMP_FontAsset fontAsset = TMPro.TMP_FontAsset.CreateFontAsset(osFont);

                if (fontAsset != null && fontAsset.TryAddCharacters(missingchars, out newMissingChars))
                {
                    string msgLog = "Adding fallback font " + fontName;
                    ClientLog.logMessage(msgLog);
                    Debug.Log(msgLog);

                    Main.Instance.MainFontAsset.fallbackFontAssetTable.Add(fontAsset);
                    DynamicFonts.Add(fontAsset);

                    if (newMissingChars == null || newMissingChars.Length == 0)
                        return;

                    missingchars = newMissingChars;

                }
                else
                {
                    if (fontAsset != null)
                    {
                        GameObject.Destroy(fontAsset);
                    }

                    GameObject.Destroy(osFont);

                    Debug.Log("FontEngine.DestroyFontEngine");

                    // https://discussions.unity.com/t/810708
                    UnityEngine.TextCore.LowLevel.FontEngine.DestroyFontEngine();
                }
            }
        }
    }

Initially it was going through every font returned by Font.GetPathsToOSFonts(). Then I reduced the list to only the fonts that contain the following text (case insensitive), this list is similar to a font list I found in the code auto-generated by Unity for iOS. Both versions crash when dealing with large amount of missing glyphs, will look into it more tomorrow.

“arial.”,
“arialhb.”,
“helvetica.”,
“helveticaneue.”,
“applesdgothicneo.”,
“hiraginokakugothic.”,
“pingfang”,
“kailasa.”,
“geezapro.”,
“laosangammn.”,
“kohinoor.”,
“kohinoortelegu.”,
“kohinoorgujarati.”,
“kohinoorbangla.”

12

Since most platforms include more than 100 fonts which come in all shapes, design and size, scanning thru all these fonts looking for individual glyphs is not a viable solution in my opinion.

Instead, I think developers / designers should carefully select what fonts they wish to use on these platform and for any given language or groups of languages. This would ensure the selected system fonts not only include the appropriate glyph coverage for the target languages but also work well from a design point of view with other fonts used in the application.

Most platforms do publish a list of fonts available on their platforms. Here is a link to the fonts available on Apple platforms. No doubt a similar list is available for Android.

Alternatively, one can always use Font.GetPathsToOSFonts() on those different platforms to get a list of those fonts to then make their selection of which fonts they will specifically load / use at runtime and for any given language or groups of languages.

There are many popular fonts available on Android and iOS such as NotoSans which have coverage from most languages.

Basically, I recommend compiling a list of the various fonts you will be searching on specific platforms and for specific languages.

“Roboto”,
“Noto Sans CJK SC”,
“Noto Sans Mono CJK SC”,
“Noto Sans SC”,
“Noto Sans CJK TC”,
“Noto Sans Mono CJK TC”,
“Noto Sans TC”,
“Noto Sans CJK KR”,
“Noto Sans Mono CJK KR”,
“Noto Sans KR”,
“Droid Sans Hangul”,
“Noto Sans CJK JP”,
“Noto Sans Mono CJK JP”,
“Noto Sans JP”,
“Droid Sans Japanese”,
“MotoyaLMaru”,
“NanumGothic”,
“Droid Sans”,
“Droid Sans Fallback”,
“DroidSansFallback”

Note that I am planning on introducing “Dynamic OS” font assets where in the Editor, they will use whatever font you have assigned to them and at runtime on the target platform lookup that font by name. There are more details but the plan is to make it much easier to work with OS fonts.

Until then, I would suggest a revised approach to your implementation when it comes to searching font files for potential characters.

Instead of creating a new Font object and then creating a new font asset, I would use

public static FontEngineError LoadFontFace(string filePath)

to load the font using the path returned by PathsToOSFonts().

then use the following function to check if a valid glyphIndex is returned. Zero means a missing glyph.

public static bool TryGetGlyphIndex(uint unicode, out uint glyphIndex);

Then when you know the font file does contain the unicode you need, create the font object and font asset.

Doing the above should be more efficient.

The above is still not ideal. As a result of this thread, I will be adding a new function UnloadFontFace(); to unload the previously loaded font face. This would make it much more efficient to iterate through lots of font files to peek into them and then unload them when needed.

Although this new UnloadFontFace() function is not available, you can use FontEngine.DestroyFontEngine(); but since you will no longer be creating a Font object first and then a font asset, you should be able to load several fonts to check for those glyphIndex and then destroy as needed. Ie. Not sure I would destroy after each face loaded.

Note, I am currently testing all of this but wanted to give you some feedback while I finish testing / exploring all of this. Ie. Standby for more information … So give this a try and let me know how it behaves.

13

I assume TryGetGlyphIndex doesn’t operate just on the font loaded via LoadFontFace, but on the entire text engine and all fonts loaded so far, is that correct?

If that’s true, it won’t work for my case. I am introducing TextMeshPro gradually on a large project that uses a lot of UGUI and it’s possible some glyphs exist on fonts used outside of TextMeshPro.

I understand there’s no good solutions for what I’m trying to do, but I do want to stress that it should be a long term goal for the text system (which I assume will eventually be 100% TextMeshPro) to support any character the user can type natively. This includes all languages (if they typed it, there’s a system font that supports it), and native emojis (even the dark skinned ones). In terms of what the vast majority of actual users expect (end users that play the games), this would be a far higher priority “next gen” feature than rendering 10% more triangles or whatever the rendering people are actually working on.

14

No. It operates on the currently loaded font face.

The idea behind using that is to initially skip the creation of many Font objects and Font Asset until you know the glyphs you need are present in that font file at which point you will create a Font object and font asset knowing they contains the requested glyphs.

The challenge here is that font selection varies per platform and that many fonts may contain the requested character. So when looking for such character, do we use some hardcoded list or do we look through fonts first come first serve.

Since fonts come in all shapes and size, we might also end up with fonts that do not work well with each other from a design or metrics point of view. For instance, imagine you picked Impact.ttf which is a heavy type and then as a user types a Chinese character and you end up with a skinny type which would look horrible.

What I want to avoid is (Unity / me) defining some hardcoded list or this random selection of some system font and instead give you control over this process just like we currently can with Primary font assets, local and global fallbacks. This will require the developer be aware of what fonts are available on those platforms but give you 100% control over it all.

With the introduction of Dynamic OS font assets, you would be able to assign those as local or global fallbacks. The fallback search sequence would remain the same as today but with the additional step of checking if the font file referenced by the Dynamic OS font asset exists in the current platform. If it does not, it would move to the next potential local or global fallback.

The above would give you complete control over what font asset and font gets used and when. Since these are pre-created in the Editor and persistent assets, you would be able to adjust any of their settings including using different sampling point size, atlas texture size, etc. and even face metrics to ensure they work well with each other.