TMPro Dynamic Font Asset constantly changes in source control

Hi everyone,

The problem is that font’s atlas texture is a sub-asset of the font asset itself. This makes sense for static fonts.
But with dynamic fonts I don’t want to store this atlas on the disk, why would I?

The whole point of dynamic fonts is to constantly change, according to glyphs used in the game. And this doesn’t work well with any source control system. It’s very annoying behavior that annihilates all pros of dynamic fonts over static ones.

I would like to make a feature request to provide some flag that will allow to not to store dynamic atlas on disk.
Something like ‘ClearDynamicDataOnBuild’.

3 Likes

The font asset “Clear Dynamic Data on Build” option also clears this data when the editor is closed in the latest preview releases which for Unity 2020.3 or newer is version 3.2.0-pre.2.

In terms of not making this data serializable, that is a challenge in the sense that some users do want this data to be persistent in the Editor.

The majority of those users run full dynamic during development since it is more convenient and until they get a better understanding of their glyph coverage. For this use case, they want the data to be serialized and remain persistent until they are ready to convert some of their assets to Static and finalize their fallback structure. Some users also do ship dynamic font assets partially filled which provides some performance benefits as those existing characters and glyphs won’t need to be added to the texture at runtime.

Having said all of the above, I am not opposed to providing another option to not serialize the data at all. I just need to find the time to look into it. In the meantime, if you are not using the latest release, please check if the modified “Clear Dynamic Data on Build” option where the data is flushed when closing the editor might work for you.

This means that before each commit I will have to close the editor, which is even worse than discarding changes in the git:(

6 Likes

@Stephan_B

I am facing the same problem.

Even if I have “ClearDynamicDataOnBuild” enabled when I close the editor, it still seems to be reset optimized for the Asset for the open scene. Ideally, the contents of Atllas are empty except in Play mode.

Is this my misconfiguration?

1 Like

Yeah this makes version control of this asset even more confusing.

2 Likes

Any update or workaround for this? We have integrated TMP into a large legacy project, which is now causing font diffs for every dev that touches the project’s new TMP content. We don’t plan to include atlases in builds (we are already toeing the size limit), which makes these atlases only used in the editor and unnecessary to commit.

Short of writing a script that traverses every scene and prefab to render all the TMP text in the project and then commit it all in one go, I’m not sure what I can do currently to prevent these diffs.

Also, even if we commit all the changes, if the atlas is cleared when the editor closes due to “Clear Dynamic Data on Build”, doesn’t that start the issue all over again? I don’t think that option should do that when the editor closes, or there should be an explicit option for clearing on editor close.

4 Likes

The workaround we’re using is to git update-index --assume-unchanged on the font files. However, this is a gigantic pain in the ass when we do want to commit changes to the font files, just not to the atlases, since the data is all stored within one file.

2 Likes

If we think about it, font atlas texture should never be present on the disc, even if this is a static atlas.
In nutshell, FontAsses is a collection of settings that we want to serialize, save and share(in git for example). While the atlas texture itself is just a generative product of those settings. Atlas texture could (and in my opinion should) be stored in the Library folder, not in the project’s assets.

AND there should be some command for “baking” atlas texture into the actual project file if user is willing to.

But again, storing the atlas texture itself does not make much sense. Especially for dynamic atlases.

So are you the solo dev on the TMPro package? Cuz if it was open source we could make some pull requests to fix common annoyances in the asset everyone love to use.

7 Likes

Very much agree that this (without investigating it much, admittedly) does seem like generated data that shouldn’t exist in version control.

1 Like

Please fix this, its a very annoying issue. If an atlas is fully dynamic please let us choose to not serialize any characters/glyphs/kerning/whatever else.

4 Likes

Is there an update on this issue? We’d like it to be purely dynamic and to never be saved to disk.

1 Like

This is a problem with addressables as it dirties the asset and thus the bundle that the font asset resides in. It means when we push out content updates that the players will have to download the bundle containing our fonts every time again and again for no reason. It is costing us $$$ money serving data. :frowning:

1 Like
The majority of those users run full dynamic during development since it is more convenient and until they get a better understanding of their glyph coverage. For this use case, they want the data to be serialized and remain persistent until they are ready to convert some of their assets to Static and finalize their fallback structure. Some users also do ship dynamic font assets partially filled which provides some performance benefits as those existing characters and glyphs won't need to be added to the texture at runtime.```

While this appears to be a noble goal, this needs to be a feature a user *opts into* not the default setting of TMP-generated fonts. The vast majority of users aren't going to know that TMP Pro can even do this or what is changing session to session. It's a bit bonkers to just have it functioning like this by default.

I'm with the 57 other users who think this is annoying and needs to be changed. I wasted 20 mins of my life trying to figure out why these were constantly changing for my team members.

 @Stephan_B  let's get a response here. It's been too long.
2 Likes

I most certainly see and understand how this can be an issue for many users. This issue was brought up by the team whose project I am currently reviewing in my new role with the consulting team at Unity.

I agree this functionality should be an option to control whether changes to font assets are persistent in the Editor or not. I did investigate this in the past but could not find any clear solution at that time.

Poke @HugoBD-Unity for visibility.

3 Likes

It was glossed over but I’d like to bring up again that this causes the addressable group that the font is located in to be dirty every time as well and requires redownload. An option is to put the dynamic font into its own bundle to minimise download size but still, it’s gross.

Another option seems to be to generate the tmp asset at game startup and add it to the fallback list. Thus the font only exists in memory and there is no disk asset to manage.

However this option requires poking the fallback list on the primary font, which dirties it, so we’re back to square 1.

Also the workflow here is messy, as you have the now manage the life cycle of this in-memory tmp asset and ensure things work correctly during runtime, edit play mode, but also in the editor when working with prefabs outside of play mode.

So anyway, just another +1 for an option on the tmp text asset to have it completely in memory and managed by tmp, no writes to disk, atlas generated on demand every time.

1 Like

Do you have “Clear Dynamic Data on Build” enabled on those dynamic font assets?

In version 3.2.0-pre.x, this option also clears the dynamic data when the editor is closed. This option also resets the atlas texture back to size 0 (1 x 1 in newer releases). As such, dynamic font assets with Clear Dynamic Data on Build enabled should not take any space in builds (they should be kb in size / tiny).

So during an Editor session, dynamic font assets will pollute the repo but after the editor is closed, that should not be the case other than Font Feature data being added which doesn’t get cleared since that is very expensive to repopulate at runtime.

P.S. Still agree that we need a better way to handle this. The above is just additional information.

2 Likes

Dynamic atlas generation is nice, but very annoying for source control when the text is being set dynamically in playmode. Would be very nice if the glyphtable would not be serialized depending on user input. Something like a HideFlag could potentially solve this issue.

As voter 73 of the people who are “annoyed with how dynamic fonts work with source control”, I figured out some kind of workaround.

Put the following code as “OnSaveProcessor.cs” into the Editor folder

using System.Linq;
using UnityEditor;

public class OnSaveProcessor : AssetModificationProcessor {
    public static string[] OnWillSaveAssets(string[] paths) {
        return paths.Where(p => !p.EndsWith("SDF.asset")).ToArray();
    }
}

The idea is to prevent saving the font asset once it was created and configured initially. The coding above is a very basic example that will remove any asset that ends with “SDF” (as font atlasses normally do) from the about-to-save-list.

Of course, you can get more advanced by filtering with a more specific name or maybe try to get and validate the assets properties by its file path. But you get the general approach :slight_smile:

4 Likes

I want to understand why the ScaleRaioA saved property from debug settings for the material in the font asset is changed. What changed that? Who changed that? Why did that happen? There were no new chars, no new generation of the font, so what is the reason ?

Thanks for the push in the right direction @WongKit ! I just made this script to auto-clear dynamic font data before any gets saved. Seems to fully fix the issue for me.

// TextMeshPro dynamic font assets have a very annoying habit of saving their dynamically generated binary data in the
// same text file as their configuration data. This causes massive headaches for version control.
//
// This script addresses the above issue. It runs whenever any assets in the project are about to be saved. If any of
// those assets are a TMP dynamic font asset, they will have their dynamically generated data cleared before they are
// saved, which prevents that data from ever polluting the version control.
//
// For more information, see this thread: https://discussions.unity.com/t/868941


#if UNITY_EDITOR

using TMPro;
using UnityEditor;

internal class DynamicFontAssetAutoClear : AssetModificationProcessor
{
    static string[] OnWillSaveAssets(string[] paths)
    {
        foreach (string path in paths)
        {
            if (AssetDatabase.LoadAssetAtPath(path, typeof(TMP_FontAsset)) is TMP_FontAsset fontAsset)
            {
                if (fontAsset.atlasPopulationMode == AtlasPopulationMode.Dynamic)
                {
                    //Debug.Log("Clearing font asset data at " + path);
                    fontAsset.ClearFontAssetData(setAtlasSizeToZero: true);
                }
            }
        }

        return paths;
    }
}

#endif
17 Likes