Source Generators? How to use it and how to go about to do this?

Unity Engine
,
1

How do I get started using those? I’ve tried installing CodeAnalysis through NuGet and it uninstalled on it’s own when using JetBrains Rider editor.

But more specifically on how to use code generators (or if a similar solution exists):
I am about to make a XML database that works on burst (configure an item, like an axe in XML, set it’s damage numbers and whatnot, then access it in Unity inside burst context, etc)

Here is the draft of what I am thinking of, most of the magic will have to happen in the generation of the contents of the variable BurstDatabase.AllOfTheData, everything else is not exactly noteworthy besides the fact that it will have A LOT of boilerplate code that I want to do away with source generators.
In the draft, surrounded by big comment blocks, is the code I’d like help figuring out how to generate automatically

public interface IDatabaseData { //declares a type as readable from XML
    public string XMLName { get; }
    public string DatabaseName { get; }//requires to ensure unique references, repeats override data to enable mods to change stuff
}

public partial struct MyTestStruct2 : IDatabaseData {
    public string XMLName => "MyStruct2"; //xml tags like <MyStruct2><Data1>0</Data1></MyStruct2> are read as this object

    //source generated?
    /////////////////////////////////////////////////////////////////////////////////////////////
    public uint DatabaseID;
    public string DatabaseName => BurstDatabase.Managed.GetName(DatabaseID);
    /////////////////////////////////////////////////////////////////////////////////////////////

    public uint Data1;
}

//[IDatabaseData("MyStruct")] this could also work instead of a interface, I guess. Maybe preferable?
public partial struct MyTestStruct : IDatabaseData {
    public string XMLName => "MyStruct";
   
    //source generated?
    /////////////////////////////////////////////////////////////////////////////////////////////
    public uint DatabaseID;
    public string DatabaseName => BurstDatabase.Managed.GetName(DatabaseID);
    /////////////////////////////////////////////////////////////////////////////////////////////

    //Accessible in burst context
    public uint Data1;
    public uint Data2;
    public uint Data3;
    public uint Data4;
    public uint Data5;

    //How to make these two be source generated?
    /////////////////////////////////////////////////////////////////////////////////////////////
    public uint _MyTestStruct2;
    public MyTestStruct2 MyTestStruct2 => BurstDatabase.Instance.LoadResource<MyTestStruct2>(_MyTestStruct2);
    /////////////////////////////////////////////////////////////////////////////////////////////

    //Can only be accessed in managed context
    /////////////////////////////////////////////////////////////////////////////////////////////
    public uint _MySpriteRef;
    public Sprite MySprite => BurstDatabase.Managed.LoadResource<Sprite>(_MySpriteRef);
    /////////////////////////////////////////////////////////////////////////////////////////////
}

public struct BurstDatabase {
    public static BurstDatabase Instance;
    public static BurstDatabaseManaged Managed;

    public NativeArray<byte> AllOfTheData; //built using a ton of managed and reflection, reading XML files and compiling it,

    //then serialized to disk until the database changes (mod install/uninstall)
    public NativeHashMap<uint, int> IndexForReference; //same as above

    public T LoadResource<T>(uint ResourceRef) where T : unmanaged {
        int TSize = UnsafeUtility.SizeOf<T>();
        return AllOfTheData.GetSubArray(IndexForReference[ResourceRef], TSize).Reinterpret<T>(TSize)[0];
    }
}

public class BurstDatabaseManaged {
    public Dictionary<uint, string> References; //loaded from file generated on the same method as above
    public Dictionary<uint, string> Names; //loaded from file generated on the same method as above

    public T LoadResource<T>(uint ResourceRef) where T : Object {
        return Resources.Load<T>(References[ResourceRef]);
    }

    public string GetName(uint NameRef) {
        return Names[NameRef];
    }
}
2

What do you want to generate?
Early last year, unaware of source generators, I wrote my own little library to generate C# scripts. Of course I did not aim for it to be full fledged, but it does support all access modifiers, readonly, static, class, struct, and so on - pretty complete for general Unity coding uses overall.

Then, I was surprised to eventually learn about source generators and understandably
this needs to support every nook and cranny of C# but still … the verbosity of that and the technicality of its terms is just insane. I could generate scripts with at most 20% of that code and it was totally readable code too.

So … maybe you are better off just to StringBuilder your way into generating scripts. :wink:

3

Hell, there’s one hiding inside the InputSystem code-base, called CodeWriter. Can’t remember if it’s internal, but you can still just view the script and write something very similar.

4

Mine is called ScriptBuilder, if I remember correctly it may even be a subclass of StringBuilder. :wink:

5 
[DatabaseData("MyStruct")]
[DatabaseAuto(typeof(Sprite), "MySprite")]
[DatabaseAuto(typeof(MyTestStruct2), "MyTestStructGetter")]
public partial struct MyTestStruct {
    public uint Data1;
    public uint Data2;
    public uint Data3;
    public uint Data4;
    public uint Data5;
}

Generated this:

public partial struct MyTestStruct {
    public uint DatabaseID;
    public string DatabaseName => BurstDatabase.Managed.GetName(DatabaseID);

    public uint _MySprite;
    public UnityEngine.Sprite MySprite => BurstDatabase.Managed.LoadResource<UnityEngine.Sprite>(_MySprite);

    public uint _MyTestStructGetter;
    public MyTestStruct2 MyTestStructGetter => BurstDatabase.Instance.LoadResource<MyTestStruct2>(_MyTestStructGetter);
}

This works great with one exception. The generated code is always 1 compilation behind. It seems UnityEditor.AssemblyReloadEvents.beforeAssemblyReload is not a good place to add new sources.
The current code:

public class BurstDatabaseLoader : MonoBehaviour {

    private static readonly string C = @"
public partial struct BBBBBBBB {";

    private static readonly string P = @"
    public uint _AAAAAAAA;
    public TTTTTTTT AAAAAAAA => BurstDatabase.MMMMMMMM.LoadResource<TTTTTTTT>(_AAAAAAAA);
";

    private static readonly string D = @"
    public uint DatabaseID;
    public string DatabaseName => BurstDatabase.Managed.GetName(DatabaseID);
";

    static BurstDatabaseLoader() {
        UnityEditor.AssemblyReloadEvents.beforeAssemblyReload += CompileCode;
    }
    private static void CompileCode() {
        StringBuilder StringBuilder = new StringBuilder();
        foreach (Type type in GetTypesWithHelpAttribute()) {
            StringBuilder.Append(C.Replace("BBBBBBBB", type.Name));
            StringBuilder.Append(D);
            foreach (DatabaseAutoAttribute DatabaseAutoAttribute in type.GetCustomAttributes<DatabaseAutoAttribute>()) {
                Type T = DatabaseAutoAttribute.Type;
                StringBuilder.Append(P
                    .Replace("AAAAAAAA", DatabaseAutoAttribute.Name)
                    .Replace("TTTTTTTT", T.FullName)
                    .Replace("MMMMMMMM", T.IsStruct() ? "Instance" : "Managed")
                );
            }
            StringBuilder.Append("}\n");
        }
        File.WriteAllText("Assets/_Generated.cs", StringBuilder.ToString());
        AssetDatabase.Refresh();
    }


    static IEnumerable<Type> GetTypesWithHelpAttribute() {
        Assembly assembly = Assembly.GetExecutingAssembly();
        Type attributeType = typeof(DatabaseData);
        foreach (Type type in assembly.GetTypes()) {
            if (type.GetCustomAttributes(attributeType, true).Length > 0) {
                yield return type;
            }
        }
    }