How do I create a modding API for my Unity game?

I want to create a class library / API in Visual Studio for my game, that modders can download and play with. but I don’t know how and where to start… Let’s say that I have a specific function in my main game to, for example, spawn an enemy called:

//keep in mind, this script is in my game.

using UnityEngine;

public class EnemyScript : MonoBehaviour
{
   public void SpawnEnemy()
   {
      //code to spawn an enemy...
   }
}

How can modders call this function, when I want to create my API? How do I get the direct reference to this function for my API, when I don’t have anything imported? I hope that’s understandable and thanks in advance.

Unity doesn’t have built-in capability like this. You’re welcome to roll your own or integrate one that someone else is using, but that lives entirely outside of core Unity function.

Before you go down this massively-deep rabbit hole (you’re essentially reimplementing a subset of Unity on top of Unity!), be sure to identify what you want, such as:

  • ability to tweak simple game parameters (damage, etc.)

  • ability to import assets (textures/models)

  • ability to script different behaviours (do you need to perhaps use something like LUA?)

etc.

I know that Unity doesn’t support this out of the box and I’m so ready to code this by myself, but I don’t know how and where to approach this. I would like to give the modders the ability to completely change the game if they want to (like adding new enemies, NPCs, worlds, with custom textures, attacks, parameters, etc…), but only in C#, so no other script behaviours like LUA or Java.

You would need to find either a C# compiler to send off to people, or have them compile with Visual Studio and produce a DLL, or else use some other solution, such as perhaps interpreting C#.

You are going to find a LOT more useful stuff on the modding boards if you find a few Unity-made games that have good modding support and ask their makers how they did it, or figure it out by modding them yourself.

You are probably not.
But like all of us you need to get burn yourself to understand.

I know that I have to provide a .dll to modders so they can start modding. My question is, how can I expose my code to my API (that I will compile to a .dll) so that my API can interact with my code? I know that the modders have to compile their mod too and my game would load their .dll mod so it can interact with my game.

That is a good suggestion, I will do that too.

Let’s start by reviewing the basics of how C# code is run in your game.

People edit C#. People don’t edit DLLs.

Compilers turn that code into assemblies and then into DLLs.

Unity ships all your scripts out as a couple of DLLs.

Except on Android and iOS where Unity transmulches your DLLs back into C/C++ code and compiles it, then digitally signs it, causing NO other code to be runnable unless it too gets signed by Apple / Google.

One common modding approach is to provide a stub DLL that people replace. Almost nobody here on these boards would care or know about how to do this. Why? Because we’re using Unity to make games, not to mod other random games.

Otherwise modders must decompile your DLL, add their code, then recompile it.

Or otherwise they must shim in their own compiled code.

So what you are saying is, that a C# class library that I will compile to a .dll to create the modding API would be the wrong approach, or did I misunderstood anything?

Alright, I’ve tested a bit and I (kinda) finally found my answer. I’ve found out, that in MyGame/MyGame_Data/Managed are all .dll’s that are used in the game. (thanks to this, I didn’t know what you meant at first, but I understand it now)

So my question now is: How should I approach my API now, and how can I provide the API to the modders, without having them searching for the right .dll in the /MyGame_Data/Managed folder?

Thanks in advance and happy new year!

If anybody’s looking for this again, AppDomain.CurrentDomain.LoadFile (Not sure at the exact method name) is the answer, modding APIs typically have to inject their own dll into the game’s code. One of the methods that is commonly used is Dll Hijacking, basically those are dlls that don’t actually exist as files and are hidden in the game directory. Those dlls exist in pretty much every process and the special thing about them is that they are pretty empty, meaning that you can create a dll in the game directory with the same name of one of those, and the game will load them and your code will already be within the game’s process.

If you are not creating a mod for your own game you already have access to your own program’s code, and your code is located in your app’s domain, meaning that you don’t have to use a method like this. So loading mods is really basic, you just have to create a Mods directory on startup and iterate through all of the dlls in this directory. Once you find all of the mod paths you have to use the load method provided by app domain as I showed earlier, to load that mod path. And now the mod dll is a part of your game’s process. Now the only thing remaining is to provide some way for modders to run their code on initialize: this can be patches, etc… You do that by searching through the mod assembly’s types and most modding APIs use an interface that has an OnLoad method, so you search for the first type that implements your modding interface. and just run the on load method using reflection.

Now if modders want to create mods for your game, they just have to create a class library project with a type that extends your modding interface, do whatever they want on loading, then build their project and put it in the Mods directory. And your mod loader is done! Now all that’s left is for their project to reference your Assembly-CSharp which contains your modding API’s code too, and they can use all of your game’s functions.
If you also want to let them create hooks for your code you can use a library such as Harmony and put the dll in your game’s directory.

That’s what they do at Nexus Mods: Unity Mod Manager at Modding Tools - Nexus Mods

Basically they are decompiling your source code and injecting their own methods I think. If you compiled your project with IL2CPP then they are out of luck.

In other words, as long as you don’t compile with IL2CPP you have nothing to do they’ll find their way through your code.

By the way I just discovered this: