C#'s general performance in Unity is orders of magnitude slower than JVM-based language

Hi all,

I have recently been trying to build a custom rendering layer in Unity using C#. The rendering layer’s design is similar to one that I previously made in Kotlin (which is a JVM-based language that is similar to Java). However, in Unity, I noticed that I have been getting subpar performance compared to my original implementation in Kotlin. After profiling, I suspect that the language itself may somehow be slower. Therefore, I made a very short benchmark in both Kotlin and C# to measure the performance of basic operations:

The kotlin part is as follows. Note that Matrix4 and Vector3 are libGDX (a Java/Kotlin’s game library) classes, and they are nothing more than just containers of data. The mul function multiplies the matrix with the vector, and stores the result in-place back into the vector.

fun benchmark(a: Matrix4, b: List<Vector3>) {
    var i = 0;
    while (i < 100000) {
        b[i].mul(a);
        ++i;
    }
}

var a = Matrix4(floatArrayOf(1f, 2f, 3f, 4f, 3f, 2f, 1f, 2f, 3f, 4f, 3f, 2f, 1f, 2f, 3f, 4f))
var b = List<Vector3>();
for (i in 0..100000) {
    b.add(Vector3(3f, 2f, 1f));
}
// warmup JIT
for (i in 0..9) {
    benchmark(a, b)
}
var t: Double = 0.0;
for (i in 0..9) {
    t += measureNanoTime {
        benchmark(a, b)
    }.toDouble()
}
println(t / 10.0 / 1000000.0) // milliseconds

The Unity C# part is as follows. Note that M4 and V3 are helper classes created to match what libGDX had.

private void Benchmark(M4 a, List<V3> b)
{
    var i = 0;
    while (i < 100000)
    {
        b[i].mul(a);
        ++i;
    }
}

var a = new M4(1f, 2f, 3f, 4f, 3f, 2f, 1f, 2f, 3f, 4f, 3f, 2f, 1f, 2f, 3f, 4f);
var b = new List<V3>();
for (int i = 0; i < 100000; ++i)
{
    b.Add(new V3(3, 2, 1));
}
// warmup JIT
for (int i = 0; i < 10; ++i)
{
    Benchmark(a, b);
}
var t = 0.0;
for (int i = 0; i < 10; ++i)
{
    var s = (double) nanoTime();
    Benchmark(a, b);
    var e = (double) nanoTime();
    t += e - s;
}
Debug.Log(t / 10.0 / 1000000.0); // milliseconds

The implementation of mul is made to match libGDX’s exact implementation (libgdx/gdx/src/com/badlogic/gdx/math/Vector3.java at 89b4c86cf91d8d8af8ca51d015ec0365d9000d59 · libgdx/libgdx · GitHub).

The device is a mid-2015 MacBook Pro. Unity version is 2020.3.0f1, building to OSX standalone with Mono backend, not a development build.

The results are as follows:

Kotlin: 0.3658762ms
Unity C#: 1.74067ms (almost 4 times slower)

On Unity C#, If I change M4 and V3 to be struct instead of class, the results are even slower: 2.51ms (almost 6 times slower).

Can anyone spot anything I did wrong? I did not expect C# in Unity to be this much slower than Kotlin/Java. Any help would be appreciated.

Do I understand right that you are comparing Kotlin in console-like environment and C# with/in opened Unity? Isn’t that obvious that code running inside engine process will be slower than plain console application?
Can’t you compare plain C# with plain Kotlin? At least you would understand what causes this big difference.

There’s almost no chance that this has anything to do with the language and rather almost certainly has to do with either the specific code and classes you’re running (most of which you haven’t shared here) or your benchmarking method (which is both unorthodox from techniques I’ve seen and, again, not shown here).

What are M4 and V3? What is the implementation of mul()? What is nanoTime()? The fine details of any or all of these could contribute a tiny amount of time to each iteration of the loop that could add up to the extra time you’re seeing. We have basically nothing to work with here.

Here are the missing parts:

private static long nanoTime()
{
    long nano = 10000L * Stopwatch.GetTimestamp();
    nano /= TimeSpan.TicksPerMillisecond;
    nano *= 100L;
    return nano;
}

private class V3
{
    public float x;
    public float y;
    public float z;

    public V3(float x, float y, float z)
    {
        this.x = x;
        this.y = y;
        this.z = z;
    }

    public V3 mul (M4 matrix) {
        return this.set(x * matrix.M00 + y * matrix.M01 + z * matrix.M02 + matrix.M03, x
            * matrix.M10 + y * matrix.M11 + z * matrix.M12 + matrix.M13, x * matrix.M20 + y
            * matrix.M21 + z * matrix.M22 + matrix.M23);
    }

    public V3 set(float x, float y, float z)
    {
        this.x = x;
        this.y = y;
        this.z = z;
        return this;
    }
}

private class M4
{
    public float M00;
    public float M10;
    public float M20;
    public float M30;
    public float M01;
    public float M11;
    public float M21;
    public float M31;
    public float M02;
    public float M12;
    public float M22;
    public float M32;
    public float M03;
    public float M13;
    public float M23;
    public float M33;

    public M4(float m00, float m10, float m20, float m30, float m01,
        float m11, float m21, float m31, float m02, float m12,
        float m22, float m32, float m03, float m13, float m23,
        float m33)
    {
        M00 = m00;
        M10 = m10;
        M20 = m20;
        M30 = m30;
        M01 = m01;
        M11 = m11;
        M21 = m21;
        M31 = m31;
        M02 = m02;
        M12 = m12;
        M22 = m22;
        M32 = m32;
        M03 = m03;
        M13 = m13;
        M23 = m23;
        M33 = m33;
    }
}

For kotlin, see https://github.com/libgdx/libgdx/blob/master/gdx/src/com/badlogic/gdx/math/Vector3.java#L353.

As a side note, the inefficiencies probably do not come from these missing parts, because when I tried it with unity’s Matrix4x4 and Vector4, the results are even slower.

I tested the Kotlin part within an existing game I made previously, in Kotlin using libGDX. I wouldn’t consider that “console-like environment”, but I also wouldn’t say that a language running inside an engine environment (especially in a standalone build, not the editor) have to be slower than running in console. For example, plenty of C++ engines embed lua as scripting language, and the embedded lua’s raw speed can be easily made just as fast as pure console luaJIT.

Mobile target performance is almost always pixel fill bound. This is a point lost on the rabid DOTS/ECS hordes a few years back as everybody bum-rushed that technology and found it made zero difference for mobile.

As always, if you have a performance bottleneck, start with the Profiler to see if it’s even something you can do anything about, at least practically and economically.

Some things:

I would only use the Unity-native versions rather than your homebrew. Of which you can use the ones you mentioned, or use Unity.Mathematics.

If you do use Unity.Mathmatics, you can also take advantage of the Burst compiler to generate highly performant native code (SIMD and all).

Make sure you do a client build and test that build, not in the editor, when looking at performance at that level.

I’m targeting MacOS standalone, which is what’s used in the benchmark. I also used the deep profiler in Unity, and found that I was indeed CPU bound. I even commented out all graphics calls, and still get similar performance.

Right. I understand that there are certain ways to optimize Unity’s C# code further. I am only worried that C# in Unity is slower than Kotlin by this much even when the code is equivalent. If this is unfortunately true (i.e. the benchmark is actually correct with no easy remedy), then I have to keep this fact in mind in future optimizations.

I don’t think you are looking at this right. C# is a different beast. You have to use it right and understand its nature. Like anything else. For example, if running as IL, there will be a JIT phase which will throw off your benchmark as presented, but not likely have a real-world impact for a game. If you compile with IL2CPP, then the IL and thus JIT is eliminated. Burst is similar, but tightens up these inner loops to some pretty optimal CPU performance for what it does.

Do you need to worry about any of this in practice? It depends on what you expect to be doing. Your algorithms and data structure usage will matter way more than anything C# related.

For reference, we’re building this in Unity: https://www.galahad3093.com/

Fair amount of vector math going on. Plenty of performance challenges in a game like this, vector math is not among them.

Quote of the year.

Very nice!

Glancing at screencaps, I’ll guess that your primary challenges are getting the lighting to look good (huge dynamic ranges of brightness), and just general sheer volume of assets and your asset flow pipeline, sharing rigs vs custom-animating each, etc.

As already mentioned, I would check if you get any performance difference with an IL2CPP build instead of a Mono build.

I tried this in a Visual Studio console app and got pretty much exactly the same results (1.7ms). Adding

[MethodImpl(MethodImplOptions.AggressiveInlining)]

above the V3 mul function dropped this to under 0.3ms

Depending on how many C# calls you have in a frame, deep profiling can make sure that you’re CPU bound, as it introduces an instrumentation sample to the begin and end of every function call (that includes properties), so it bloats up your scripting timings based on how many there are.

I assume you had Incremental GC on, which introduces write barriers to be able to do it’s job, which can, depending on how script heavy some computation is, reduce performance. So try turning that off.

Also, as mentioned before, Mono is unlikely to be your scripting backend when you ship the game, so I’d measure against IL2CPP.