Today I ran the automated performance test of my game with Unity 2018.1.0b6 and present the results below, just like I did before, during older beta cycles:
[EDIT: 2018.2 Performance Overview can be found here .]
2018.1.0b6 is the fastest Unity 201x.x version with my project yet. Unity 2018.1.0b6 performance is slightly faster (0.5ms in the best case) than 2017.3.0b6 for me.
How does the performance test work?
The (first-person) game I’m working on features an automated performance test. It works like this:
Camera is placed at a defined position in the scene
Camera makes a 360 degree rotation around the Y-axis within 20 seconds (slowly rotating around y)
Every second the test captures the average CPU frame time of the last second
The game runs just like it would normally do, but the camera/player does not move and the AI in unable to see the player.
It measures the “base-line” of the game so to speak. If an actual gamer plays the game, more things are going to happen, which is missing in the test, such as visual and sound effects when firing weapons, additional AI to hunt the player and more navigation pathing.
I run this test in a Standalone Windows 64bit Player. The following Physics settings are applied:
Physics.autoSyncTransforms=false
Physics.autoSimulate=true
Physics2D.autoSyncTransforms=false
Physics2D.autoSimulate=false
The y-axis represents the average CPU time in milliseconds, how long “one frame took”. The x-axis represents the sample points while the camera makes its 360° rotation. Each test runs for 20 seconds, one sample every second, thus 20 samples for each test.
Thanks for the additional benchmark, it actually surprised me a little as I didn’t expect overall better perf from IL2CPP. The figures I’ve seen from other users benches has often given worse results for IL2CPP (which is understandable as it can create it’s own overhead).
If IL2CPP is the same performance, that will happen sometimes as you can’t really be optimising basic logic that well, however if it’s slower that should not ever happen IMHO and it’s worth filing a bug.
I admit that I haven’t benchmarked myself but I still think there are cases where IL2CPP will just be slower due to the extra work it can create. Of course when it works straight forwards, it’ll be faster.
In this post: Job System not as fast as mine, why not?@TBbadmofo benchmarked IL2CPP to run his task in 250ms where mono ran it in 120ms. I not sure what “Non Development” actually means in this context, you’d expect him to mean release build. Also worth noting that they might have not optimized c# jobs for IL2CPP much since they plan them to be compiled with Burst eventually.
Here’s also article about possible IL2CPP overhead: JacksonDunstan.com | Three Surprises I Encountered While Reading IL2CPP Output. He didn’t actually benchmark the difference there so it’s not fair to say IL2CPP would have been slower but for sure generated code will occasionally make lots of steps that one could bypass on manual design.
I’ve seen other occasions where IL2CPP hasn’t really sped things up on mobile platforms but again, I don’t have any concrete evidence myself, just what I’ve read from other peoples experiences. I’m mainly interested on IL2CPP on Windows Desktop and since it’s brand new feat in Unity now, there’s not much perf info around yet
@rz_0lento some people do not understand that you can only benchmark IL2CPP, ECS, Jobs with standalone. This is because there are lots of extra checks and tests during editor, so people are naturally seeing slower results in editor.
This guy is doing a great job dissecting the Il2Cpp generated code. I wonder if his findings somehow find their way back into Unity. It’s just too precious information to get lost, since there are probably a few quirks that shouldn’t be too hard for UT to fix.
We are aware of the issues raised. We’ve been focused on other things recently, but I expect we will free up soon for some performance tweaks to generated IL2CPP code.
Release build (stand alone) is what I meant. I called it non-development build because I also unchecked the development build setting. Despite what @hippocoder said, I also complained about my code running faster in the editor than a mono release build; though, in IL2CPP it was better. I figured the editor was using mono and that it would have more overhead, so why does it run slower in a release?
If things run slower in release please report the bug and drop the case number. If IL2CPP is slower than mono then clearly it’s purpose is broken there.
Here is a comparison of 2018.1.0b9 between Mono and Il2Cpp (release mode).
