Unity 2020.2 Performance Overview

Hello everybody, here we go again with another performance overview update. I know... I know... I'm early this time :)

Today I ran the performance tests of my game with Unity 2020.2.0a19 and 2018.1.9f2. I present the results below, just like I did before, during older beta cycles.
Older Reports

[spoiler]

[/spoiler]

According to my test, 2020.2.0a19 is slower than 2018.1, which is still the fastest Unity version with my project. I profiled all builds listed in the graphs below during the same session.

You can run the test using the project attached to the following bug-report:
(Case 1108597) Project to find performance regressions and bugs

How does the test work?

[spoiler]
The indoor first-person shooter I'm working on features an automated performance test. It works like this:

  • Camera is placed at a defined position in the scene
  • Camera makes a 360 degree rotation around the Y-axis within 20 seconds (slowly rotating around y)

  • Every second the test captures the average CPU frame time of the last second

The game runs just like it would normally do, but the camera/player does not move and the enemy AI in unable to see the player.

The performance test measures the "base-line" of the game basically (rendering, shadows, realtime lighting, physics, audio, NavMesh, UI, game code). If an actual gamer plays the game, more things are going to happen, which is missing in the test, such as visual and sound effects when firing weapons, additional AI to hunt the player and more navigation pathing.

I run this test in a Standalone Windows 64bit (Mono) Player:

  • 2018 = .NET3.5
  • 2020 = .NET4.x

I've also enabled "Incremental GC" on 2020.

The following Physics settings are applied:

  • Physics.autoSyncTransforms=false
  • Physics.autoSimulate=true

  • Physics2D.autoSyncTransforms=false

  • Physics2D.autoSimulate=false

The resolution is set to fullscreen 320x240 (D3D11) to make sure the bottleneck isn't GPU. The game uses the built-in deferred renderer.

[/spoiler]

The y-axis represents the average CPU time in milliseconds, how long "one frame took". The x-axis represents the sample points while the camera makes its 360° rotation. Each test runs for 20 seconds, one sample every second, thus 20 samples for each test. Fewer milliseconds (vertical axis) indicate better performance.

I profile each build three times and the graphs below display the minimum sample value (best performance) of these three profiling runs. I profile multiple times, because it improves the overall robustness of the test. In the past I only profiled a build once, but it caused that things like OS activity sometimes affected the timings and the overall graph was a bit more spiky.

6209489--682205--scene_4_3.png

6209489--682208--scene_5_4.png

6209489--682211--scene_6_8.png

These issues are probably not reproducible if you're running the game on high-end hardware. Please use a rig similar to the hardware I used to submit the bug-report.

I've also recently profiled various LTS versions for comparison. If you've missing the post, you find it here: https://discussions.unity.com/t/780489 page-2#post-6103425

11 Likes

@Peter77 do you have results for 2018 on .NET 4 runtime too ?
Surely there are more things at play, still surprised that 3.5 is noticeably - all things considered - faster


Yes, this was asked already, thus I have the numbers :)
https://discussions.unity.com/t/780489/36

2 Likes

[quote=“Peter77”, post:3, topic: 804740]
Yes, this was asked already, thus I have the numbers :slight_smile:
https://discussions.unity.com/t/780489/36
[/quote]
ok seems like runtime doesn’t matter - which I found a little bit surprising tbh, that’s why it’s good to test it seems ) - I would, however, conduct all testing with the same one regardless - just to eliminate the variable, and given that new runtime is default on 2018, with 3.5 deprecated there already
But in any case I’m sure there are/were valid reasons for this, don’t let internets distract you from doing these ]

1 Like

Maybe I'm missing something, but why are results different for 2020.2 performance overview, and the one for 2020.1? For example, 2018.1 is starting at 3ms on Scene4_3 (current overview) but at 3.5ms on your older overview?

1 Like

I explained it in this post:
https://discussions.unity.com/t/780489/16

2 Likes

[quote=“Peter77”, post:6, topic: 804740]
I explained it in this post:
https://discussions.unity.com/t/780489/16
[/quote]

Makes sense, thanks!
Nice to see that 2020.2 is already faster than 2020.1 while still being an alpha.

4 Likes

if you ever add editor operations we'll get a sense of editor regression too and see
that
[editor refresh (1s)]
this
[editor refresh (2s)]
version
[editor refresh (5s)]
is...
so
zzzzz...
slow

[quote=“laurentlavigne”, post:8, topic: 804740]
if you ever add editor operations we’ll get a sense of editor regression too
[/quote]
I could repeat all the tests I did during the 2019.3 beta cycle:

(Case 1171423) 2019.3: Asset import significantly slower than in Unity 4.6
https://forum.unity.com/threads/cas…ignificantly-slower-than-in-unity-4-6.714986/

(Case 1171344) 2019.3: Focusing editor significantly slower than in Unity 4.6
https://forum.unity.com/threads/cas…-for-2-secs-before-it-gets-repsonsive.714791/

(Case 1161371) 2019.3: Compiling empty project significantly slower than in Unity 4.6
https://forum.unity.com/threads/cas…ty-project-takes-significantly-longer.691996/

(Case 1161373) 2019.3: Enter playmode time significantly slower than in Unity 4.6
https://forum.unity.com/threads/cas…playmode-time-significantly-increased.692005/

(Case 1161272) 2019.3: Time to open editor significantly slower than in Unity 4.6
https://forum.unity.com/threads/cas…o-open-editor-significantly-increased.691951/

(Case 1158368) 2019.3: Hierarchy window performance significantly slower than in Unity 4.6
https://discussions.unity.com/t/744666

The only issue is that it takes so much time and I’m so not interested in it at the moment :frowning:


nm. One day maybe Unity will have the courage to publicize their internal benchmark.

[quote=“laurentlavigne”, post:10, topic: 804740]
nm. One day maybe Unity will have the courage to publicize their internal benchmark.
[/quote]
Ha. I remember a runtime benchmark thread being locked because apparently, Unity having their internal benchmarks is a good reason to silence user benchmarks.

EDIT: [Here it is.]( https://discussions.unity.com/t/751653 page-4#post-5607796)

2 Likes

[quote=“Peter77”, post:3, topic: 804740]
Yes, this was asked already, thus I have the numbers :slight_smile:
https://discussions.unity.com/t/780489/36
[/quote]
Nice Work.

[quote=“Ramobo”, post:11, topic: 804740]
Ha. I remember a runtime benchmark thread being locked because apparently, Unity having their internal benchmarks is a good reason to silence user benchmarks.

EDIT: [Here it is.]( https://discussions.unity.com/t/751653 page-4#post-5607796)
[/quote]
I think they locked it because it had evolved into users trying to help this one unity dev narrow down a problem - which he had - so there was no reason to keep it open. Seems fair although arbitrary given the number of other threads that go off topic and remain open.

But to go back to my point, which only had us as beneficiaries, I just thought of another benefit, for Unity: no one at Unity has to be the bad guy who tells a team “hey guys you’re down 10% get your acts together” - we get to be the bad guys.
And some of us really enjoy it :smile:

2 Likes

It's graph time again! :)

Today I re-ran the performance test with 2020.2.0b4 against 2018.1.9f1 and 2020.2.0a19. I profiled all builds listed in the graphs below during the same session.

So far no performance improvement visible, perhaps even a tiny performance drop compared to a19.

6361146--707808--scene_4_3.png

6361146--707811--scene_5_4.png

6361146--707814--scene_6_8.png

PS: Don't miss to check out my new tool on github ;)
https://github.com/pschraut/UnityMiniEditorIterationProfiler

7 Likes

Thanks for graphs @Peter77 :)

I think I have missed something :( Can you elaborate you compare 2018.1.9 BuiltIn Renderer to 2020.2 BuiltIn Renderer
or 2018.1.9 X Renderer to 2020.2 URP 10 ?

In case you compare to 2020.2 BuiltIn can you please add line for last URP?

1 Like


The project uses Unity's built-in deferred renderer in all tests.


That's unfortunately not as simple.

The project uses deferred rendering, which isn't available in URP at the time of writing. The last time I checked URP forward, it caused all sorts of light-flicker issues in this project. I'd also need to recreate all the custom shaders in URP.

When URP deferred is available, I'm looking into how to add this to the test. Until then, only built-in deferred renderer.

4 Likes

Just a few days after b4, we have b5 already. Lets see if this version runs as fast as they roll out new versions! :)

Today I re-ran the performance test with 2020.2.0b5 against 2018.1.9f1 and 2020.2.0a19. I profiled all builds listed in the graphs below during the same session. If you want to know what the test is actually doing, what rendering tech it uses, please see the first post .

So far no performance improvement visible in b5, perhaps even a little performance drop compared to a19.

6374007--709572--scene_4_3.png

6374007--709575--scene_5_4.png

6374007--709578--scene_6_8.png

6 Likes

Looking for the Performance Overview for Unity 2021.1? It's here: https://discussions.unity.com/t/813931

Hey ! 2020 LTS is now out, are you planning on doing a performance overview for this ? Thanks a lot for your work btw :)


LTS is just bug fixes, so I don't think that it's worth doing within the first few versions. But who knows, maybe they fucked something up while incrementing the version number and now everything is slower.