Performance Overview

Unity Engine
,
1

Today I ran the performance test of my game with Unity 2019.1.0b4 and present the results below, just like I did before, during older beta cycles:

According to my test, 2019.1.0b4 is slower than 2018 on average.

You can run the test using the project attached to the following bug-report:
(Case 1108597) Project to find performance regressions and bugs

How does the test work?

The (first-person) game I’m working on features an automated performance test. It works like this:

  • Camera is placed at a defined position in the scene

  • Camera makes a 360 degree rotation around the Y-axis within 20 seconds (slowly rotating around y)

  • Every second the test captures the average CPU frame time of the last second

The game runs just like it would normally do, but the camera/player does not move and the AI in unable to see the player.

It measures the “base-line” of the game so to speak. If an actual gamer plays the game, more things are going to happen, which is missing in the test, such as visual and sound effects when firing weapons, additional AI to hunt the player and more navigation pathing.

I run this test in a Standalone Windows 64bit (Mono) Player. The following Physics settings are applied:

  • Physics.autoSyncTransforms=false

  • Physics.autoSimulate=true

  • Physics2D.autoSyncTransforms=false

  • Physics2D.autoSimulate=false

The resolution is set to 320x240 to make sure the bottleneck isn’t GPU.

The y-axis represents the average CPU time in milliseconds, how long “one frame took”. The x-axis represents the sample points while the camera makes its 360° rotation. Each test runs for 20 seconds, one sample every second, thus 20 samples for each test. Fewer milliseconds (vertical axis) indicate better performance.

4272520--382510--4_3.png

4272520--382513--5_4.png

4272520--382516--6_8.png

Playing the game inside the editor, is also significantly slower than in earlier Unity versions, like sometimes dropping to 20fps which I didn’t observe in Unity 2017.

These issues are probably not reproducible if you’re running the game on high-end hardware. Please use a rig similar to the hardware I used to submit the bug-report.

10 Likes
2

I’m thinking, maybe I should repeat the test with every Unity version.

This would make sure the performance regression is not related to some Windows and/or graphics card driver update that was installed in the meantime. I mean there were always several months between I ran those tests.

2 Likes
3

About 4 hours later, I’m done. I’ve updated the first post with the new numbers. The results look pretty similar to the ones I measured earlier though.

2019.1.0b4 is slower than Unity 2018. Unity 2018.1 continues to be the fastest Unity version with the provided test project.

7 Likes
4

Thank you for your hard work!

5 Likes
5

Thanks for noticing :slight_smile:

1 Like
6

It’s great to know that the highest delta is around 0.5ms. By this knowledge, if you touched 16.16ms per frame you are at risk of not getting 60FPS from just changing Unity version. (probably will be magnified by a certain things in the game that’s not in the test project though)

1 Like
7

If you target VR, it’s about 5% of your total frame time. :hushed:

1 Like
8

Is there a specific reason why Unity is getting consistently slower over time when the features you are using remain the same?

I guess I ask because someone will at some point.

8 Likes
9

Probably all the added features, aka everything running backend

10

thank you for making these charts, can you do tests with vulkan

1 Like
11

Unfortunately I can’t, I only have a Windows PC.

12

The audio in newer version is definitely glitching and sounds like buffer underrun, like Unity cost more spec but doesn’t update the buffer size, so Unity can’t keep up with the old buffer size anymore even in editor. (If you go set something not Best Latency it will improve, at the cost of latency)

Here’s my case where UT said just change to something not “Best Latency” : https://fogbugz.unity3d.com/default.asp?1118164_ip87hse66aelc9od but the point is, it was working under Best Latency before the recent version. It is a regression.

Unfortunately I don’t know how to benchmark the “buffer underrun” other than using my ears. If that is possible programmatically (you may have to ask audio driver that it found nothing to process and send to speaker… but how?) we can dynamically adjust buffer size and restart audio system until it doesn’t crack.

1 Like
13

Please make your own topic, don’t drag thread off topic.

1 Like
14

Vulkan runs on Windows.

2 Likes
15

please

it runs on windows, please do a test?

16

Holy moly, will do a Vulkan test at the weekend then!

Do you know if I can start the build/player with a command-line to select a specific graphics api (Direct3D11, Vulkan, etc)?

1 Like
17

there are a bunch of -force-XXX switches (Unity - Manual: Command-line arguments), maybe try -force-vulkan or something

2 Likes
18

or just remove the automatic graphics setting for windows on player settings and force it to use vulkan there :slight_smile:

1 Like
19

I will give this a shot. I thought they’re for the editor only. We’ll see :slight_smile:

Yeah I’m aware of this option. I was trying to find a way to build the player once only and then test all graphics api’s, rather than having to build a new player for each test.

1 Like
20

Here are the tests for the different graphics api’s in Unity 2019.1.0b4. I tested D3D11, D3D12 (experimental) and Vulkan.

I’ve to admit that the game is not really graphics demanding. It’s quite simple what’s going on visually and it has only a few draw-calls. There are most likely other projects better suited for a graphics performance test than mine.

Anyway, here are the results.

4298446--385624--4_3.png

4298446--385627--5_4.png

4298446--385630--6_8.png

------------------
System Information
------------------
         Operating System: Windows 10 Home 64-bit (10.0, Build 17134) (17134.rs4_release.180410-1804)
      System Manufacturer: System manufacturer
             System Model: P5K/EPU
                     BIOS: BIOS Date: 06/19/08 17:02:36 Ver: 08.00.12 (type: BIOS)
                Processor: Intel(R) Core(TM)2 Quad CPU    Q9400  @ 2.66GHz (4 CPUs), ~2.7GHz
                   Memory: 8192MB RAM
          DirectX Version: DirectX 12

---------------
Display Devices
---------------
           Card name: NVIDIA GeForce GTX 650 Ti
Driver File Version: 25.21.0014.1917 (English)
      Driver Version: 25.21.14.1917
         DDI Version: 12
4 Likes