Today I ran the automated performance test of my game with Unity 2018.3.0b3 and present the results below, just like I did before, during older beta cycles:
According to my tests, 2018.3.0b3 is the slowest of the 2018 series with my project. It’s about 1ms to 1.5ms slower than 2018.1 for example.
I haven’t looked into what exactly is causing this performance difference yet.
How does the performance test work?
The (first-person) game I’m working on features an automated performance test. It works like this:
Camera is placed at a defined position in the scene
Camera makes a 360 degree rotation around the Y-axis within 20 seconds (slowly rotating around y)
Every second the test captures the average CPU frame time of the last second
The game runs just like it would normally do, but the camera/player does not move and the AI in unable to see the player.
It measures the “base-line” of the game so to speak. If an actual gamer plays the game, more things are going to happen, which is missing in the test, such as visual and sound effects when firing weapons, additional AI to hunt the player and more navigation pathing.
I run this test in a Standalone Windows 64bit (Mono) Player. The following Physics settings are applied:
Physics.autoSyncTransforms=false
Physics.autoSimulate=true
Physics2D.autoSyncTransforms=false
Physics2D.autoSimulate=false
The y-axis represents the average CPU time in milliseconds, how long “one frame took”. The x-axis represents the sample points while the camera makes its 360° rotation. Each test runs for 20 seconds, one sample every second, thus 20 samples for each test. Fewer milliseconds (vertical axis) indicate better performance.
Physics regressed several times with new Unity versions in my project. I have a simple test project that I used to submit those physics performance regression issues in the past. Unfortunately, I wasn’t able to reproduce the performance drop with this test-case in 2018.3 anymore, so I’m not really certain it’s (only) physics.
I need to investigate this in detail, just need to find time.
What I could find so far, it seems there are at least two performance regressions in 2018.3.
For physix you may also give it a go to the physx experimental editor based on 2018.2 that includes 3.4.1 in my tests that did run like a missile. In .3 betas it did not run that smoothly but there where many other things broken that could be making it lag.
The regression was introduced during the 2018.3 alpha and doesn’t affect the experimental build or earlier versions than 2018.3a6. Let’s see if we can get 2018.3 back to missile speed once this has been resolved.
If this is the same test each time from Peter, is there a good reason his test isn’t part of Katana or other automated testbed yet given it’s been consistently useful
Yeah I know, I need to find time to investigate these issues. It’s a more complex issue that takes time to analyze and come up with an isolated repro.
I just remembered I actually posted this problem a few weeks ago. I first noticed it using b3, but I don’t remember if I earlier tested versions too. https://discussions.unity.com/t/716914
Knowing this issue existed when I tested b3, I guess the performance drop is not directly related to it, since b3 ran faster than b6.
It’s the same test. I created a copy of my game a few months ago, which I since then used to test all betas.
It’s the same thing, no modification done to the project copy since the very first performance test, to make sure the tests are meaningful.
In order to run the test, I have to build a player and start the generated .exe file, which then spits out a .csv file once it’s done.
I was investigating the issue and noticed after I profiled b3 last time, I turned off various PostProcessScene steps in the project, to check if it affects the “Destroying a Prefab…” error and I didn’t turn those on again.
Sooo… the huge performance drop from my previous b6 profiling session can be seen as wrong data, because it was measuring the performance without any optimization steps, thus significantly slower than before and not a meaningful comparison to earlier profiling results.
However, since PostProcessScene causes some issues here , I ran those optimization steps outside “build player” to have the proper optimized scenes and then did a build afterwards.
The results are much more like Unity 2018.2 now, see below. 2018.3 is still slower than 2018.1 with my project though.
Thank you for the info and at the same time it’s quite sad to see such results. There is a new version of PhysX which is quite faster but we have the same performance… I hope it’s due to the beta version and the final one will be more better.
Yeah it’s pretty underwhelming. I had hoped that PhysX 3.4 will boost physic performance but after some tests it’s far from the case and it’s even worst. Going back to 2018.2 right now
Same here, a VR experience was runing at 72FPS in 2018.2.14, and around 65FPS (noticeable frame drops) on 2018.3.0b9 (Still trying to find what is happening… no luck)
I guess this depends a lot on what kind of physics your game relies, in coming from unity 4.7 in 2018.2 I had to reduce the timestep to 1.5 for a joint based tank vehicle not to have explosive physics. In the experimental 2018.2 eidtor using 3.3physx and 2018.3 using 3.4 I can double the dimestep without having noticeable issues hence the cpu stress release, and I could even push higher.
Is the “Gfx command not handled” a known issue? The error was printed only twice and the game looked fine to me, no idea what’s wrong.
Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
1/6 scenes use different skyboxes.
(Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
1/7 scenes use different skyboxes.
(Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
1/8 scenes use different skyboxes.
(Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
1/9 scenes use different skyboxes.
(Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
Your current multi-scene setup has inconsistent Lighting settings which may lead to different lighting when loading scenes individually or in a different order! Consider homogenizing the following:
1/10 scenes use different skyboxes.
(Filename: C:\buildslave\unity\build\Runtime/Graphics/LightmapSettingsManager.cpp Line: 123)
I reported the “inconsistent Lighting settings” issue during 2018.2 beta, seems it’s not fixed yet.
Did this get reported with a reproduction case? These error messages are an indication of something wrong in the internal graphics command queue and it would be good to be able to reproduce it and debug it and fix it
