The performance is significantly slower than the older version. At 0.5.1-preview.2, it’s faster than 1.0.0-pre.65 quite a lot.
Thanks for bringing this to our attention. We will have a look at this.
Did you notice any particular areas in the profiler which show where the change in time consumption originates from?
Note that depending on the amount of content in your scene, there can be a significant overhead that is caused by an integrity check that is only done in the Unity Editor for validation.
See here for more information.
From my observation, the stall mostly comes from BuildPhysicsWorld that looks like it caused by forcing to complete job immediately and run multiple times in one frame. Another huge stall is caused by EntitiesGraphicsSystem that looks like it’s slowdown by dots physics and most of the time is doing nothing.
1 Like
Thanks for the added information. Could you share a profiler screenshot that shows this?
I have not observed more than one BuildPhysicsWorld run per frame except if you use Netcode for multiplayer which does predictive simulation (multiple steps per frame). Are you using Netcode here?
This is what I am seeing (one time BuildPhysicsWorld system update). Example profiler run from the scene “Assets\Tests\Pyramids\Pyramids.unity” in the official Unity PhysicsSamples project.
And here is the hierarchy view for some other frame in the same run. As you can see the BuildPhysicsWorld time consumption is low and it is called only once.
My PhysicsInitialize / BuildPhysicsWorld step has become comically slow. I was looking for a reason, but it might be the new Preview version.
Used to get 300+fps, now it’s more between 3 and 30 fps. Almost all of the frame time is lost in that step. Unfortunately, I cannot drill down any lower, it seems. It’s just “lost” somewhere on that system. (will try a deeper profiling session after work today)
That’s very odd.
Can you try the scene I mentioned above, to make sure we get a baseline to work with? I tested all this with the latest version.
1 Like
ECS Samples repo has a bunch of issues when imported from scratch into Unity 2022.2.17f1. The prefab mentioned here actually exists, too, so this message is doubly bewildering.
I’ll try to get it to work and do some profiling.
1 Like
Ok, the framerates are bad on ECS Samples changeset 02bbced477b0458a9e31279fdc942f5676ff4909, in Unity 2022.2.17f1, with the following peculiar details:
There are no errors, but there’s a weird warning I have never seen (I think we’ve all seen the “1 entitirs in the scene… one”, I mean the "2 settings provider one)
Systems Window doesn’t show the timings correctly, they always disappear after a few frames, then reappear.
On a 60 Hz display at 2048x2048 (all other things as default when checking out the project, this is just my default editor layout), I get around 70-85 fps. It certainly feels slower than that, though. (I’d say about half)
Using Unity’s Default layout on a 120 Hz display gives SLOWER framerates even though the times are… lower?
Profiling gives no super baffling results, other than a significant portion of time being lost on the editorloop:
It very much does not feel like 5 ms per frame, but if I look at the Editorloop, that takes up around 11.5 ms.
The plot thickens - I have now DISABLED the Collider Integrity checks, and frame times more than doubled, with time lost exclusively in the playerloop. This now feels very sluggish.
The sluggishness went away after switching tasks, I suspected it was a Burst JIT issue or something, turning off Burst Compilation gives me pretty identical performance.
However, it didn’t go away on its own the first time, so it felt more like related to the Unity EditorApplication (even though the massive frame time “spike” does not occur in the EditorLoop) because it sped up only after I switched tasks away from Unity.
Maybe some flip/flop bi-stable glitch in Burst that sometimes just hangs compilation, resulting in (even more) abysmal performance.
This super slow performance is pretty much in line with what I see in my larger project since about 2022.2.15f or .16f - I’ll try .18f in a little bit.
On the 120 Hz display, when Burst seems to be doing its job, performance feels okay looking at the profiler (2.5ms, give or take)
However, the Editorloop still eats 12+ milliseconds out of every frame, even if the editor displays nothing - empty inspector, sceneview hidden just default layout the way Unity intended. (ok, that’s a lie, I run at 150% UI scale)
Overall performance is much better in 2022.2.18f1. 
I can no longer reproduce the drastic performance loss in my larger project.
(there’s still a loss, but it’s not in PhysicsSystemGroup, but 6:1 proportionally in RenderPipelineManager.DoRenderLoop_Internal(), and this is probably a per-camera optimization I must make thanks to some URP changes)
If I create a lot (around 20k) physics bodies, capping my game’s budget, I see an interesting pattern of small performance spikes where “BeforePhysicsSystemGroup” takes 6x as long as usual, but this could to be a sync point in my own code that I need to work around)
Overall, 1 ms average time for 20k bodies is very good (I’ll look at the sync point separately, it’s probably my pathfinding system)
(EDIT: Just compared to 2022.2.16f1 with all other things exactly equal, and .18’s performance in physics is more than double [<1 ms per tick vs >2ms per tick], though the 10x to 20x loss is gone and I would ascribe that to a Burst issue, lacking another explanation).
Edit: And one more, there was an issue in pre-2022.2.18f1 where Unity on editor closing would hang on CleanupMono about 2 out of 3 times, I haven’t observed this even once since the new URP that came this week.
1 Like
Here’s the screenshot. I didn’t use use dots netcode. I just spawn like 20000 balls and when they hit the ground and after awhile it will suddenly become extremely lag. You have a look at IN-39547 repo project.
1 Like
@Thygrrr : Great stuff! Thanks for the thorough investigation.
One thing I am wondering is how the 120 Hz display plays into all of that. Could V-Sync and/or the default ECS timestep of 60Hz play a role here?
Btw, the case above where you noticed time lost in the editor loop, wouldn’t that just be the loop enforcing real-time framerate with a 60Hz simulation time step (which is the default in ECS; see SystemAPI.Time.DeltaTime)? If there was no “pause” somewhere, the simulation would run faster than real-time.
After having had a closer look at your screenshot, I can see that the whole physics system group is run 5 times here per single fixed update.
The reason for this is likely that the FixedRateCatchUpManager is trying to catch up with the real-time simulation rate, but needs to step the physics an excessive number of times since a single step time consumption is likely too high in your scene.
The Entities framework employs simulation sub-stepping for cases in which the system needs to “catch up” due to excessive time consumption in the game loop.
This is done in the FixedRateCatchUpManager, which implements system update semantics similar to UnityEngine.MonoBehaviour.FixedUpdate().
The method can be configured in a way that a single “catch up” step does not exceed the World.MaximumDeltaTime value, which prevents simulation issues such as tunneling and the like.
However, due to this clamping of the variable time step, sometimes the system needs to step multiple times to catch up which then aggravates the issue in a negative feedback loop, as is likely the case here.
So, in summary, have a look at the stepping parameters used by the FixedRateCatchUpManager, and either configure it in an attempt to reduce this effect, or solve the issue at the source by optimizing your scene further.
I see but the same amount of balls perform much better at 0.5.1-preview.2 compares to 1.0.0-pre.65 which I believe it’s performance regression. I would like to know why 0.5.1-preview.2 is so much faster? One performance regression issue I know at 1.0.0-pre.65 is that when there’s lots of physics entity stack together it will regress performance significantly.
I totally understand that. We have your test project in hand and have filed an internal bug report for investigation and are going to look into this. Stay tuned!
2 Likes
Hi. Any new update? Btw I think official really needs to completely eliminate stalling at build physics world that this causing insane slowdown at dots netcode prediction that build physics world will running multiple times in one frame. So when total up it becomes really large ms to execute. Not sure how official going to solve it but looks like need to further improve dots physics pipeline.
Thank you for the feedback! We continue to investigate this issue as part of our overall backlog. We’ll share more when we have additional details.
There are two major stall in the physics 1.0 that cause the build physics world to wait for other jobs and
Daniel is investigating on it. There are indeed some little more stalls now by “default” on the main thread in the physical loop.
In particular, the internal BuildPhysicsWorldDependencyResolver system is usually the one that add some of these stall, since it introduce a sync point on the main thread: it will wait for all jobs that touches PhysicsWorldSingleton, BuildPhysicsWorldData and / or PhysicSimulation singleton entities.
That being said, the game just entered into the classical death-loop: there was a long frame (maybe because of physics), that cause physics run multiple times, that will produce again an even longer frame, etc etc up to a point where it become like that.
This is never ending, unless the catch-up mechanism decide to say something like: “you know what, let’s simulate bigger steps (imprecise) to exit this loop”. But this is not the way it is implemented.
I would suggest to implement your own FixedRateManager and do some logic there to determine how to clamp and handle cases where the elapsed time in between frame may cause physics to run to slow and enter into this situation.
Another solution is to not-rebuild the world at all, apart the first time. And that can be implemented by a custom system that selectively:
- Disable the BuildPhysicsWorld
- Ask instead to update the AABB tree (imprecise collision detection though)
That requires some little addition at the moment to the BuildPhysicsWorldData to make this cleaner but it is a viable solution, that led to very good performance improvement.
Because for netcode we had similar problems, if we have high latency the client need to simulate a lot of physics step in one frame to catch up, we experimented on a bunch of thee.
By applying both a sort of “adaptive physics-step rate” (that lead to misprediction, and non-determistic results, as I described) as well as by avoiding to rebuild the physics world, you can limit the problem.
2 Likes
I guess currently it’s not possible to override current physics system without modify the dots physics package right? I would like official to make overriding dots physics much easier without required to modify dots physics package and provide utility methods and others to further make it much easier.
But still if official can improve all these options for unity developer to toggle on/off the required parts then it’s even nice. To be honest I not sure how to implement these solutions. Looks super advanced for me.
1 Like
For me, the issue is back with a vengeance in 1.0.10.
I created a thread because I don’t even know how to dig deeper into profiling, and it impacts the LTS version rather than pre65. Just
https://discussions.unity.com/t/919600
It seems to compound drastically above 10k entities.
All my other physics jobs are pretty simple, so I wonder what the stalling is all about. They should be long, long finished when BuildPhysicsWorld even starts. (maybe they don’t, the scheduler is still pretty opaque to me - I read a “UpdateBefore” and “UpdateAfter” as “execution of the other system is guaranteed to finish before the other updates”)
Specifically, any jobs in BeforePhysicsSystemGroup should be completely done executing before BuildPhysicsWorld updates, right?
Unfortunately, it seems beyond my capabilities to set breakpoints or reliably profile or inspect burst code to validate that assumption, or even to find out which SPECIFIC JOB it is waiting for here in “240 instances on the main thread” (maybe a good idea to add that to the profiler marker or make a profiler counter with a parameter for it?):