Game crash on fast computers only, can't find any hints of what's wrong

Unity Engine
, , ,
1

Hi,

I have a very strange issue with the builds of my game, and after hours of debugging and profiling I still have no clue what it could be or what to focus on trying next. Here is what I figured out, maybe it could ring a bell for some of you:

  • It crashes only in builds.
  • It always crashes fast computers (tried on 3 different ones, AMD/NVIDIA doesn’t seem to matter). Slow computers seem to do just fine and don’t seem to crash at all (also tried on 3 different ones)
  • It crashes kind of randomly (after loading), usually after a few seconds but could take at least a few minutes (which makes debugging really annoying, since I can’t know if it’s not gonna crash ever or it haven’t crashed yet).
  • Nothing in logs other than the normal unity init stuff. It’s not being picked up by the crash handler either.
  • The crash does not close the game, it only freezes the frame and stop responding.
  • Turning on VSync seems to mostly prevent the crashes (may have still happened once).
  • I started the project on 2021.3, upgraded to Unity 6.0 at some point. I didn’t build on 2021, but the crashes happened on multiple versions of Unity 6.0.
  • I tried disabling the new Render Graph thing, doesn’t change anything.
  • Couldn’t find anything in profiler (even linked to the build on a computer that crash). There are some spikes sometimes but it may be normal at 1000 fps (see below).
  • I tried building on another computer, same.
  • Nothing out of ordinary in this project, just some custom toon shaders and some render pass (full screen and render objects).
  • I work on another, heavier project on the same computers, same versions. This one never crashed.

About the scene:

  • I couldn’t really find a meaningful relation between the crashes and the number of objects in scene.
  • I tried removing all scripts in scene and all async (so no custom code running at all). Still happens. Just a few models are enough to crash (seems like skinned mesh renderer are the worst offenders).
  • I tried building with only with certain models, only skinned mesh renderer, only mesh renderers, only particle system, UI, light/no light. Each time I think I figured which thing might be the issue, it either stops crashing or something else make it crashes when I sanity check…Thus I don’t think it’s a particular shader or mesh but I don’t even know for sure as I’m highly confused.
  • Empty scene doesn’t seem to crash (or don’t crash quickly at least).
  • Doesn’t seem to matter if another scene load in single or additive mode.

Profiling Spikes just in case:

Crash_Spike2GPU
Crash_Spike2GPU1269×748 117 KB

Crash_Spike2CPU_NextFrameDetails
Crash_Spike2CPU_NextFrameDetails1267×753 116 KB

The only way I found to try stuff is to make changes, build, wait for it to crash or not (except I can never know if it would crash eventually) but it’s quite time consuming. Also, of course it never crashes on my computer, I can only try a few hours at work from time to time. So if you have any idea or tips to help me debug this that would save me so much trouble.

2

Just wondering aloud here… for these weird times when there is just no clues left in logs or anything… would something like Windows Process Monitor help?

It’s been ages since I used it “for reals…” I wonder if it lets you drill into signals sent back when an app quits.

It might still just tell you “general protection fault”, but who knows, it might have additional crash indication data in the crash, or you might see it happens when the process attempts X-and-such, which might also give intel about what is dying.

Similarly I wonder if some of that video card middleware has logging or development tools you can activate on it to get more info.

3

That’s not a crash but a freeze. Crash != Freeze :wink:
This is an important distinction that helps avoid confusion.

Most likely cause: an infinite loop, and most likely in your code. Understandably there’s nothing in the log - unless you happen to log something in the loop that might run infinitely.

Make sure to create development builds. Connect the debugger to running builds. If it freezes, pause in the IDE and go through the thread callstacks - should be easy since almost certainly you’ll have the main thread freezing.

You can also try to narrow it down by adding spot-check logs in potential infinite loops. Especially scrutinize any and all uses of while. These are traps! Example coroutine:

IEnumerator SomeCoroutine()
{
   while (true)  // BAAAAD pattern!!!
   {
       // lots of code
       // lots of conditions like these
       if (whatever && wut)
           yield return null;
       else if (!whatever && wut) // BAD pattern, don't repeat and negate conditions)
           yield return new WaitForEndOfFrame();
       else if (!whatever && !wut)
           yield return new WaitForSeconds(1);

       // HIDDEN PROBLEM:
       // whatever == true and wut == false => infinite loop
   }
}

You are on 44f1. Try updating to the latest. You should do so anyway due to the recently discovered security issue affecting builds (ie not you but your users).

4

I’ve never used it to get logs but that’s an idea, I’ll look into it thanks!

That’s what I assumed was going on at first, some kind of racing condition that would arise only if things loaded fast enough. So the first thing I tried is to debug, then eventually remove all scripts from the scene and RuntimeInitializeOnLoad attributes, but the issue still happened. It could be a good idea to check this again just to make sure tho, as I might have forgotten something the first time.

Oh yes I always forget to do that with builds, that might lead me to the issue indeed!