Burst In Editor with Safety On Running 7-8x Slower In 2022.1.0f vs 2021.3.2f1

[colinleet]

First of all I just want to say that I’m a giant fan or the Burst compiler + the Jobs system, it’s the core tech that runs the heart of my game, and I am floored basically daily at how fast it almost all of the time… however…

I’m encountering a new very annoying (bug? or) general 7-8x slow down when running burst compiled code with the safety checks on in the first release of 2022.1 with Burst 1.7.1 vs 2021.3.2f1 with Burst 1.6.5.

My test case is a very large and complex Burst compiled job system that I’ve made for high throughput audio processing. I’m used to having the safety checks roughly double to tripling the overall runtime of my code which up to this point hasn’t been a large issue normally (for my case in 2021.3 it added just an extra 1-2 sec max). However after trying out 2022.1 today that wait has increased to around 13-14 seconds (vs 600 ms w/o safety checks).

My test setup: Here are some example timings I’ve gather from builds from both versions of the editor, and their current standalone builds for my use case. All of the code being run is identical, and they’re all tested on the same 4 min long track. All of these tests shown are from the second runs of the code (or later) to remove any effect the JIT Burst compiler has in editor.

These first two tests are running safety checks in-editor for both Unity versions:

This is the case running 7-8x slower (1.76 → 13.45 sec in the new version). The last line indicates the actual runtime for each set of job, what I’m measuring on the first line is just the time it takes to launch and prepare all of the jobs in the series.

Here is the same task run in 2022.1 without safety checks turned on:

Luckily build times are unaffected (as they should be):

Although I can get around this just by turning off safety checks every time I load the editor, this is going to remain a sizable annoyance to me in the long term as I end up developing new burst code when it will inevitably need to be debugging it.

I also feel like it’s generally bad form to be writing burst code w/o that option turned on most of the time; especially when testing for bazar corner cases… which sometimes may not-pop-up on their own for months after a game breaking error is introduced / then it’s next to impossible to debug without leaving this feature on, or disabling burst entirely for a particular job (both of which blow my VR time budget out the window currently in most circumstances).

Additionally when I inevitably (often) end up exiting out of play modes accidentally w/o turning the safety checks off beforehand, this (bug?) is resulting in my having to wait sometimes close to a full minute just for the series of jobs to complete if a longer track (like 10 min) is randomly selected-- as there is no way of canceling a job once it’s scheduled; and exiting out of play mode (even accidently) while the jobs are running results in memory leaks that force me to reboot the whole editor. The my only real fallback to complete the series of jobs on the main thread when exiting play-mode… However this still ends resulting in a 1-2 minute full-editor-locking-task while it wait for the job to complete for longer tracks, and then deallocated all of the working data.

I guess mostly what I’m looking for is information as to if this is expected behavior? And if so what so radically changed from the previous system? I’m not even sure what I should/would report for this in a bug report (apart from what’s above) as it isn’t actually causing any of my code to break, just perform horribly when debugging…

[colinleet]

I reached the photo limit in the OP; here is the setting causing all of my headaches in 2022.1.0f1:

[sheredom]

Could you provide us either with a repro project for this, or if you open the Burst inspector for that job on each editor, if you could grab us the LLVM Optimized tab’s contents, that will help us start narrow this down.

[colinleet]

@sheredom

Could you provide us either with a repro project for this, or if you open the Burst inspector for that job on each editor, if you could grab us the LLVM Optimized tab’s contents, that will help us start narrow this down.

https://github.com/colinleet/Burst-Saftey-Slowdown

I’ve included two sample jobs (source code) with vastly different complexity levels as examples. For both of them I’ve added timing screen shots of the profiler showing them running the same track as above with and without safety checks for 2022.1.0f1. I also added a single test of the same code and track running in 2021.3.2f1 with safety checks for reference. I also included the full LLVM Optimized output, with and without safety checks, for both versions.

The simple job only contains around 9 operators – mostly read / write. All it does is split a stereo wav file into three audio streams (left, mono, right). It’s only showing around a 75% slow down with safety between the two versions.

The other job is a full implementation of a FFT (around 500 lines of uncompiled code w/ comments). It’s showing over a 10x slowdown between the two editor versions in the included tests (result). I included the original C# version I adapted it from for clarity.

I can’t provide a full working unity project with these as that would require me putting too much of my game’s core IP online, but I did include the full source for these two job as their algorithms are easily available everywhere; if not in this coding environment.