Audio Safety Suggestions

Hi all,

I am developing an application that is using live microphone input, which is routed through different audio mixers with different plugins altering the voice. The plugins are developed externally but should work just fine. Additionally, the microphone input controls lip animation using the SALSA lipsync suite, and everything is run in VR.

Overall the application works well, but maybe 1 in 50 times I experience audio issues, mostly with the audio not appearing or suddenly lagging.

Do you have any suggestions on how to handle audio streams in Unity in a safe way? E.g. thread safety, checking that input is detected, checking that buffers are filled etc.?

Or are there any online sources about this topic?

Looking forward to hear your inputs :slight_smile:

Can you elaborate on what goes wrong when it happens? Do you have sound at all, is it super slow and choppy or high-pitched and distorted? This might allow to identity the cause of the problem.

In general, monitoring for buffer under/overflow at multiple points of your signal could allow you to identify weaker points in the chain. Then you could try to rebalance buffer sizes of plugin parameters that might affect the performance of various nodes.

If that is relevant, you could try to leverage the Job system for concurrent and async work.