Native Audio
Lower audio latency via direct unmixed audio stream at native side
Requirement : Unity 2019.3 LTS+ For iOS and Android.
Asset Store Link : Native Audio | Audio | Unity Asset Store
Release Note : in the website Changelog | Native Audio
So your Unity game outputs WAY slower audio than other apps even on the same device? Turns out, Unity adds as much as 79% of the total latency you hear. But not to worry because Native Audio will take care all of it. I have researched into the cause for so long and the solution is right here.
What can we skip by going directly to the native API?
Unity has an internal mixing system, backed by FMOD. You can go crazy with a method like AudioSource.PlayOneShot
and the sound magically overlaps over itself,even though at native side Unity only asked Android for only 1 “Audio Track”. How can that be? Because Unity spend time mixing them together to 1 bus. Plus you have all the wonders of Unity 5.0.0 introduced audio mixer system. Effects, sends, mixers, etc. all stacked into that.
A great design for a game engine. But we certain subset of game developers absolutely do not want any of the “spend time” if possible. Unfortunately Unity does not give us a choice to bypass and just go native.
For some genre of apps and games that needs critical timing for audio (basically any response sound from input, rhythm game, etc.) this is not good. The idea to fix this is to directly call into the native methods and have it read the raw audio file we prepared without Unity’s importer. Bypassing Unity’s audio path entirely.
I have researched for long time into the fastest native way of each respective platform. For iOS it is to use OpenAL (Objective-C/Swift) and for Android it is to use OpenSL ES (NDK/C). For more information about other alternatives why they are not good enough, please go to implementation page.
But having to interface with multiple different set of libraries separately from Unity is a pain, so Native Audio is here to help…
“Native Audio is a plugin that helps you easily loads and plays an audio using each platform’s fastest native method, from the same common interface in Unity.”
Android High-Performance Audio Ready
It improves latency for iOS greatly, but I guess many came here to fix the already-horrible-without-Unity Android latency.
I am proud to present that Native Audio is following all of Google’s official best practices required to achieve High-Performance Audio in Android. I have additionally fixed all the latency-related mistakes that Unity had to unfortunately chosen for their “versatile” audio engine.
- Uses C/NDK level OpenSL ES and not Java
MediaPlayer
,SoundPool
, orAudioTrack
. Plus, most latency-critical interfacing methods from Unity are byextern
to C and notAndroidJavaClass
to Java. Feature set of OpenSL ES that would add latency has been deliberately removed. - Ensuring “Fast Track” audio being instantiated at hardware level, not a normal one. Native Audio does not have any kind of application level mixer and each sound goes straight to this fast track. Currently Unity does not get the fast track due to sample rate mismatch, moreover somehow using a deep buffer thread, designed for power saving but with the worst latency.
- Built-in resampler. Resample your audio on the fly to match “device native sample rate”. Each phone has its own preferred sample rate. (Required for fast track)
- Minimum jitter by zero-padded audio, so that the length is exactly a multiple of “device native buffer size” to ensure consistent scheduling. Each phone has its own preferred buffer size.
- Double buffering so your audio playing start as soon as possible, unlike a lazy single buffering which we must push entirely of the audio into the playing buffer. (This is not the same step as loading the audio, we must go through this step on every play.) Combined with the previous point the workload of each callback is deterministic.
- A support for AAudio, the new and even faster standard from Google is coming in the future. Players that has Oreo (8.0) or higher will automatically get AAudio implementation with no modification from your code.
- Automatically receives better audio latency from future system performance improvements.
Of course, with publicly available thorough research and confirmations. This means it can perform even better than native Android app that was coded naively/lazily in regarding to audio. Even pure Android developers might not want to go out of Java to C/NDK.
How faster?
Here are some benchmarks. While this is not a standard latency measurement approach like loopback cable method and the number alone is not comparable between devices, but time difference of the same device truly show how much the latency decreased without doubt.
The website contains much more to read : Native Audio - Unity Plugins by Exceed7 Experiments
Please do not hesitate to ask anything here or join the Discord channel. Thank you.