Unity listen and analyze audio playing (from another app)

Hey all,

This is my requirements:
I am playing a video game and the audio output is on a audio channel such as “SteelSeries Sonar - Gaming”
I want to create an unity application which listens at realtime to that audio channel and then do some fancy processing on it.

My current flow for initial testing was using a record clip to set that as an audio clip on the audio source and I was able to do the following to get some data which I required - it was an audio clip which flipped audio from left ear to right ear for testing and had 2 channels

audioSource.clip.GetData(clipSampleData, audioSource.timeSamples) ;

This all seemed to work perfectly and I got the results I wanted

Now the next step was to try and get unity to realtime listen to that audio channel…I went down a rabbit hole of using VB-Audio cable to route the output of the game into the input cable and then did the following

 audioSource.clip = Microphone.Start(SelectedMicrophoneName, true, 1, 44100);
audioSource.loop = true;
audioSource.Play();

now while it somewhat plays the audio the problem I instantly see is when reading the clips channels it now only has 1 channel and I therefore have lost my left and right channels

Is there a proper way to make the audio clip be the realtime audio channel?

Thanks in advance

Hi!

I think you’re hitting a limitation of our Microphone API. The buffering AudioClip that is created to contain the audio you capture is only mono, based on what I see on the native side of the engine.

If you are only targeting Windows, I’d recommend you look at NAudio (GitHub - naudio/NAudio: Audio and MIDI library for .NET) but if you need something cross-platform, you might have to get your hands dirty with some C/C++ and go for things such as PortAudio, JackAudio, libsound or miniaudio.

Godspeed!