iOS Audio Latency, Capture and Playback from Microphone

Unity Engine
,
1

Hey guys, I’m working on a voice changer app, currently in version 5.3.1. The app captures the users voice from the microphone, adds some audio effects, and then plays that clip via an audiosource. With ‘DSP Buffer Size’ set to default on my iMac, the playback sounds great with very little latency. Unfortunately, when built to iOS device (iPhone 6 and 6+) with the same settings, there is .5-1 second latency between talking and the playback of the voice.

I don’t believe the problem to be hardware related, profiling on device shows the app using between 7-15% CPU and approx 60MB of memory.

I have not been able to find in docs or elsewhere a way to manually set the DSP Buffer Size, outside the predefined options in Project Settings> Audio. While choosing ‘Best Latency’ or ‘Good Latency’ in DSP Buffer Size alleviates most of the latency, they cause stuttering during playback. Maybe due to the buffer filling/dumping? Only ‘Default’ and ‘Best Performance’ eliminate the stutter, but of course with those settings the latency returns.

The way I’m capturing voice and playing back is very simple. In a nutshell:

public void InitMic()
    {
        if(Microphone.devices.Length > 0)
        {
            Debug.Log("Mic Init");
            mic = Microphone.devices[0];
            //Debug.Log("Mic name: " + mic);
       
            source.clip = Microphone.Start(mic, true, 30, 44100);
            source.clip.LoadAudioData();

            source.Play();
            source.mute = true;
            isMuted = true;
            _isInitialized = true;
        }
    }

    void Update()
    {
        if(_isInitialized)
        {
            //match the samples of the player to the mic clip, so that we eliminate lag in playback
            source.timeSamples = Microphone.GetPosition(mic);

            micLoudness = LevelMax();

            //if our voice is lower than the threshold and we are currently muted (were already talking), start the end playback portion
            if(micLoudness < volumeCutoffThreshold && isMuted == false)
            {
                StartCoroutine("EndPlayback");
            }
            if(micLoudness > volumeCutoffThreshold)
            {
                source.mute = false;
                //source.Play();
                isMuted = false;
                StopCoroutine("EndPlayback");
            }

            if(!source.isPlaying && Microphone.IsRecording(mic))
            {
                source.Play();
            }
        }
    }

Look forward to your ideas or feedback. Thank you!

2

I used Best Latency in my last game and it worked on iOS without any issues. But it’s still less than ideal because Unity uses FMOD if I’m right, which is still a ‘higher’ level audio. To achieve extreme low latency you have to write plugins to utilize the core audio of iOS directly.