AudioClip.PCMReaderCallback has insane latency

EDIT:
For anyone finding this post in need of some extra context or a solution. I gave up with trying shortly after I posted this thread. After a year, I decided I would try again. Here is what I found:

the OnAudioFilterRead callback works fine. The delay was because of the way I read the microphone. I didn’t wait for the microphone to reach position 0 in samples. (With Microphone.GetPosition) before reading the data. Resulting in a delay.

For the PCMReaderCallback I could not find a solution. The latency of half a second still exists, even with a small DPSBufferSize or low sample count. It always seems to pre-buffer. This is unfortunate, because this callback works properly with spatializing (with Steam Audio too!) while the OnAudioFilterRead needs some extra effort to make that work.

original post
Hello, I’m trying to implement VOIP in my game using Opus and Mirror. My efforts so far don’t seem to be making any progress in solving the latency issue that I get.

When the audio is received in the client, I play it using the AudioClip.PCMReaderCallback where I just change the data to the latest info, but this seems to have a big latency of about 500ms/1s, which is just way too long.

I’ve also tried using the OnAudioFilterRead method, but the samples there are completely wrong and makes the voice sound very choppy, and like a chipmunk, but then the latency is minimal.

What’s up with this? Does anyone know how to fix the delay issue that I get with the callback?

2 Likes

I just stumbled upon this issue.

This script demonstrates the huge delay of PCMReaderCallback.

  • sound is played via left mouse button
  • if using OnAudioFilterRead then there is minimal delay
  • if using PCMReaderCallback (OnAudioRead method) then there is a huge delay
using UnityEngine;

[RequireComponent(typeof(AudioSource))]
public class PcmReaderCallbackTest : MonoBehaviour
{
    public int sampleRate = 44100;
    public int outputChannelCount = 2;
    public int lengthInSamples = 2048;
   
    private AudioSource audioSource;
    private int totalSampleIndex;

    private bool shouldPlay;
    private bool oldShouldPlay;

    private bool filledSound;
    private bool oldFilledSound;
   
    private void Awake()
    {
        audioSource = GetComponent<AudioSource>();
       
        // Comment out this line to use OnAudioFilterRead instead of OnAudioRead
        audioSource.clip = AudioClip.Create("PcmReaderCallbackTest", lengthInSamples, outputChannelCount, sampleRate, true, OnAudioRead);
       
        audioSource.Play();
    }

    private void Update()
    {
        shouldPlay = Input.GetMouseButton(0);
        if (shouldPlay != oldShouldPlay)
        {
            Debug.Log("shouldPlay: " + shouldPlay);
            oldShouldPlay = shouldPlay;
        }
    }
   
    private void OnAudioRead(float[] data)
    {
        FillBuffer(data, outputChannelCount);
    }

    // Uncomment this method to use OnAudioFilterRead instead of OnAudioRead
    // private void OnAudioFilterRead(float[] data, int channelCount)
    // {
    //     FillBuffer(data, channelCount);
    // }
   
    private void FillBuffer(float[] data, int channelCount)
    {
        if (!shouldPlay)
        {
            for (int i = 0; i < data.Length; i++)
            {
                data[i] = 0;
            }

            filledSound = false;
            if (filledSound != oldFilledSound)
            {
                Debug.Log("filledSound: " + filledSound);
                oldFilledSound = filledSound;
            }
            return;
        }

        int frequency = 440;
        for (int sampleIndex = 0; sampleIndex < data.Length; sampleIndex += channelCount)
        {
            for (int channelIndex = 0; channelIndex < outputChannelCount; channelIndex++)
            {
                data[sampleIndex + channelIndex] = Mathf.Sin(2 * Mathf.PI * frequency * totalSampleIndex / sampleRate);
            }
            data[sampleIndex] = Mathf.Sin(2 * Mathf.PI * frequency * totalSampleIndex / sampleRate);
            totalSampleIndex++;
        }
       
        filledSound = true;
        if (filledSound != oldFilledSound)
        {
            Debug.Log("filledSound: " + filledSound);
            oldFilledSound = filledSound;
        }
    }
}