Accurately timed audio - Both OnAudioFilterRead and audio.PlayScheduled give irregular intervals

Unity Engine
1

Is there method that I can use with Unity to allow for accurate scheduling of audio events?

I have tried using a FixedUpdate method to play audio at regular intervals:

        int i = 0;

        void FixedUpdate()
        {
            if (++i % 50 == 0)
            {
                audio.PlayScheduled(AudioSettings.dspTime);
                i = 0;
            }
        }

In the same vein I have set FixedUpdate to be called every 1 ms using:

    Time.fixedDeltaTime = 0.001f;

but the playback of audio is still irregular.

I have also tried filling the sample buffer directly using OnAudioFilterRead:

        sample.GetData(sampleBuffer, 0);

        void OnAudioFilterRead(float[] data, int channels)
        {
            var buffer = emptyBuffer;

            if (++ii % 2 == 0)
            {
                buffer = sampleBuffer;
                ii = 0;
            }

            int numSamples = data.Length;

            for (int i = 0; i < numSamples; i += 2)
            {
                data[i] = buffer[i];
                data[i + 1] = buffer[i];
            }
        }

but OnAudioFilterRead does not appear to be called at regular intervals either.

Has anyone got any suggestions based on their experience?

2

If you want accurate scheduling, you usually want to schedule ahead, which is common to all FMOD based audio engines. Rather than wait until it’s time, once you know you need to play the next sound, compute the delay and then play the sound with that delay specified. That should give you very accurate results.

3

Thank you for your response. I have a number of questions.

How do I compute the delay in FixedUpdate when I know I need to trigger the sound?

Is this method accurate enough that sounds can be triggered at regular intervals which are 1ms apart?

Is there a good example you can point me in the direction of?

Apologies for all the questions, I am at the beginning of my quest to understanding the potential of audio within the Unity framework.

4

You don’t need fixed update, scheduling the audio to play at exactly the right time in the future is already a way of getting away from frame dependency. It also doesn’t restrict you to playing audio only at “regular intervals”, although you can obviously set it up to do so.

A simple example:
If you schedule the audio to play 0.5 seconds after you schedule it, then the timing will be accurate unless you drop belong 2 frames per second, at which point you would not be able to schedule the audio by the time it should have already played, because a frame is taking more than 0.5 seconds to run. (depending on the load settings of an audio clip, it can also take a little bit of time to initialize and be ready to play)

audio.PlayScheduled(AudioSettings.dspTime);

You’re not scheduling audio to play in the future, you’re scheduling it to play now, which is frame dependent,
and exactly what Play() does, which is not what you want.

5

But what if the events which trigger the playback of audio occur in the fixed update loop?

Isn’t that just increasing the latency further if I schedule the audio to play AFTER the fixed update event which I would like to be concurrent with the onset of the audio?

6

We can probably provide more useful advice if we knew more about what you are trying to accomplish.

In general, scheduling is good for chained sounds like dynamic play by play in a sports game. If you need to play audio based on player input, perhaps syncd to music, that’s very hard to do. Games like guitar hero cheat a lot, usually playing right sounds even as you screw up and only turning right sounds off after you’ve screwed up for a while.

7

Good point.

I would like to synchronize events which take place in the physics engine with corresponding audio events. The events may take place as little as 1 ms apart where:

Time.fixedDeltaTime= 0.001f;

From what you guys are suggesting, the only way this can be achieved is to predict when the interactions will take place in the future and schedule the audio accordingly.

8

Knocking the time step down to 1ms and using the FixedUpdate to try to get access to a faster game loop doesn’t work the way you think it works.

9

That would explain why the audio is neither concurrent or regular, thank you.

The question now is how, without introducing a large delay, can I have tightly synchronised audio which can be relied on for regularity?

The answer I have come up with so far is to predict the onset of events and schedule accordingly.

10

One thing from reading that thread that I noticed was:

Is there any way of accessing the audio thread directly without scheduling audio?

11

No. That’s all internal to FMOD.

12

This means that I am either going to have to predict the physics using a simplified deterministic simulation or delay the drawing of the screens somehow. I can’t store that many screens in a buffer, so deterministic is the only way forward.

Anyone got any solutions otherwise?

13

Hi,

I’m the author of the “Truth about FixedUpdate” post.

I’ve written a few posts on timing and synching already on the G-Audio support thread, here’s the gist of it:

You do not need to synchronize more accurately than frame rate( would be useless ), and cannot more accurately than the audio thread’s update frequency ( audio buffer size / outputSamplerate ). There’s only one way to lower latency, and that’s to use smaller audio buffers, which will increase overhead but should be fine on modern machines. Use SetDspBufferSize to achieve that. Try powers of 2, not lower than 128. I personally use 512 when I want low latency on iOS.

Also, don’t forget that synching does not need to be that accurate: what you hear happening 3m away from you takes
10 ms to reach your ears, and you’re not shocked by the lack of sync!

For musical use cases, it may be useful to synchronize playback of multiple sounds with sample level accuracy. This can be done with PlayScheduled. My framework, GAudio, also enables sample accurate, sub frame rate fading( as well as lots more! ).

Cheers,

Gregzo

14

You could also look at using a plugin to give you access to fmod and it lower latency drivers. I was able to get it working with asio and asio4all on windows and get <5ms accuracy for music stuff. Midi input from external controllers to trigger drums sounds live. Maybe overkill for your purposes.