Audio streaming corruption

Unity Engine
1

Hi,
I’m getting audio corruption when using UnityWebRequestMultimedia.GetAudioClip using an external https:// address to stream audio. This happens in Unity 2020 and 2021 LTS versions.

I believe it’s to do with buffer starvation as waiting for the full clip to download or a larger amount of downloadedBytes will fix my issue but that isn’t an ideal solution.

Is there any way to debug audio streaming or detect buffer starvation to pause / unpause the audio source?
It looks like you can’t create a custom download handler DownloadHandlerAudioClip as it’s a sealed class so perhaps there’s another way to do this?

Thanks in advance,
Neil

2

Huh? Is there any OTHER solution? If your network latency can be up to X seconds in any given context, I am not sure how you could successfully stream audio with a buffer shorter than X. But if you know a way… I’m all ears.

See if there’s any buffering settings on the downloading API you’re using. I haven’t streamed anything with UWR… it may or may not support changing its buffering heuristics, or you might need to make your own custom downloader that defers playing the stream until you have an adequate buffer.

3

when you say corruption, do you mean the audio file actually gets corrupted and cannot be played or the audio file starts to lose quality? because a spike in latency can skip samples depending on how you go about things, which sounds like static. or a decrease in bandwidth can possibly be telling your connecting server to decrease the bit-rate or sampling rate which would most likely be the cause of a corrupted file.

4

Hi, sorry not corruption of the actual file just in the playback. A spike in the latency is an interesting one actually and I wonder if I should put the downloading on a separate thread.

I can increase the buffer but that could still have issues if bandwidth is low and you still end up with not enough data being downloaded so I was wondering if there was a way with the existing DownloadHandlerAudioClip implementation to detect such things so I can pause / unpause the audio source similar to YouTube for example.

Perhaps I do need my own download handler but that is going to be tricky with supporting different audio file formats.

5

But also there’s lots of examples doing something like the following:

using (UnityWebRequest webRequest = UnityWebRequestMultimedia.GetAudioClip(fullPath, type)) {
((DownloadHandlerAudioClip) webRequest.downloadHandler).streamAudio = true;
yield return webRequest.SendWebRequest();
while (!webRequest.isNetworkError && !webRequest.isHttpError && webRequest.downloadedBytes <= 1000) {
   yield return null;
}
AudioClip clip = DownloadHandlerAudioClip.GetContent(webRequest);

But the yield return webRequest.SendWebRequest() waits until the whole of the downloaded bytes have been downloaded. Removing the yield makes it work properly but the I get the static / bad audio output.
That feels like a bug to me.

6

I’m still wondering if anyone has had any success with actual streaming audio in Unity.

I’ve looked into these packages:
Streaming Management | Unity Render Streaming | 3.1.0-exp.3 (unity3d.com)
Audio Streaming | WebRTC | 2.0.5-preview (unity3d.com)
but unfortunately I don’t think they are suitable as we are looking for a cross-platform, versatile streaming solution for wav, mp3 and ogg files with compatibility on Android, iOS, UWP, Mac and PC.
FMOD is an option but as Unity is built with the FMOD api, it feels like there should be solution in Unity.