I have a lot of audio which I need to load in each of my scenes. Each scene can easily contain a few hundred moderate length audio files, but it only ever needs one or two at a time. Precisely which clips are needed is not possible to predetermine, so I’ve dismissed AssetBundles as impractical. Resources.Load would be ideal, but it seems to load them into memory even if they’re marked as streaming.
So at the moment, I’m doing:
VoiceClip = Resources.Load(Voice) as AudioClip;
Resources.UnloadAsset(VoiceClip);
Amazingly, this actually works. Unloading the asset doesn’t make it unusable - it plays correctly when I attempt to play it, and the maximum audio memory usage is never higher than the size of the largest single clip.
This cannot be the most efficient way of doing this though, surely? There has got to be more efficient way to dynamically load streaming audio. If I referenced the audioclip in the inspector, I imagine that would achieve this precisely, so it must be possible to achieve the same effect. It’s just that I can’t reference the clips as they’re determined at runtime.
Yes, this is currently the way this has to be done. The reason is that AudioClips currently open and start prebuffering the stream as soon as they are created – that way the clips are able to play immediately without any delay and all relevant properties can be queried. This is actually a behavior we would like to change at some point, but it requires that we store more information in the asset, so that all the properties that can currently be queried still return correct information while the sound is still loading, and we may also need to add a callback or use AudioClip.isReadyToPlay to return true when loading has finished. The hard bit is obviously to make this change without breaking the current behavior wrt the scripting API.
As far as I know it’s not possible to load resources in the background (correct me if I’m wrong), but there are some alternatives to this that you could try out:
Use the WWW object to load the sound from the local asset folder (“file://”) then use GetAudioClip(false, true) to open it as a streamed audio clip (this is what the last argument does. You may also experiment with setting the last argument to false, in that case the WWW will load the file completely and can be played when AudioClip.isReadyToPlay is true, but the actual loading will still happen in the background.
Wrap your AudioClip into an asset bundle, as these can be loaded asynchroneously. Requires a bit more work and some scripting, but may be worth it anyway, if you need to process a large quantity of audio files.
Thanks for the reply. It would definitely be nice if the default behaviour was changed, but I can see that it would be difficult to change without breaking anything. As I know well, API changes which break existing code are incredibly frustrating, so I can see that you guys need to tread lightly with this.
The two suggestions seem promising. I didn’t really think of using asset bundles on a 1-to-1 basis with clips, but that does sound doable, since I can automate things with an editor script. The WWW object and the streaming assets folder sounds even better, since it would probably require no changes to the data I’ve already created. I’ll give them both a go and see whether either one looks as though it suits my needs perfectly.
I’ve tried implementing suggestion #1 but I can’t set streaming to true. If I set streaming to true, the start of the sound is skipped and we begin a fraction of a second into the audio. If I set the streaming flag to false, it’s fine. I am, of course, waiting for isReadyToPlay to be true. I’ve tried both OGG and WAV, but the results are always the same. I tested to see whether isReadyToPlay was returning true too soon, but it’s irrelevant. No matter how long I wait, setting the Streaming flag to true always causes the audio to skip the first half a second or so of the clip. This seems to be a bug in Unity 3.5? Is it solved in 4? I see nothing in the release notes to suggest that it is.
I’ve also had time to try your other suggestion now, but I’m stumped. I don’t see any way to access the content of the asset bundle before it’s fully loaded. If I can’t access the content of the bundle before it’s fully loaded, I can’t see any point in it. The whole point of streaming audio is to begin playing it before it’s fully loaded.
I’ve submitted a bug report with the streaming audio clip issue. In response, I got an email telling me that the list of bugs is huge and it probably won’t be fixed any time soon. If I’d like to pay $1,500 for a three month premium support subscription, they’ll take a look at and make sure it’s really a bug as a matter of priority. As far as I can see, I wouldn’t actually stand any better chance of getting it fixed, despite that $1,500 investment.
Is this now official Unity policy? Call me odd, but most people would be ashamed to admit their bug list was so large it was unmanageable. Actually turning that into a pitch for why I should pay another $1,500 for you to determine whether one of your major features you’ve been advertising for age is actually completely broken took me by surprise.
I emailed back for clarification if I had understood correctly, but have received no reply. So I guess you do have at least a little shame.
EDIT: In case it matters, the case number is 522051.
Streaming audio from asset bundles is not possible, only loading the entire file out of the bundle into memory and decompressing first. The code for streaming .ogg files from the StreamingAssets folder is included a few posts below.
Unfortunately Unity won’t work with fmod for integration. They really should. This form of audio engine is primitive and becomes more primitive every day.
Many thanks for the suggestion, Games Foundry. I’ve attempted to implement it, and it seems to be working fine, except that there is a huge pause before the first WWW request returns true for isDone when I build to a standalone. From the editor, it works fine. I’ve timed each section of the process: Loading the data, asynchronously loading the bundle and then streaming the audio from the bundle.
From the editor, the times are something like
0.05 seconds
0.2 seconds
0.05 seconds
From the Standalone Windows build, the times are something like:
2.2 seconds
0.2 seconds
0.05 seconds
I added code to check the progress each frame. Progress is zero every single frame until the frame it loads, when it ticks to one. Which seems to suggest that loading is not really taking place for the first two seconds. Very odd.
Note that this ONLY seems to happen on the first request. So there is a two second gap before the first line of a conversation begins playing, but all subsequent lines load at normal speed. Am I doing something wrong? Do I have to do something to initialize use of the WWW class or something?
I believe we witnessed this too. As a quick fix I preload all the AssetBundles but I haven’t gotten around to looking at the memory impact of doing this. I’m assuming that if all the audio files in the bundle are set to streaming, pre-loading the Asset Bundle should have minimal memory impact. But then you know what they say about assumptions. I’ve just been working on our AssetBundle code so it’s a perfect time for me to run some tests and post the results.
UPDATE 1:
So it looks like using this method ends up loading the entire audio file into memory and decompressing, even though it’s set as “stream from disc” and in ogg format within the bundle. Audio Memory jumps from a few hundred K to 29MB ( the size of a single uncompressed soundtrack file ) from an ogg file of approx 7MB. This explains the delay while the background thread loads and decompresses and entire file. Next I’ll try www.audioClip to see if that works.
UPDATE 2:
So I tried not using asset bundles and instead adding a .ogg file marked as “stream from disc” to the StreamingAssets folder and used www.audioClip. Alas, it still loads the entire file into memory.
UPDATE 3:
Tried audioSource.clip = www.GetAudioClip ( false, true ); and that does stream, taking 130ms before actually playing and adding about 350K to memory. However, it does mean your .ogg file is available to everyone in the StreamingAssets folder. It doesn’t look like it’s possible to stream from asset bundles so I’ve updated the original post.
I can also confirm the bug in clip.length for streaming clips, and audioSource.isPlaying not reseting to false when the clip has finished playing. The temporary workaround solution is to pass in the lengths manually. Strangely, pausing the editor causes isPlaying to reset.
if ( Application.platform == RuntimePlatform.IPhonePlayer )
filePath = "file:///" + Application.dataPath + "/Raw/";
else if ( Application.platform == RuntimePlatform.OSXPlayer )
filePath = "file:///" + Application.dataPath + "/Data/StreamingAssets/";
else
filePath = "file:///" + Application.dataPath + "/StreamingAssets/";
// The above can now be replaced with Application.streamingAssetsPath ( old code )
filePath += oggFilename; // including .ogg file extension
WWW www = new WWW ( filePath );
yield return www;
if ( www.error != null )
Debug.Log ( www.error );
audioSource.clip = www.GetAudioClip ( false, true );
if ( audioSource.clip != null )
{
if ( audioSource.clip.isReadyToPlay )
{
... continuation of original code
I was actually going to ask you whether loading assetbundles was actually streaming or not once I had it working properly, so thanks for answer that. Now that you’ve gone back to GetAudioClip(false, true) though, isn’t that exactly what I posted above? When I use GetAudioClip(false, true) the beginning of the sound gets cut off.
We’ve encountered a streaming sound having the first bit of it being skipped in Unity 4.0 as you described. Happens during cut scene dialogue for us, when streaming the 30th sentence audio file in the dialogue ( we play a sentence, then clean up both the audio clip and the www about 0.5s after; profiler shows everything behaving as expected ). The bug happens every time at exactly the same spot, no idea why. In our case we are able to work around it by combining audio files.
I’ve received an email from support telling me that the bug has been confirmed and telling me that they are now passing it over to be fixed. I don’t know if it’s going to be a fix for 3.x, 4.x or both, but it’s definitely good news.
Resources.Load on an AudioClip currently seems to always load/decompress the whole thing in to memory. The only work around I’ve found so far is to create a prefab for each AudioClip (which just contains a script which references the audio clip), and do the Resources.Load on this.
I’ve described it in this other thread, here !
I don’t quite understand when is the audio streamed and when not (provided I have selected StreamFromDisc as LoadType on my music).
It might not be entirely according to the thread title since I’m not using Resources folder now, but I guess you here will have the most info about it so I’ll ask here, hope it’s ok.
Let’s say I have a folder Music, with some, well, music… Pretty big files, pretty long, StreamFromDisc on them. In scene, I have an ‘Ambient’ object (simple object with AudioSource on it). If I simply assign my music file to this AudioSource, will it stream the music without loading it whole into memory?
Second case - I want to change the ambient music. I have some trigger for example, with script attached. To this script, I assign another music from the inspector (to an AudioClip field). When I enter the trigger, I assign this AudioClip to the audio.clip in my ‘Ambient’ object. In this case, when is the music loaded (StreamFromDisc selected, of course)? When the scene loads, the trigger is entered (-> the clip is assigned to the audioSource), or is it streamed without previous loading as it should be?
Thanks, I’m really confused about this loading behavior since many people write about bad loading even with StreamFromDisc selected, I’m just not sure if it’s always weired or only while using Resources.Load etc?
Edit: Btw I use Unity 3.5 Indie (+Android/iOS) if it matters
(i’m not a pro with unity and I don’t know this for a fact, this is just my opinion)
when you load a scene, the objects within the scene that have audio clips attached to them will decide whether or not to load the audio to ram. if the setting is ‘stream from disk’, no loading will occur until the moment of playback.
So when you run a script (your trigger) to change the audio.clip field, no audio is loaded, still, until the very moment you wish to PLAY that audio file, then the audio is streamed.
as long as you work with gameobjects that have the playback setting set to streamfromdisk, you can change that audio.clip as many times as you wish, and nothing will pass through the ram / cpu until you push play on that object.
this is how it typically works - but again, i’m no Unity guru, and the unity audio engine is quite terrible.
It’s such a shame that this issue has been ignored for so long. I’ve contacted support several times but the only responses I’ve had so far were to confirm that they haven’t fixed it and an offer that I could pay $1,500 USD to have them confirm it as a bug. (But not fix it). They have subsequently confirmed it as a bug despite me not paying the ransom. I tried contacting Graham, since he seems to be the main man on these issues, but I’ve had no reply at all from him.
All hope is not lost, however. Unity were announcing on Facebook that the IssueTracker votes are used to prioritize which bugs they fix. This seems like a very unwise way of prioritizing your resources, but at least now I know why the issue was being ignored. I didn’t even know I could vote for a bugfix, so it was on zero votes. A few kind Unity users on FB have already added some of their votes to it, so if you would like to get this issue resolved, please add all the votes you can spare here :
Maybe, with luck, we can get something done about this. Creating 10,000 prefabs doesn’t seem like a solution to me, as kind as it was of wickedworx to share that solution.
