Compressed in memory sounds have unexpected memory bloat

Unity Engine
,
1

I’m capturing memory snapshots on an Android 5 device using the Memory Profiler package. One of the things that surprises me is that Vorbis sounds that are set to be “Compressed in Memory” end up being much larger than what it estimates in the Editor to “Imported Size.” For example, a sound that compresses to 314KB ends up taking 1.1MB as a native audio object. That’s more than 3 times greater! I tried MP3 compression instead and the sound ends up taking the same size in memory: 1.1MB

Are these sounds being uncompressed or changed into another format that the particular device is capable of playing natively? If this is true, for longer and larger audio clips, is there a format that is universally supported (such as ADPCM) that I can use to guarantee that the sound won’t bloat to such a large size? Assuming the sounds are being uncompressed in memory when the compression format isn’t supported natively, then it would be nice to be able to set the fallback format and sample rate, in the same way that you can select how Android ETC2 textures will fallback to 32-bit, 16-bit, or 32-bit (half resolution) when the device only supports GLES2.0

2

Following-up to this: As a test, I set the offending sound to ADPCM, which does not have as good quality or compression as Vorbis, but at least it’s not 1+MB size. As a 22KHz ADPCM, it now takes up around 408KB, which matches (within a few KB) the size given in the AudioClip inspector. This really emphasizes the need for some kind of selective ADPCM fallback option for large audio clips, since having these uncompressed in memory when we’ve selected a compression method is misleading. This seems to be a common misconception across the internet. Example from a Medium article:

Compressed in Memory means that the audio data will be loaded into the memory still compressed in the format you’ve defined, whether it’s ADPCM, Vorbis, MP3, or something else. Decompression and decoding happen only when the audio asset is played.

You might ask why not use streaming for large looping sounds like this? Well, because we’ve already got 2 streams going for our music and the CPUs on older devices simply cannot manage more than 1 or 2 streams.

So is this really the expected behavior for Vorbis Compressed In Memory audio?

3

Another update: I turned off “Preload Audio Data” for this sound and now it’s taking up the correct amount of memory (matching the Vorbis compression size shown in the AudioClip Inspector). Seems like a bug of some kind.

4

Hey there, thanks for the details.

Without being able to profile your project to confirm there are a couple of things that could be at play here:

  • On mobile devices, the destination format is MP3, even if you choose vorbis in the default import section (Sorry that is not clear), this because of the decode performance characteristics of MP3 on Android and iOS devices.

You can override this by choosing the android tab in the AudioClip importer and selecting “Override for Android”, then selecting vorbis explicitly, just be careful of performance implications.

  • The preload audio data thing sounds strange, it should not affect the ultimate loaded size of the clip, but if it does then thats a bug.

Let us know if the suggesting makes all the problems go away.
W

5

Thanks for the reply. :slight_smile:

I would have expected this actually because Unity would want audio hardware decoding if possible. However, assuming this would be true (that during the Android build process, sounds would get converted to Mp3 format in the bundle) then I would expect that if Default compression was set to Vorbis then if I don’t override the Android settings, the grayed-out box would say Mp3 instead of matching “Vorbis.”

This also doesn’t explain why the native memory usage as I’ve explained above was 1+MB, whereas Mp3 and Vorbis are pretty close to one another (in the 300-400KB range).

I’ve also read that certain Android devices aren’t licenced for Mp3 and so if you use this format then on those devices it will get converted to PCM. I’d love to get an official statement on this from Unity as there may be some misinformation out there from bloggers.

Yeah it seems strange to me too and might be related to the fact that some of these sounds have meta files from way back to Unity 2017 or maybe even Unity 5 days. So perhaps by changing the settings I was just forcing something to recompress or update and start honoring the settings. I’d need to do more testing to see if I can fall back to the 1+MB size again. If so I’ll submit a bug with an attached project.