Audio system

[cirocontinisio]

Thread for discussing the Audio System in the game. From the card:

A system that is able to play music and SFX on demand.
Characters, effects, events, can ask the audio system to play a sound once, repeated, and specify requested volume, pitch, location, etc.
The audio system could also have a few extras to do things like picking a random sound from a list and/or randomise pitch (for footsteps).
It acts as a centralised system to control volume. The settings modify the volume on the audio system, and the audio system outputs music and sfx with the correct volume each time.

Extra: start thinking of how this also integrates with the Timeline. Maybe just through a common AudioMixer?

Card on the roadmap

[Fuzzeh]

Hi There!

A while back, I opened a simple PR related to Audio. This was before this card became available.

It’s quite simple right now, but I think it at least provides a foundation in which we can expand on. Right now, it offers the following functionality:

• Play background music
• Play sound effects on demand
• Does not destroy the object on scene load. This allows the background music to continue to play as we transition scenes.
• Static access to make it easier to use
• Basic support for using AudioMixerGroup

[Neonage]

One question - Adaptive music?

[system]

I would like to work on Audio System, I have worked on something similar to this for Managing Audio.

https://open.codecks.io/unity-open-project-1/decks/15/card/16u-audio-system

[system]

What we can do is have 2 structs one for Theme and one for SFX
Theme will be decided according to which area or the Scene we are in.
And the SFX can be controlled according to which the events or functions called by the characters.

[system]

@Neonage

One question - Adaptive music?

Music design is done by the Unity Team as per Code Deck Roadman.
But if they decide to use it then it may be great.

[drod7425]

@Neonage

One question - Adaptive music?

Do you mean like having music in two or more parts, solo and full band (for example) and fading up/down full band/other parts based on current game intensity or some other parameter? Because that would be awesome.

[Neonage]

@drod7425

Do you mean like having music in two or more parts, solo and full band (for example) and fading up/down full band/other parts based on current game intensity or some other parameter? Because that would be awesome.

Yeah, we can have unique arrangements for each location or section of the game, and add new layers as the player is making progress. Also we can create seamless transitions between beach, town, forest or cage, all just by changing instruments of the main theme!

For direct examples, check out this beautiful video by GMTK that covers this topic:

https://www.youtube.com/watch?v=b0gvM4q2hdI

And let me know if were going for it! I’d love to help with making arrangement tools, and maybe even with some chunks of music (I’m an amateur composer, but it would be hell lot of experience)

[Zantis]

Well, the idea is obviously pretty nice. I also don’t see anything hindering us there. If i’m not mistaken, we should be able to easily achieve this by some triggers (invisible colliders), which would then change the music once the player enters a specific area (this would also work on any other condition, not just on collisions of course).

I also don’t see too much work from a programming side, as this should be solved by a simple fade-in fade-out of two soundtracks, right?

So as much as programming goes the AudioManager only needs one method like transitionMusic(newSong), which would then fade-out the old song and fade-in the new song. Though this might need a bit of tweaking on where to start the new song.

[koirat]

First of all create ScriptableObject that will hold reference to audio and never use bare AudioClip on your prefab/components. Only drag and drop AudioClipScriptableObject.
Naturally fields on components should be of type IAudioClipScriptableObject.

In my project I have something like this.

public abstract class IAudioClipScriptableObject : ScriptableObject{

    public AudioClip GetAudioClip() {
        return OnGetAudioClip();
    }

    protected abstract AudioClip OnGetAudioClip();
}

using UnityEngine;
using System.Collections;

[CreateAssetMenuAttribute(menuName = "Game/AudioClipScriptableObject")]
public class AudioClipScriptableObject : IAudioClipScriptableObject{
    public AudioClip audioClip;

    protected override AudioClip OnGetAudioClip() {
        return audioClip;
    }
}

[Zantis]

@koirat

First of all create ScriptableObject that will hold reference to audio and never use bare AudioClip on your prefab/components. Only drag and drop AudioClipScriptableObject.
Naturally fields on components should be of type IAudioClipScriptableObject.

In my project I have something like this.

public abstract class IAudioClipScriptableObject : ScriptableObject{

    public AudioClip GetAudioClip() {
        return OnGetAudioClip();
    }

    protected abstract AudioClip OnGetAudioClip();
}

using UnityEngine;
using System.Collections;

[CreateAssetMenuAttribute(menuName = "Game/AudioClipScriptableObject")]
public class AudioClipScriptableObject : IAudioClipScriptableObject{
    public AudioClip audioClip;

    protected override AudioClip OnGetAudioClip() {
        return audioClip;
    }
}

Hey koirat,

i would be interested in knowing

a) what you are getting at, as i can not connect this to anything within this discussion
b) why it is important to wrap the AudioClip in a ScriptableComponent

?

[koirat]

a) the topic is about audio system, why it is not connected ?

b) Flexibility.
You can always swap the content (even runtime).
You can extend ScriptableObject whenever needed, you could for example chose random AudioClip from a list.
You can create multiple AudioClipScriptableObject that refer to the same AudioClip, than just change some of them in the future.

In my project I put this class on a component that is going to spawn the sound. (some things are specific to my project).

[Serializable]
public class SoundInfoData{

    public IAudioClipScriptableObject audioClipObject;
   public float baseVolume=1;
    public float basePitch = 1;
    public float SoundGeneratorRadius=-1;
   public SoundPriority soundPriority;
   public bool isMusic;
   public bool ignorePause;
   public bool ignoreTimeScale;
   public AudioSourceType audioSourceType;
    public GameObject customAudioSourcePrefab;
}

//40 is arbitrary.
public enum SoundPriority{
  
   Mandatory = 0, //music || ambient
   VeryHigh = Mandatory + 40, // GUI || InfoSounds
   High = VeryHigh + 40, //Events
   Medium = High + 40, //Characters
   Low = Medium + 40, //Background
   VeryLow = Low + 40, //World
  
}

I definitely will change if possible

   public AudioSourceType audioSourceType; // <--my enum
    public GameObject customAudioSourcePrefab;

for a ScriptableObject in the future.

And damn it i just realized I have got SoundGeneratorRadius in PascalCase. I’m so angry right now.

[Yilos]

I opened a PR with a sound system implementing a sound manager that has functionality to modify, save and load volume of mixer groups, manage a pool of audio sources wrapped in a custom class so its easily extendable while being relatively simple.

Link to the PR:

Audio System

[drod7425]

Hah I was just working on almost that exact implementation last night. First impression is that it looks solid! I will try to leave a detailed review over my lunch break.

[Peter77]

@cirocontinisio

Thread for discussing the Audio System in the game.

In the video below I present my idea for the audio system. I’m going over how to integrate sounds and how to trigger those sounds through scripting, using the audio system prototype.

It’s a rather lengthy video, but I wanted to explain it in more detail to make sure it’s fully understood. I hope my explanation communicates the idea well enough, but you can let me know.

What I completely missed to mention in the video is that the AudioCue picks a random sound from the list that is then played. That’s why the list exists in the first place.

Please post constructive feedback what you think.

https://www.youtube.com/watch?v=vnjMy4Knfs0

Tagging Unity staff who seem active in the open projects forum @cirocontinisio @superpig @MileyUnity , hope you don’t mind.

PS: Sorry for bad English/accent

[Neonage]

Hmm, the idea is solid, but I’m not sure whats the benefit of using prefab for that.
May we have a link to your branch or PR?

[Peter77]

@Neonage

I’m not sure whats the benefit of using prefab for that.

I recorded a video where I try to explain it:

https://www.youtube.com/watch?v=0YmqSr8Ncqs

@Neonage

May we have a link to your branch or PR?

There is no branch or PR yet. I wanted to get the overall workflow-design approved first. Then I go and make the code “compliant” and push it.

[koirat]

Generally I’m using abstracted ScriptableObject as you can see in my example. (IAudioClipScriptableObject)
The reason for this is that It gives me flexibility. For example I can create ScriptableObject that will use prefab.

Additional when you use ScriptableObject you will be able to chose from a list when filing field on a component. With pure prefab you don’t have this functionality.

Now this IAudioClipScriptableObject only holds AudioCip.
I got different components to play them, there you can have more settings, as AudioSourceType etc.

[Peter77]

@koirat

Generally I’m using abstracted ScriptableObject as you can see in my example. (IAudioClipScriptableObject)
The reason for this is that It gives me flexibility. For example I can create ScriptableObject that will use prefab.

It’s a great idea to use an interface.

In order to use the interface to its actual potential, so that it doesn’t matter if the implementation is a ScriptableObject or Component, requires to expose the IAudioClipScriptableObject interface to the Inspector, rather than the type that implements that interface.

How do you make Unity 2019.4 serialize an interface? Using the SerializeReference attribute does expose it, but there is no UI:

public interface IAudioClipScriptableObject { }

public class NewBehaviourScript : MonoBehaviour
{
   [SerializeReference]
    public IAudioClipScriptableObject audioThingy;

Unity does show the field “audioThingy” in the Inspector, but it doesn’t allow me to pick anything. Do I need to implement a custom property drawer for it?

I would like Unity to display a “Selection window” for ScriptableObjects that implement IAudioClipScriptableObject, as well as Prefabs that have components that implement the IAudioClipScriptableObject. How would I do this?

Exposing the ScriptableObject that actually implements the interface, rather than using interface, unfortunately defeats the purpose to use an interface in the first place in my opinion.

Because then it’s again bound to a specific type, in this case the “abstract ScriptableObject”. Means if I implement the IAudioClipScriptableObject in a Component instead of a ScriptableObject, I would need to expose the component to the Inspector. So we would end up with different types and the user needs to understand when and where to use what. That’s not ideal in my opinion.

public class NewBehaviourScript : MonoBehaviour
{
    public AudioClipScriptableObject audioSO;

@koirat

Additional when you use ScriptableObject you will be able to chose from a list when filing field on a component. With pure prefab you don’t have this functionality.

That’s true. It reveals another weakness of the Unity editor that Unity Technologies should solve. The Open Projects initiative is a great opportunity for us to show Unity staff where the tech lacks. This is another prime example where functionality is missing.

@koirat

Now this IAudioClipScriptableObject only holds AudioCip. I got different components to play them, there you can have more settings, as AudioSourceType etc.

Can you provide examples of those different components that implement the IAudioClipScriptableObject interface? What is the AudioSourceType for?

[koirat]

For a little more take a look at few previous posts.
Since we cannot use interface we have to cheat it out with abstract class.

[INDENT][LIST=1]
[*]public abstract class IAudioClipScriptableObject : ScriptableObject{
[*]

[*]    public AudioClip GetAudioClip() {
[*]        return OnGetAudioClip();
[*]    }
[*]

[*]    protected abstract AudioClip OnGetAudioClip();
[*]}
[/LIST][/INDENT]

The system is more complicated, I got my SoundManager that plays SoundInfo object. Programmer almost never use AudioClip or AudioSource during programming. All this is hidden from him.

I use for example this component to play sound.

Sound Source - position of the sound.
Audio Clip Object - Audio clip that will be played (abstracted)
Base Volume - Base volume of the sound. This will let you simulate the volume of your sound. It will work like your AudioClip had lower volume if lower than 1.
Base Pitch - Same as base volume but for pitch.
Sound Generator Radius - As I said I’m using my own layer around unity3d audio system. In my system only sound inside a FOV of camera are played but sound got radius so if the sphere is intersecting with fov it will be played. -1 = infinite radius.
Sound Priority - Basically audio source priority but turned into ranges and “enumed” (look my previous post).
Is Music - is sound a music. Mostly for volume control/muting of music in game.
Ignore Pause - Sound is played when pause is turned off.
Ignore Time scale - Sound ignore scaling of time. (no sound distortion)
Audio Source Type - this were predefined AudioSource configurations (like long range / short range sounds etc.). But I will change it for ScriptableObject in a short future.
Custom Audio Source Prefab - when Audio Source Type set to custom will use this prefab to create AudioSource, but as i have said this will be changed for scriptable object in the future.
Loop - loop
Start At Random Position - sound start at random position, useful for some looping sounds, especially when started simultaneously.
Enable Disable Aware - When component disabled/enabled should sound stop/start.
Smooth Enable Disable - on enable disable sound should gradually increase/decrease it’s volume (no cutting hard, no starting out of the blue) (useful for sound transitions)
Smooth Dimnish/Amplify Speed - the speed of this volume change.