Dynamic music: Possible with Unity 3D?

Unity Engine
1

Hello,
I’m a musician who’s a complete programming nerd but I’ve started to get very interested in making some simple games.
I’m currently using the extremely simple Game Maker software, which I’m quite enjoying, but I’m very aware of its limits.

One of these (somewhat) limits is in terms of dynamic music. That is to say, music that change depending on what’s happening.

By this, I don’t mean, “at x point the music changes to a different point”, rather, for example: “if character gets hurt, remove an instrument from the currently playing son”.

For example, let’s say this is a Super Mario clone. The default state could have a regular song with drums, piano and guitar. If you grab a mushroom, a new track, the violin for example, would be added as a counter melody to the guitar’s part. Then, let’s say you lose your mushroom and you’re just about to die, then maybe all that is left is a thumping bass drum. The song and rhythm remain the same throughout, but the instruments playing the song change based on certain things that the player does.

With Game Maker, I can very clumsily make this happen.

As far as I can tell, Game Maker only recognizes individual sound files. So if you wanted to add an instrument, you’d have to swap out your current song with the same song but with more/less instruments playings. The obvious problem here being that you would lose your place in the song and it would have to start again from the top. What I would want is for the new instrument to start playing/drop out at the end of the bar.

The one solution I have (somewhat) come up with, and its not pretty, would be:
HUNDREDS of small audio files. Each one containing a bar of music, everyone different since they all contain different amounts of instruments.
For example: Track 1 = First bar with bass drum alone. Track 2 = First bar with bass drum + piano. Track 3 = First bar with bass drum + piano + guitar. etc. Track 4 = Second bar with bass drum. Track 5 = second bar with bass drum + piano etc. etc. etc.

This would allow you to create variables such as "if collision with badguy, and you are using track 2 (1st bar; piano + drums), then switch to track 4 (2nd bar; drums). In other words, you would maybe get hurt during the first bar of the song and by the time you get to the second bar it will have dropped from two instruments to just one.

This, as you can see, is a real headache and would create thousands of song files within a game.
I’ll admit I do not know how to use Unity 3D yet… I’m mostly “browsing” around the different game engines to see if this is possible on any of them. From what I’ve read, Unity 3D is a fantastic engine which is (relatively) straightforward to use.
That said, I realize my current query is rather technical, so maybe this isn’t possible with Unity 3D – and if not, is there a game engine you could recommend for doing something similarly?

ideally, I wouldn’t have countless small song files, but rather larger ones that I could tell to remove/add instruments with certain variables (sorry, I only know the Game Maker jargon – not Unity’s!!).

Thanks for your help. I’m really interested in this game engine and I hope this first question wasn’t too long!

2

Simplest way: if you have, say, 4 instruments in the song, make each instrument into its own track. Basically you have the song recorded 4 times, and if you play all tracks simultaneously, you have the complete song. So at the start of the game, you play all tracks at the same time, and mute/un-mute each one as desired. Pro: dead simple. Con: takes a lot more memory.

Hardest way: make your own mod player. Pros: total dynamic control, uses little memory. Cons: quite a bit of programming, less “real” sounding depending on the type of music you’re doing, OpenAL (used by Unity for sound) doesn’t handle pitch changes properly on all machines.

–Eric

3

brilliant. Thanks a lot for the quick, helpful reply! This is much more obvious/logical than my complicated affair :stuck_out_tongue:

I do have a couple follow up questions however.
First of all.

  1. What are the chances of song files not syncing up? Obviously, it would be imperative that all the music files start exactly at the same time, or else it would make a complete mess. Is this easy to do? (again, complete Unity 3D newb)

  2. You mention that this would take a lot of memory. Is running 4-6 song files simultaneously that power-hungry?

  3. Would it be easy to tell Unity to mute/unmute tracks when specific things happen? Can you relate your character’s health bar status, for example, to the amount of tracks through scripting? How about triggering new instruments on your tracks by arriving at x spot? Is this all easy to program?

Again, thanks for the reply. :slight_smile:

4

Yes, it’s trivial to start them all at the same time; it would be harder not to. There’s no guarantee the tracks would stay in sync indefinitely, but in practice I doubt there would be any drift; if something delayed one it should delay them all, I would think. Although just to be on the safe side, I’d re-start all the tracks manually when the song needs to loop, instead of relying on the loop property. That way if they do get out of sync for some reason, it wouldn’t be permanent.

It would take 4-6 times the space, obviously, but as far as CPU usage, naturally it would use more than if you just had 1 track, but nothing particularly noticeable on a modern system.

Also, you might be able to get away with making each track mono rather than stereo, and then manually positioning each track spatially in Unity for a stereo effect (as long as at least a couple tracks are playing anyway), which would then use half the RAM.

Yes, that’s simple.

–Eric

5

Ya dark basic is really limited.

6

@ Eric5h5:

Great, thanks a lot. I look forward to using Unity now! I’ve got a lot of reading to do to be able to make much use of it first…

@ asterix:

…what’s dark basic?

7

The way I do this (thanks to Eric) is to use InvokeRepeating at the same interval as the sample loop length, i.e.

var clipList : AudioClip[]; //array of audio clips of the same length
var audioID : int;

function Start() {
InvokeRepeating("clipPlayer",.1,4.789); //4.789 is base loop length
}

function clipPlayer() {

audio.clip = clipList[audioID];
audio.Play();

}

for multiple tracks, you could externalize the InvokeRepeating into a separate script that sends messages to the player functions in the separate audio objects. For longer tracks you can use AudioSource.time to know exactly what the playback time of the base track is, and send messages based on that, which I’ve found useful for lipsyncing.

8

Ok thanks for the help guys.

Cookies all around!!

9

Thank you a lot for posting this thread, i would like to put dynamic sounds too in my game, and i find this very helpfull.There is only one question that i would like to clarify:

How can i play for example 6 audio clips at the same moment in the same Audio Listener ?

As you know, we can only have 1 Audio Listener in the scene, and i never find how to play more then 1 song in it. (Mabe in unity 3 is possible, but i steel use 2.6.1 :frowning: ) Any solution will be Welcome :smile:

10

you don’t play audioclips in the audio listener. it just listens to it (its your “position for listening”)

the playing happens through the Audio components you add on game objects and the audioclips you play

11

Ah here was my wrong ! I was not understanding the interraction of each and i was allways putting my level music in the same Game Object then the Audio Listener (was the only way i knew for play the music in stereo…) and the solution is just to uncheck the “3D Sound” in the Audio Importer of the music, then the music will play in Stereo no matter where she is in the scene… thank you @dreamora !

12

putting the audio component in the same gameobject is indeed a good way to especially get around the distance falloff or stereo balancing :slight_smile:
For game music I would definitely use an audio component on the audiolistener object (or a child of it at the same position) :slight_smile:

13

game music trax you can just have the clip not use 3d sound.

14

I wrote a blog post about doing dynamic music a while ago. Adding it here for future reference: Dynamic Music in Unity3D - PreviewLabs

You don’t actually need to keep all music tracks looping and in memory, you can load/unload them. The key is setting the timeSamples value to the right value before playing, so it start playing exactly in sync with other loops.