Making a Music Game in Unity3d/iPhone&iPad/android/etc

Target Platforms Mobile
1

Hello all! I’m developing a music game and chose unity3d iphone basic (plan on pro soon) to help me out.

I am at a roadblock. Unity currently only supports some functions like play stop sample, change pitch, change volumme etc.

I need to be able to do more to a sample - EQ, filter, delay reverb, etc.

I think the most important thing I need however, is a method to receive accurate and precise measuring of time or musical time.

Currently I’m doing the following (dont laugh).
update()
{
elapsedTime += Time.deltaTime;
currentBeat = elapsedTime*NUM_BEATS_IN_SEC;

if (currentBeat > beatArray[beatIndex] - cuePlayerTime)
{
CuePlayerToHitButton();
++beatIndex;
}
}

So I looked around the forums and stuff but a lot of the posts are over a year ago.

Whats the current status on using an audio interface? Unity 3 (coming soon) is supposed to come with Magic Audio or something. Hopefully this will solve all my issues.

If not, can’t I just find some audio sdk made in mono js or obj-c and include it in my project?

Thanks all! I’m sure many are as interested as I in hearing what has to be said on this issue.
Paul.

2

for this kind of lowlevel access and control you will need to use CoreAudio or at very least directly AudioSessions and alike through ObjC, you won’t be able to do it with unitys sound capabilities.

unity 3 with a deeper fmod integration than unity 2.x got some nice feature but part of them (which also are in your list) are desktop only from what I recall

and there is no such sdk to use unless you are interested in Unity Pro only and windows or mac only standalone
if any other platform, especially mobile, is of interest to you, you will have to implement it yourself

3

so i guess my next course of action is to pay for someone to do it for me as i haven’t done that before. what skills are required to build an audio sdk for unity3d iphone? would obj-c be enough? thanks

4

yes and some basic knowledge on unity and plugins in unity iphone advanced / unity iphone 3

5

I think you brought up two different topics, and in 3.0 you can do the first – the audio filters in 3.0 are fairly basic but flexible enough for a lot of uses, and at the moment (beta 5) include hi and lo pass filters, echo, distortion, reverb and chorus. And of course there are the reverb zones as well, though they’re probably less relevant in your case.

But Marc/Dreamora gives the best guidance on the second case. Precise beat-mapping still isn’t that workable, and the AudioSource.timeSamples property isn’t as helpful as it first appears for this, either. What’s really needed is a means of dealing with streaming samples outside of the context of a framerate altogether, plus low-level access to the data (responding to xm events might work well enough, for example, if you were using mod music instead of raw audio).

So when I faced this problem, I felt that digging into CoreAudio was the best way to tackle it. It doesn’t necessarily take a plugin, though; you can get surprisingly far with the PlayerPrefs/NSUserDefaults mechanism for this (I just did this for the “Lowriderz” iPhone game, in which the cars race while bouncing like lowriders in time with musical elements).

Unfortunately I found CoreAudio to be among the more convoluted of Apple’s API’s (it confused me, anyway), but I still agree with Marc that it’s the best place to look if you need precise beat-mapping. Here’s a link: http://bit.ly/bRbdd5

Best of luck with your project! I’m a fan of music games and look forward to seeing yours.

-Sean

6

hi all! thank you for your input. im still facing the same problem. i’ve continued development with my game and im almost complete minus audio manipulation. im very excited to have this released soon.

to handle the audio issue, i am using playerprefs to communicate with xcode in order to execute functionality using core audio. and yes, core audio is a VERY convoluted library!!! :frowning:

however, i would like to deploy my game to iOS and other platforms ie. android, etc

if i work out the problem using core audio i would have to find a similar solution for other platforms. not good.

i am going to have to make a custom library to use as a plugin within unity 3 pro.

any help would be greatly appreciated to help me create a custom audio library.

thanks.

7

ok so heres the deal: i’ve implemented my requirements using unity 3 pro’s audio engine. it handles all my needs and im pretty impressed! ie it handles pitch (including negative pitch for reverse play) manipulation pretty well, it allows for filtering effects, etc.

However, the engine itself is not designed for music games as it is not responsive enough for a music game. the solution is to use a custom library as a plugin or use core audio (making this game only available on iOS and not android, etc).

8

Probably too late but I’m doing something a little similar now and have come across your post. From the looks at it you have mapped out the beats in the song? (interested to know how you did this) - so your approach should work giving the play time and an array full of beats.

If you don’t have the beats mapped out then it gets a little trickier - I’m using the beats (BPM) to determine the dance style and speed. To work this out (which does NOT work when playing music from the users iTune library) I’m listening to the microphone and measuring the beats in real-time (which gives a reasonable gauge ). It works but obviously not perfect and using simple energy analysis which is where you compare the instant average (energy i.e. decibels) with a local history average (average of the samples taken from 1 second).

Good luck

9

Hey, interesting thread, i’m very interested in music games myself, I was wondering what the progress of this custom library/plugin is?