If you want to create something based on an arbitrary music file (mp3, wav, ogg) you must analyse the sound spectrum with GetSpectrumData and check the volume of certain frequency ranges. Usually, the rhythm is marked by the bass guitar and the bass drum - both in a lower frequency range, about 20 to 200 Hz. You can detect peaks in this frequency range to sync something with the rhythm, for instance. You can also monitor other frequency ranges to add other events - as in the game you’ve linked at - YouTube
In some cases you must analyse the music with some anticipation in order to create objects ahead of the player that will be in sync when it reaches them (like in GuitarHero or in the referenced game). To do that, you should play and analyse the song in an object located far away from the listener, and play the music in a near object with the required delay: if you need 0.5 seconds of anticipation, for instance, start the music in the far object and 0.5 seconds after start it in the near object.
The function BandVol below calculates the volume of a given frequency range. You must call GetSpectrum to get a snapshot of the sound currently being played in the object’s AudioSource, then you can analyse several different frequency bands. This script must be attached to an object with AudioSource, and you must drag three objects (cubes, spheres, whatever) to bass, mid and treb in the Inspector. When the song is playing, the Y coordinate of each object will be proportional to the frequency range it monitors. You can extend this script to several bands, until you find the ones you want.
private var freqData: float[];
private var nSamples: int = 1024;
private var fMax: float;
function GetSpectrum(){
// get spectrum: freqData[n] = vol of frequency n * fMax / nSamples
audio.GetSpectrumData(freqData, 0, FFTWindow.BlackmanHarris);
}
function BandVol(fLow:float, fHigh:float): float {
fLow = Mathf.Clamp(fLow, 20, fMax); // limit low...
fHigh = Mathf.Clamp(fHigh, fLow, fMax); // and high frequencies
var n1: int = Mathf.Floor(fLow * nSamples / fMax);
var n2: int = Mathf.Floor(fHigh * nSamples / fMax);
var sum: float = 0;
// average the volumes of frequencies fLow to fHigh
for (var i=n1; i<=n2; i++){
// sum the relative amplitudes at each frequency multiplied by i (to
// compensate the typical "equal energy per octave" distribution of most sounds)
sum += freqData _* i;_
*}*
*return sum / (n2 - n1 + 1);*
*}*
*var bass: Transform;*
*var mid: Transform;*
*var treb: Transform;*
*var volume: float = 20;*
*private var yBass: float;*
*private var yMid: float;*
*private var yTreb: float;*
*function Start(){*
*// GetSpectrumData initialization*
*fMax = AudioSettings.outputSampleRate/2;*
*freqData = new float[nSamples];*
*// example initialization*
*yBass = bass.position.y;*
*yMid = mid.position.y;*
*yTreb = treb.position.y;*
*}*
*function Update(){*
*GetSpectrum();*
_bass.position.y = yBass+volume*BandVol(20, 200);_
_mid.position.y = yMid+volume*BandVol(300, 1800);_
_treb.position.y = yTreb+volume*BandVol(5000, 20000);_
*}*
**
Don’t expect to get too much precision - like detecting exactly which note is playing, for instance. Songs generally have lots of instruments and voices sounding together, which makes a precise pitch evaluation very hard - if not impossible at all.
NOTE: I didn’t test this whole script, so please let me know if you get any errors.
EDITED: Fixed some typo errors, and included a compensation for the typical “equal energy per octave” distribution of frequencies in most sounds - the harmonics are more spaced at higher frequencies, so the average value returned by BandVol is reduced at higher frequencies.
