[WIP] RhythmTool - Import and Analyze Music ingame realtime

What is RhythmTool?
It is a straightforward scripting package with all the basic functionality for creating rhythm games based on the player’s own music collection.

RhythmTool gives you the ability to import and analyze mp3 files (mainly music) at run time. This means that the player can select a song from his/her computer. Subsequently there will be data generated based on that song. You get to have the data for the song more than 10 seconds in advance.

Demo (stand alone) | Video

Looking for requests and tips
If there are any features you’d like to see, or if you have feedback regarding current features or design please tell me.

Features:

  • Import mp3 files at run time.

  • Real time analysis

  • Analyze songs based on your custom parameters.

  • Consistent beat- and onset detection

  • Works well on most genres.

Planned features
Because I am working on a rhythm game myself (that’s where this project originated), I am constantly testing and adding new ideas. These are some features I’m planning on adding in the future:

  • Wide range of data for visualisation
    Most of the data that is currently generated by analyzing a song, is aimed at use for gameplay elements.
  • More reliable detection for calm music
    Most music works quite well with the analyzer, but calm music gets too many false positives
  • System to save pre-calculated data
    pre-calculation allows for some extra analysis, but it takes time to do it over and over again for the same song.

Added a video:

I’d like to hear if people are interested.

I was thinking of making a rhythm based game from Android, and this looks very interesting! I’ve never used Unity before, even though I’d been meaning to try it out. So I was wondering, would your tool work for an Android app too?

Currently only partly. Analyzing songs should work fine, but the mp3 decoding library only works on Windows (and mac without too much effort).

I’m not that familiar with Android (or IOS for that matter), so I can’t say if it’s feasible. Eventually I want to get it to work on as much platforms as possible.

Thanks for anwsering. That’s too bad, it seemed just the sort of thing I needed now.

I don’t exactly know how you decode the mp3, but from what I’ve seen so far regarding beat detection, the analyzing part is the tricky one, so when you’ll decide to make it work with Android too it shouldn’t be too hard. From what I’ve read, at first Android’s API sucked when it came to analyze PCM data, but they eventually added a few classes that helped with that.

This is awesome its exactly what I’ve been looking for. I’d use this for every game, I dont know why more games dont sync events up and create more synaesthetic experiences… I dont know if your still working on this or what but…

Is there no way that this can be used with the free version of Unity(cant afford pro version)?

It’s already on the asset store, but I’m still working on it, to get it more fleshed out before I’m going to promote it further.

I want it to work on more platforms and non-pro first.

@KoningStoma I recently purchased Rhythm Tool. I love how it functions however I am experiencing an issue where all the inspector options on the Rhythm Tool object are not available to me (for example if I open the drop down of analysis nothing is displayed)

I am using a Mac and Unity Pro 4.3. Does it run better on later versions of Unity? If so I will update, however I’ve never seen any of these very nested inspector drop down menus, do I need a plugin to display said menus or how does it work? Are there any quick alterations I can make to it to make the inspector more visible?

Here is an image of my exact problem.

I have had the same issue. It appears to be an issue with some versions of Unity. It could have been 4.3 or 4.4. Maybe a more recent version won’t have the issue.

It’s something I am looking into. For now I’m using 4.0 myself and it works properly.

Hi! I’m not very good at programming, and just wanted to create a rhythm game that I choose the song ahead of time (player does not need to upload their own song), and an object will spawn during the beats (like guitar hero) Is it possible to do this with your plugin?

Yes, you can use an AudioClip instead of an MP3 file.

You’d get much better results with a system where you could manually assign when certain things happen along with the music. That’s how games like Guitar Hero do it. Something similar could be done with RhythmTool, but I don’t recommend it over a system where you can manually do things.

Here is the documentation, so you can get an idea of how it works and if you’ll be able to use it.

Please, Make it run on android, I have built in android and get error: libmpg123-0 not found! Thanks

What a great tool, just started playing with it for a current project!

We are wanting to play music on a looping basis and I’m not sure how that works with how the audio is analyzed. I did see that the audio gets stopped when the end of file is reached and for testing I commented that out. What I’m mostly noticing though is that when reading the onsets during Update or FixedUpdate that it does not see the onset for certain beats - the value is 0. Is there a way I can look in more detail at the data? The file format looks binary and didn’t seem particularly eyeball friendly. The audio loop is pretty straightforward, no unusual frequency shifts or other artifacts that I am aware of.

Hello,
first of all, thanks for a nice package.
I have a question. I also would like to have something happen just when a beat occurs.
This is how I’m trying to do it:

void OnReadyToPlay()
    {
        for (int i=0;i<rhythmTool.TotalFrames; i++){
            if (rhythmTool.IsBeat(i,5) == 1) {
                TimedBeat tb = new TimedBeat();
                tb.frame = i;
                tb.timeStamp = Time.time + rhythmTool.TimeSeconds(i);
                timedBeats.Push(tb);
               
            }
        }
       
        Debug.Log (timedBeats.Count);

        //Start playing the song
        rhythmTool.Play ();   
    }
   

    // Update is called once per frame
    void Update ()
    {       
        TimedBeat tb = null;
        // pop beats
        while(timedBeats.Count > 1 && timedBeats.Peek().timeStamp < Time.time) {
            tb  = timedBeats.Pop();
        }

        if (tb != null) {
        DoTheBeat();
        }
}

Basically I assign a timestamp to all beats and then in Update I test if any have happened and clear them from my stack.
Sadly I only get 8 beats (much less than expected) and if I check on PreCalculate it’s 0…
Can this approach work? And if, what am I doing wrong here?

Thanks in advance,

Seb