How to play an animation on next beat of music

This might be weird to explain. I have a player character and I have audio that is at a specific bpm. Each beat has 8th notes in between beats. When I press my melee button, I want my character to trigger the melee animation but only on the exact beat of the music. So if I pressed the melee button in between the beats of the music, the animation would only trigger on the next beat. Is this possible? And how would I implement this? I am very new to unity so go easy on me.

Provided that you bring your own music to your game, Koreographer [and Pro] can do exactly what you describe. The workflow would look like this:

  • Open the Koreography Editor window.
  • Create a new Koreography asset.
  • Load your music into the Koreography asset.
  • Set the BPM of the music.
  • Create a Koreography Track asset.
  • Adjust its “Track Event ID” to something like “Melee”.
  • Set the “Divide beat by” option to 2 (this shows a grid of eight notes).
  • Click and drag the mouse over the waveform. You will see little red lines appear at each beat marker. These are “OneOff” events.
  • Click “File->Save Project”.
  • Add a “Koreographer” component to a GameObject your scene.
  • Add a “Simple Music Player” component to a GameObject in your scene.
  • Specify the Koreography you made in step 2 in the “Koreography” field of the “Simple Music Player” component’s inspector.
  • In your input-handling script, adjust the logic to set a boolean “bWantsPunch” flag that you set to true when the attack input is detected.
  • At the top of your input-handling script, add the following line:
using SonicBloom.Koreo;
  • Add a function to your input-handling script that looks like the following:
void CheckForMusicalMelee(KoreographyEvent evt)
{
    if (bWantsPunch)
    {
        DoMeleeAttack();
        bWantsPunch = false;
    }
}
  • In your input-handling script’s Start() method, add the following line:
Koreographer.Instance.RegisterForEvents("Melee", CheckForMusicalMelee);

You will notice that the first parameter to that RegisterForEvents function is the same one that you assigned to your Koreography Track’s “Track Event ID” in “Melee”. This tells Koreographer to call your “CheckForMusicalMelee” function whenever it encounters a Koreography Event. Those events are the red lines that you authored in step 8 above.

The workflow outlined above is very similar to what we show in this tutorial video.

Setting Koreographer aside, there isn’t really a way to do this in Unity without lots of your own coding work to handle music time, data authoring, event triggering, etc.

Hope this helps!

1 Like

You wouldnt happen to know how to get this to work with Corgi Engine would you? I am having a hard time finding out where to add this to get it to work.

Nevermind. I got it working like 5 minutes after I asked. I had to do some wizardry with their classes to get it to work. And now it works like a charm! Thanks for all the hard work youre doing!

1 Like

I take it back. It got it working to some extent. I will allow me to press a button and get it to sync to the beat, however, it will allow me to press the button when its off the beat before the next beat any time interval between beats. If I hit the buttons really fast, it will trigger on the beat and then not allow anything button to pressed until the next beat rolls around. But that means that if I dont press a button after an event is triggered and then wait right up to the next beat, it will allow me to trigger it off beat. Does that make sense?

I really need it to kind of force it to only allow button presses on the beat or rather, in a range tolerance area to where it still allows a button to only be pressed right up to beat or a tad afterward. Is this possible?

It sounds like what you need here is input gating - i.e. you need a separate set of logic to determine that the “player pressed the attack button!” is “close enough” to a beat to be “accepted”.

There’s a lot of nuance happening here. If you want your animations exactly synced to the music then they should always “start” at exactly the right position. If it’s okay to have them “almost exactly synced”, then you can simply trigger the start of the animation anytime your user actually successfully hits the attack button within your “acceptable window” around your target beat timing.

To keep things simple, I’ll explain a way to handle the “almost exactly synced” approach (the exactly synced one requires a bunch more nuanced state handling).

I assume that your KoreographyTrack is full of OneOff events. Those are instantaneous and will only trigger on exactly one frame. You have two options to [easily] open up a window around them:

  • The OneOff approach:

  • Select all OneOff events in the track and shift them a certain number of samples to the left. The amount of samples depends upon your target frame rate and your audio file’s native sample rate (e.g. for 60fps gameplay, 48kHz audio, and a “window” of 3 frames, you would want: 3 * 48000 / 60 = 2400 samples. This will push them “ahead” of the actual event you care about.

  • When the event triggers you either set a “frame count” variable to something like 5 frames (i.e. 3 ahead, 2 behind) or set a time window (e.g. 200ms) that you decrement (or decrease) every frame. As long as that value is positive, the input can be accepted.

  • The Span approach:

  • Select all OneOff events in the track and shift them a certain number of samples to the left. This is the same as sub-step 1 above.

  • Then convert all OneOff events into spans of a set number of samples in length. This would mean something like 5 * 48000 / 60 = 4000 samples for a “five frame window” using the math outlined in sub-step 1 above.

  • Span events trigger every frame. So whenever you receive a callback, you would simply set the “is accepting input” gate flag to true when:

bool isInputWindowValid = false;

void OnBeatEvent(KoreographyEvent evt, int sampleTime, int sampleDelta, DeltaSlice deltaSlice)
{
    isInputWindowValid = (evt.EndSample > sampleTime);
}

That simple code checks to see if the current frame’s sample position is “beyond” the end of the span. If it is, then we’re within the valid window as the Span is still ongoing.

Note that this will require that you use the “RegisterForEvents**WithTime**” event registration variant that gives you more detail as to what the state of audio is when the event is triggered.

Either way you go, you can use the tests outlined above to establish an “input acceptable” window of time to allow your player some flexibility for when the attack can hit. Then, if your player has input outside of that window, you can do with it as you please (combo break; sad sound; etc.).

Hope this helps!

Thanks for the quick reply. I like the simplicity of the Span Approach. I was actually think about this today as a possible way to fix my issue. I tried a similar way but the results were way worse. I tried using 2 OneOffs, 1 before the beat and 1 after the beat that would send an int that I could determine to turn off a flag that would allow a parameter of time to press the button. That did not work very well. Ill try your method and see if that works better.

Thanks again!

1 Like

While there could always have been something else going on, this method might not change anything if the separation of events was such that both would be triggered on the same frame. Just something to keep in mind

For my testing on using the Span method, that seemed to do the trick. Since its triggering every frame, it made it more accurate as far as how it sets the bool flag to determine when the player can attack. I think it will work quite nicely. Thanks for the help Eric!

1 Like