Dance Sync, animation loses the synchronization after some time

Hello :slight_smile:

I downloaded a Dance sequence Animation from Mixamo. I adjusted the Animation in Cinema 4d to match the bpm of a song. I imported both the song and the fbx(30 fps) to unity and let them Play together. They do match at the beggining, but after some time, the Animation starts “getting slower” and loses the beat of the song.

My Goal is to Play an Animation that during the whole song matches the beat, and achieve this by preaparing both data in advance.

My guess is that the Audio Plays constantly at the same rate, right? and the Animation Speed depends on the Frame rate of the game. BUT my Frame rate accroding to a script i downloaded from asset store is always between 50 and 60 fps…and the Animation accroding to unity is 30fps.

So am I right that the Animation Speed is not 100% reliable? If so how can i achieve my Goal?

i tested it without Frame reduction, with Frame reduction and optimal.

just to double check i opened the Animation with Maya and letting Play there in a Loop does match the song, so i dont know whats Happening inside unity that i can fix.

Before implementing an unorthodox solution(like restarting the Animation after some ammount of seconds) i wanted to ask you if my theories are right and if there is any solution to it.

Thank you very much.

using UnityEngine;
using System.Collections;

public class womanForward : MonoBehaviour
{
    //look at partner
    public Transform LookAt;
    public Animator anim;

    //react
    public KeyCode forward;
    public KeyCode backward;
    public KeyCode finish;

    bool keyB = false;
    bool keyFinish = false;
    private float PercB;

    private bool waitingForF=false;
    private bool waitingForB=false;

    public GameObject woman;
    private Vector3 startPos;
    private Vector3 endPos;

    public float distance = 0.5f;
    //time from start to end
    public float lerpTime = 2;
    //this will update Lerp time
    private float currentLerptime = 0;
    bool keyF = false;
    private float PercF;

    // Use this for initialization
    void Start()
    {
        startPos = woman.transform.position;
        endPos = woman.transform.position + Vector3.forward * distance;
    }

    // Update is called once per frame
    void Update()
    {
        anim.speed = 1.0f;//1.568f;

        if (Input.GetKeyDown(finish))
        {
            keyFinish = true;
        }
        if (keyFinish == true)
        { anim.SetBool("finish", true); }
        //transform.LookAt(LookAt);
        if (Input.GetKeyDown(forward))
        {
            keyF = true;
        }
        if (keyF == true && waitingForB == false)
        {
            currentLerptime += Time.deltaTime;
            if (currentLerptime >= lerpTime)
            {
                currentLerptime = lerpTime;
            }
             PercF = currentLerptime / lerpTime;

            woman.transform.position = Vector3.Lerp(startPos, endPos, PercF);
            anim.SetBool("forward", true);
            anim.SetBool("backward", false);

            if (woman.transform.position == endPos)
            {
                keyF = false;
                currentLerptime = 0;
                PercF = currentLerptime / lerpTime;
                waitingForB = true;
                waitingForF = false;
            }

            if (keyFinish == true)
            { anim.SetBool("finish", true); }
        }


        //Backward
        if (Input.GetKeyDown(backward))
        {
            keyB = true; Debug.Log("away1");
        }
        if (keyB == true && waitingForF == false)
        {
            currentLerptime += Time.deltaTime;
            if (currentLerptime >= lerpTime)
            {
                currentLerptime = lerpTime;
            }
            PercB = currentLerptime / lerpTime;
            woman.transform.position = Vector3.Lerp(endPos, startPos, PercB);
            Debug.Log("away2");
            anim.SetBool("backward", true);
            anim.SetBool("forward", false);
            anim.SetBool("idle_cancel", true);

            if (woman.transform.position == endPos)
            {
                keyB = false;
                currentLerptime = 0;
            }
            if (woman.transform.position == startPos)
            {
                keyB = false;
                currentLerptime = 0;
                PercB = currentLerptime / lerpTime;
                waitingForF = true;
                waitingForB = false;
            }

            if (keyFinish == true)
            { anim.SetBool("finish", true); }
        }
      
    }
}

there is also the possibility that i am calling the Animation in an inefficient way, so here is my Code (just a normal press key). i m aodin that here: { anim.SetBool(“finish”, true);

i just noticed the inspector of my Animation says: length 4,767…30 fps. but in fact…when playing it takes around 6 seconds :frowning:

Hi polkuj,

The desynchronization is expected, the audio is ticked with a DSP and most of the rest of the engine with the internal timer.

Right now the only way to make this work would be to manually tick your animator with audio DSP time.

So the audio become the master and the animator the audio slave

To make it work you will need to disable the animator, use PlayInFixedTime and then tick manually the animator

public class AnimatorTicker : MonoBehaviour
{
    private AudioSource audioSource;
    private Animator animator;

    void Start()
    {
        animator = GetComponent<Animator>();
        audioSource = GetComponent<AudioSource>();
        animator.enable = false;
    }

    void Update()
    {
        // Change time for currently played state
        animator.PlayInFixedTime(0, 0, audioSource.time);
        animator.Update(0.0f);
    }
}

So I just had a chat with our audio team leader

He proposed a different solution that will give you better result because audioSource.time is not as precise as the dspTime.

So basically he proposed to start your audio clip with PlayScheduled and save the time at which you start your audio clip as the dspStartTime.
And then at each update compute the diff between the current dspTime and the dspStartTime and use this diff to play your animation clip in sync with audio with PlayInFixedTime.

In case someone is facing the same issue,

I needed to fire Animation Events manually too.

public class AnimatorTicker : MonoBehaviour
{
    public AudioSource AudioSource;

    Animator animator;
    AnimatorStateInfo currentAnimatorStateInfo;

    double startTick;
    double deltaDSP, oldDeltaDSP;

    void Start()
    {
        animator = GetComponent<Animator>();
        animator.enabled = false;

        currentAnimatorStateInfo = animator.GetCurrentAnimatorStateInfo (0);
        startTick = AudioSettings.dspTime;

        AudioSource.PlayScheduled (startTick);
    }

    void Update()
    {
        if (currentAnimatorStateInfo.fullPathHash != animator.GetCurrentAnimatorStateInfo (0).fullPathHash)
        {
            currentAnimatorStateInfo = animator.GetCurrentAnimatorStateInfo (0);
            startTick = AudioSettings.dspTime;
            deltaDSP = 0;
        }
        oldDeltaDSP = deltaDSP;
        deltaDSP = AudioSettings.dspTime - startTick;

        // Change time for currently played state
        animator.PlayInFixedTime (0, 0, (float)(deltaDSP));
        animator.Update (0.0f);

        foreach (var clip in animator.GetCurrentAnimatorClipInfo(0))
        {
            foreach (var animEvent in clip.clip.events)
            {
                if(animEvent.time >= oldDeltaDSP && animEvent.time < deltaDSP)
                {
                    foreach(var j in gameObject.GetComponents(typeof(MonoBehaviour))) {
                        try {
                            var mi = j.GetType().GetMethod(animEvent.functionName);
                            if(mi != null) {
                                var pers = mi.GetParameters();
                                if(pers != null && pers.Length > 0) {
                                    if(pers.Length == 1) {
                                        if(pers[0].ParameterType == typeof(string)) {
                                            mi.Invoke(j, new object[] { animEvent.stringParameter });
                                            break;
                                        }
                                        else if(pers[0].ParameterType.IsByRef) {
                                            mi.Invoke(j, new object[] { animEvent.objectReferenceParameter });
                                            break;
                                        }
                                        else if(pers[0].ParameterType == typeof(float)) {
                                            mi.Invoke(j, new object[] { animEvent.floatParameter });
                                            break;
                                        }
                                        else if(pers[0].ParameterType == typeof(int)) {
                                            mi.Invoke(j, new object[] { animEvent.intParameter });
                                            break;
                                        }
                                    }
                                }
                                else {
                                    mi.Invoke(j, null);
                                    break;
                                }
                            }
                        }
                        catch(System.Reflection.AmbiguousMatchException e) {
                            Debug.LogError("Cannot find animation event method : " + animEvent.functionName);
                        }
                    }
                }
            }
        }
    }
}

credits : Animation events won't trigger if animation time set manually - Questions & Answers - Unity Discussions

yes in your case this is expected, as you are not increasing the time in the animator, but rather sample the clip at a precise time.
If you change your logic a little bit and manage to call
animator.Update(deltaDSP)
like this

 void Update()
    {
        if (currentAnimatorStateInfo.fullPathHash != animator.GetCurrentAnimatorStateInfo (0).fullPathHash)
        {
            currentAnimatorStateInfo = animator.GetCurrentAnimatorStateInfo (0);
            startTick = AudioSettings.dspTime;
            deltaDSP = 0;
           animator.PlayInFixedTime (0, 0, (float)(deltaDSP));
        }
        oldDeltaDSP = deltaDSP;
        deltaDSP = AudioSettings.dspTime - startTick;
        // Change time for currently played state
       
        animator.Update (deltaDSP );

the animation events should start to be fired again by the system.
The reason why it doesn’t work in your case is because we do fire all events that are in the range ]previousTime, currentTime] and since you are always reseting the time of the clip without providing a deltaTime your play range is always ]deltaDSP, deltaDSP]

1 Like

I tried the attached code,
Now the animation is playing very fast.

yes this is probably because you animator is enabled so the system also tick the animator.
When you take control of the animator timing you must also disable the animator component otherwise you will get exactly what you are getting.

Hmmm, strange because Animator is already disabled.
I will continue use the other script posted before, If you think it could be a bug, then let me know to test it on another scene and if so would report it.

Thanks,

Hello! I have a similar issue…

Have you found a solution to your issue?

Very old thread, but wanted to chip in for anyone who finds their way here and is having issues with Animation Events.

Mecanim-Dev’s code doesn’t work, and I don’t even want to think about the performance implications of the other solution to fire Events manually. My alternative: create empty objects containing a MonoBehaviour with “OnDisable()”, and disable them from the animation in time with where you’d have put your Events. You can also just re-enable them if you want the same event fired multiple times.

It’s such a cheesy workaround that I actually laughed out loud when I thought of it, but I had a cutscene synced to music that needed events to fire off in sync and it worked perfectly.