HI guys, I wanted to make real time animation sequence for characters and i don’t know ho to implement audio to that animation, like is there a way to bake audio into animation or do we have to play animation and dialogs at perfectly synced timing? or is there any other way to implement this.
Thanks in advance.
Well from what I know, the easiest way is to produce it instead - most times that your character models mouth is moving, is during a cut scene, so instead of exporting as FBX, youd export as a MP4, and play the file instead of an animation - so audio would be embeded and synced up in your 3D software.
If its some in-game experience that some event causes the characters to talk outside of a cut scene for some reason, and you want to match the models mouth up with the audio, youd probably have to do a similar idea, where you bring the audio into your 3D software and sync it there, then export it as FBX with audio, and if that fails, you would essentially have to try and be as accurate as you can with the Unity Anmation window and your audio file…
If you dont need the animation to be realistic, you could try a “frequency” method which youd have to look into separately, the same way youd make a “visual music player” in Unity, or detect “speech to text” essentially, based on the frequency of the audio file, determines how low to animate the jaw dropping and raising between each frequency of certain ranges.
There is one other solution, but I cant seem to think of it right now, and I know its fairly complicated, as well.