Good questions!
The speech recognition is setup to be phonetic so it works on other languages.
I don’t have exact specs for web cam. My one is a low end 1080 if that helps
I currently use Salsa Lipsync Suite and I’m pretty happy with it.
It gives really great results and matches any audio in realtime, so it’s great for multi-language setups, since you don’t have to prerecord the lipsync animation and will match any language. (although it’s true that your plugin should match phonemes more accurately)
But I like your plugin very much for the blendshape sculptor and webcam motion capture for the head animation.
I would like to use it along with Salsa but I don’t know how the two would work together.
It would be nice to have an integration path for the two, since Salsa seems to be the standard right now and it’s honestly working flawlessly for lipsync alone right now.
Something like using the webcam motion capture to record the movements of the head + triggering emotion blendshapes.
Edit: Does the webcam capture record the movement of each eyebrow independently or does it have the ability to recognize custom expressions to blend-in custom blendshapes? (like “happy”, “sad”, “surprised”… and blend them in-between)
I don’t have much need for the facial features right now but the blend shape sculpting is very intriguing. If it works the way I assume it does, it would be a huge time saver for tweaking existing morphs and creating new one without having to switch out of the editor. I’m actually surprised I haven’t seen a tool like that for Unity before. There are a lot of blend shape mixers and controllers but nothing for sculpting per se.
If you are planning to create any new videos, I’d like to see a demonstration of the blend shape sculpting and manager specifically. I understand the asset is new and has just been published, but more examples or documentation would be nice so I can see how this could work within my pipeline before purchasing it. (Also, you might want to create a link to this forum in the description so we don’t have to hunt it down.)
Puppet Face can be used to support other assets - so you could use it to improve your blend shapes using the sculpting feature, or you could use Puppet Face’s lip sync for your hero/ cutscene where you want the quality of the facial animation to be much better. You can use the performance capture to animate the head rotation, and it currently registers smiles so you can use it to animate the smile blend shape and use it in conjunction with the other lipsync. I’m hoping to track more expressions as I further develop this feature.
Agreed, I have actually submitted an update with links in the description, just waiting for Unity to make it live.
I should be making more tutorials going forward
After watching the tutorial video a few times I think I’ll go and give this a try now. Might even be able to use the head tracking and eye motion features to breathe extra life into my current assets.
I have some 3d models that were created without blendshapes as we weren’t sure we were going to need facial animation. Now that the project will require the characters to talk, we need to add blendshapes and are currently looking for someone to create them. I saw this tool and wondered if this would indeed allow us to take a character and create new blendshapes when none were created originally. The characters are realistic humanoids and your video shows a cartoony styled character so I wondered how well this worked on other styles.
I’m currently looking into why for some people the InitialPuppetFaceLaunchEditor isn’t running on import (this will cause it to error).
Its easy to fix by just re-importing this script. If you’ve experienced this please get into contact - I’m trying to figure out what the cause is so I can fix it in the first place.
Hi there
Is it possible to have eyebrow movements in performance capture?
I mean more control over face using this system
Maybe some emotion presets?
Angry
Happy
Sad
Etc?
