Includes three comprehensive products to simplify lipsync, eye movement, and facial expressions:
SALSA LipSync
Simple Automated Lip Sync Approximation provides high quality, language-agnostic, lip sync approximation for your 2D and 3D characters. Audio dialog files are processed in real-time to automate the lip sync process. Results are superior to basic jaw bone animations, without labor intensive phoneme mapping.
Custom component inspectors provide code-free configurable options, while our comprehensive API provides accessibility within your custom code.
Supports Unity & Unity Pro. Tested on Windows, Mac, Linux, Android, IOS, and UWA. WebGL supported via our Amplitude product. See our requirements documentation for more information.
Some questions before I need to sign up for your forums:
When you say you support morph targets from Blender, Maya, Maya LT, Modo, etc is there a means of assigning the proper morph targets in the FBX model to your SW control or does you SW search and find the morph targets because they all the same names and identifying information?
I’ll have morph targets exported both from Blender and Maya LT. No problem?
Do you have pictures of how we should do our morph targets (shape keys / blend shapes) do have minimal fuss using your SW?
SALSA achieves quality results using four defined mouth positions: sayRest, saySmall, sayMedium, sayBig (these shapes are outlined in the forums and covered in detail in the manual). When you import your FBX with these BlendShapes into a Unity scene, Unity adds a Skinned Mesh Renderer component to you models GameObject. All of your BlendShapes are in this component. For 3D characters, you add our components to the same GameObject as the Skinned Mesh Renderer. SALSA lets you link to these four shapes by entering their index number into the corresponding index field in our custom inspector. These four indexes and an audio clip are all that’s required for SALSA to work.
RandomEyes works the same way, but uses five positions: lookUp, lookDown, lookLeft, lookRight, and blink.
As for morph target support, as long as you can see the BlendShapes in the Skinned Mesh Renderer component after importing into a Unity scene, then SALSA and RandomEyes can link to those indexes. The order of the indexes doesn’t matter, since you can set the index values in our custom inspectors.
Hello Goat!
Here’s a little more clarification on some of your questions. To minimize your workflow efforts with SALSA, the component will automatically assign indexes for shape keys as follows (matching the mouth positions identified in the screenshot of our previous reply):
Keep in mind, these automatic assignments are easily changed to match your model information. We’ll try to post up some more details and screenshots later today.
There also is no issue with using exports from Blender and MayaLT. They should work together just fine.
NOTE: When working with BlendShapes (at least with Blender), make sure to apply any modifiers prior to creating your BlendShapes. Additionally, modifying the mesh (adding/removing verticies) after creating your shapes, generally results in broken BlendShapes.
This does indeed look a lot lottt cheaper than the big name alternatives, like faceshift and such, and a lot less hassle - might not be accurate but 4 blendshapes instead of 1-2 dozen is so much easier and lighter and in this case, a hell of a lot cheaper. I’m planning to use daz/poser models as the art is getting excellent and very flexible, so i’m assuming i do my decimation in daz studio rather than whatever i’m importing it to. I suppose i could always test.
The 4 lip syncs is good if you are not learning lip reading. This is an old project on Blendswap that uses all the phenomes of speak (InfameDavid) or there is, I think it’s called targarela to base off of.
The other question I have need clarification for although I think you answered it above is when I make Blender models (I am still learning Maya) I often use subdivision to help guide me into using good topology however I leave subdivide unapplied when exporting as FBX because it makes the polygons too high. So to work with Salsa I will need to leave subdivision either turned off or delete that modified before exporting as an FBX.
In the future I’d like to see a whole gamut of emotions expressed such as the attached picture shows but honestly I’ll have to create all the blendshapes and see how big that would make the file. It might make it too big complex for mobile.
Hi Goat,
Yes, SALSA’s main goal is to provide great results with as little effort as possible. Making all of the phoneme shapes necessary for super-realistic mapping is very daunting and is not required of most game projects. We’re trying to keep it simple but awesome!
As for creating your BlendShapes in Blender. Our experience is all modifiers (that modify geometry) need to be applied prior to creating any BlendShapes. The reason being, BlendShapes are very fussy about vertex counts. Even one vertex add/subtract usually prohibits the ability to add or change existing shapes. And most likely will prevent the BlendShapes from working properly, if at all, once exported to Unity.
Emotions/expressions are something that we are toying with for the future.
Thanks. You are right, I forgot the power of ‘control-Z’. I’ve only created blend shapes with Blender so far and haven’t tried them in Unity so your advice is helpful. I look forward to your emotion extensions.
Hey adamz!
Currently, SALSA only supports up to 4 shapes. You can use fewer shapes, but the max is four.
As for the text-to-speech option (which is very cool, btw), the tts module would need to produce an actual audio object that could then be linked into SALSA. After taking a brief look at this tts solution, it doesn’t appear to do this; therefore it wouldn’t work as an integrated solution with SALSA in its current form. We’d have to take a look and see if it’s possible to link up with this sort of dynamic output. Obviously, using tts output files does work since there is an actual audio object to link to.
Hi lazygunn!
I’m only mildly familiar with the capabilities of Daz, but decimation is one of those processes that you could really use any tool to perform. The biggest considerations would be quality and output format. Blender certainly has decimation and re-topo features as do most full modeling packages. Of note to keep in mind is keeping vertex count/instance static while working with BlendShapes. While using Blender, any changes to vertex counts or instances (i.e. adding a vertex and removing another) will break or cause lots of problems with BlendShapes…pretty much creating a need to start over… So, creating BlendShapes should be the last step in the workflow pipeline.
lazygunn needs to know that DAZ FBX exporter isn’t exporting the morphs of all the old Poser CR2 files I licensed. I haven’t licensed all the DAZ owned content (as that’s another $500) so I haven’t tried Genesis morphs as an FBX export but I don’t think they are supported either. You need MegaFiers/MegaDAZ ($150/$30) to import the morphs. MegaDAZ does not yet support Poser CR2s as he is working on that support but it does support the newer DAZ Genesis format.
Blender does a good job of decimating DAZ/Poser characters too although if you’ll notice it’s the areas that have the morphs that tend to come out the worse in decimation (DAZ Decimator or Blender Decimator).
Process for the Genesis is documented at west-racing.com and on his YouTube channel.
Awesome goat!
Thanks for the update! Automated decimation is never a perfect science. It’s certainly getting better and better. For Blender, it is possible to take some control in the decimation process on where/what is decimated.
In edit mode, select the vertices that you want decimated, use CTRL+G to assign the selection to a group.
Add a Decimate modifier to the mesh and select the vertex group to apply the modifier to.
When the modifier is applied, it will operate only on the selected vertices.
I’ve updated the original post with some new overview information and images to help everyone get an idea of how the system works and what it’s requirements are. Bear in mind that this is still just an overview, the manuals and example code go much deeper.
goat, genesis models export their blendshapes just fine with studio’s fbx exporter, you just need to set up the morph export rules correctly, i havent triied the older poser formats yet. In the case of setting up the rules, for me it has been the case for me to set a match for anything based on the character you’re using (your base fbm) and perhaps anything else youre using as specifically as you can and it will export the genesis model and the items its wearing perfectly fine, the bottom rule ‘anything else’ will have everything not specified baked and not available as a blendshape
I did actually buy mega daz only a few days ago not realising i didn’t actually need it, i’ve contacted chris west and see if he’s amenable to a refund as it would greatly aid in buying another of his products that id find extremely useful.
You have to be careful what you specify as the rules in the morph export however, keeping it as minimal as possible, or the model can end up extremely heavy and will crash something like max, and sometimes screw up the rigging too. On the other hand it’s also been very amenable to putting on several layers of clothes to a figure and binding them all to the genesis skeleton in the exporter, which obviously makes the model heavier but means you can very simply swap out clothes as needed. If you wish to customise the entire model with blendshapes, you can, just add them as rules, but i’v found changing the absolute width or height of the figure will have it mismatch its skeleton
The export, i’ll add, is perfect for things like this btw, as you can supply the shapes asked for easily - but you can also export full sets of phonemes or visemes if you like. Main thing ive noticed that dont export properly (or at all) are mcms which help non-standard fbms express properly cause their dimensions are different humans
To prove my point, however, here’s the ribbit character in Unity, exported using the Daz Studio fbx exporter:
