Word Detection - Verbal Commands

sonicviz · October 16, 2012, 1:26am

Hi,
Is there a release notes as to what has changed in the 1.3 update?

ty!

theylovegames · October 16, 2012, 1:46am

I wanted to port back to Unity 3.4.2 to get access to publishing an Android Live Wallpaper. Unfortunately, I could only port back to Unity 3.5.0 which added the microphone interface.

1.3 is the same as 1.2 posted with Unity 3.5.0.

theylovegames · October 16, 2012, 3:17pm

Feel free to make feature requests. I’m trying different accuracy methods. I’m going to add another example where you push to talk when trying to match a profile.

theylovegames · October 20, 2012, 4:10pm

In the 1.4, with push to talk, the voice samples line up better which increases accuracy.
http://theylovegames.com/WordDetection_1_4.html

theylovegames · October 22, 2012, 8:24pm

Version 1.4 is now available, which adds a push to talk demo.

goat_1 · October 25, 2012, 9:57pm

How does this compare to the build-in Voice Command on iOS? Speech to Text on Android? Can we supply our own custom dictionaries? Is there a banned word list that it will ignore?

theylovegames · October 25, 2012, 10:31pm

The difference is this is a cross platform plugin. It’s not going to provide you with Text to Speech yet. It’s going to require users to record a dictionary of words that it will detect. Yes you can provide words to ignore.

The more users and feedback I get, I’ll be able to add more polish. Perhaps even some controls for editing the ignore list. Most of this is controlled via scripts. It’s almost time for more videos to explain the 6 example scenes in detail.

theylovegames · November 6, 2012, 6:32am

Just for the fun of it, another use of spectral analysis is to hook up an Emotiv headset and run it through the matching algorithm.

Technically it’s still word detection, it just comes from your mind!

http://emotiv.com/store/apps/applications/117/4021

Blinking is nice to move around. But I want to be able to think of a word and detect that.

theylovegames · November 8, 2012, 11:42pm

Here is a very interesting video about the history of speech recognition. The problem with the most advanced approach is that it requires a network connection to a server that has a massive database to find a match.

The advantage of my system, is that it doesn’t require a network connection or a large data set in order to detect the patterns from wave forms.

runner · November 12, 2012, 8:46pm

Have you run this thing through the profiler to measure overall performance? mainly the reason i ask would like to include but resources are rather tight with resources in my full game.

theylovegames · November 12, 2012, 8:56pm

In the demo examples on the mobile devices, the performance isn’t ideal because it’s drawing the textures to plot the graph. Disabling the plot would make the performance much better. I’ll put it on the todo list. In the meantime, you can remove any reference to textures in the example code.

runner · November 12, 2012, 9:27pm

okay it’s not mobile but pc, And can we have it so the push to talk is always held down in talk mode for verbal detection ? And meant from the previous post that performance between my game and this might just be too much for the asking.

theylovegames · November 12, 2012, 9:30pm

The difference between the last two example scenes is that one is talk mode, the other is push to talk.

runner · November 12, 2012, 9:36pm

1 last dumb question to ask Duh, Does it save Load the word list? Such as users of the game need to enter their words everytime they launch the game or is that something i will need to code up ?

theylovegames · November 13, 2012, 12:58am

Feel free to ask away.

Saving is not currently supported.

I’ll add saving profiles to the todo list. Currently it’s just binary data which could be converted to a base64 string and saved into player prefs. You could save in a device local storage. In standalone you could save the profiles into the user data. Or you could even publish the data to a server using WWW and a POST. The word details holds the profile information which just needs to be serialized and saved.

runner · November 17, 2012, 2:17pm

purchased yesterday

samples projects work and the word detection is pretty good at detecting words.
ummm the set button for setting word profiles is kind of confusing at first.

theylovegames · November 17, 2012, 9:04pm

Thanks for purchasing and welcome to our club. I’d like to see how users are using this package in their demos.

The Example scripts are purely examples. And you can always take those to customize and build your own UI with something fancier like NGUI.

There’s no reason why you need to show the spectrum data as a texture, as that was just an example.

Yeah I could always add a custom panel for easy configuration to see what’s going on.

Right now it’s modular while I get the algorithms polished.

@Todo:

Saving profiles
Make a clean Editor panel for setup
Performance hide the mic plot graphs by default and add a button toggle to get better performance on devices.
Create an android wallpaper that uses this package

theylovegames · November 27, 2012, 10:53pm

I’ll have to try MindWave Mobile for input and see if the verbal detection works. I’ll let you know if I can “think” the words in the example demos on my Nexus 10.

ina · December 1, 2012, 5:43am

curious about your results as well!

theylovegames · December 1, 2012, 5:34pm

A user wants to be able to detect drum beats and other instruments. Adding to the todo list.