I just downloaded the IBM Watson SDK asset from the Asset Store today, and am keen to get speech-to-text working in even a demo app. But after a couple of hours with it, no luck.
I know I have my cloud service set up, and the credentials correct, because I tried the simple Curl examples here, and they work.
But the only example I could find in the asset (ServiceExample scene, ExampleSpeechToText asset) does not appear to do streamed recognition. It does run, and after much time appears to have done whatever it set out to do (except for a couple of errors because I have the Lite service), but I poked through the code and don’t see it using StartListening.
So, by crawling through the SpeechToText.cs source file, I attempted to hack out my own script that would do streaming recognition:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using IBM.Watson.DeveloperCloud.Services.SpeechToText.v1;
using IBM.Watson.DeveloperCloud.Logging;
using IBM.Watson.DeveloperCloud.Utilities;
using IBM.Watson.DeveloperCloud.Connection;
using IBM.Watson.DeveloperCloud.DataTypes;
public class WatsonListenTest : MonoBehaviour {
[Header("Service Credentials")]
public string username;
public string password;
public string url;
[Header("Debug Stuff")]
public Text statusText;
SpeechToText speechToText;
AudioClip recording;
void Start () {
LogSystem.InstallDefaultReactors();
UnityObjectUtil.StartDestroyQueue();
// Create credential and instantiate service
Credentials credentials = new Credentials(username, password, url);
speechToText = new SpeechToText(credentials);
speechToText.OnError = OnError;
speechToText.StreamMultipart = true; // use Transfer-Encoding: chunked since we are sending multiple chunks to stream
speechToText.DetectSilence = false; // for now!
if (!speechToText.StartListening(OnRecognize)) {
Debug.LogWarning("StartListening returned false");
} else {
Debug.Log("StartListening OK");
}
Log.Status("Start()", "Checking whether Watson's logging system works");
}
void Update () {
string status = "";
if (speechToText.AudioSent) status += "Audio Sent; ";
if (speechToText.IsListening) status += "Listening";
else status += "Not listening";
statusText.text = status;
if (Input.GetKeyDown(KeyCode.LeftShift)) {
Debug.Log("Recording (5 seconds)");
recording = Microphone.Start(null, false, 5, 44100);
}
if (Input.GetKeyUp(KeyCode.LeftShift)) {
Microphone.End(null);
AudioSource audioSrc = GetComponent<AudioSource>();
if (audioSrc != null) {
Debug.Log("Playing recorded clip");
audioSrc.clip = recording;
audioSrc.Play();
}
Debug.Log("Analyzing clip");
float[] samples = new float[recording.samples * recording.channels];
recording.GetData(samples, 0);
float max = 0;
foreach (float sample in samples) if (sample > max) max = sample;
Debug.Log(samples.Length.ToString() + " samples, max = " + max);
Debug.Log("sending clip to Watson");
var data = new AudioData(recording, max);
bool result = speechToText.OnListen(data);
Debug.Log("OnListen returned " + result);
// speechToText.StopListening();
}
if (Input.GetKeyDown(KeyCode.Return)) {
Debug.Log("Stopping listening");
if (!speechToText.StopListening()) Debug.Log("StopListening returned false");
}
}
void OnError(string error) {
Debug.LogWarning("Watson error: " + error);
}
void OnRecognize(SpeechRecognitionEvent results) {
Debug.Log("Something recognized! " + results);
}
}
But it doesn’t work. Because I have it set up to also play each recorded clip via an AudioSource, I know that the recordings are fine. And from the debug logs (I also hacked some additional debug output into SpeechToText.cs), I can see that it’s sending data to the server. But the only response I ever get from the server (i.e., the only time OnListenMessage is invoked) is with a “state: listening” message. I never get any results. (And so of course my OnRecognize callback is never called.)
This is true apparently no matter how many chunks I send, or how long I wait in between. I’m saying very simple things like “Hello,” “one, two”, etc. I must be doing something wrong, but I swear by this point I’ve looked at every method in SpeechToText, and I can’t figure out what it is.
It doesn’t help that the asset comes with no readable docs… some of them appear to be in Open Office format, and I don’t know what the heck a .shfbproj is. Can we get a simple PDF or HTML file please?
All very frustrating. Does anybody have a simple example of continuous recognition with this SDK?
Please disregard the highlighted part. Thank you.
