My WMR VR (openXR) Desktop Unity App - Speech commands don't work on Headset

Unity Engine
,
1

Hi All,
I’m having an issue with Unity/Microphone/WMR Portal. My VR app target is Standalone/Desktop using WMR and OpenXR. I’m using the UnityEngine.Windows.Speech namespace to create a Keyword Recognizer and allow me to control things in the app using voice commands.

It works perfectly fine when I’m just testing in the Unity editor without a WMR headset attached, but if I attach a WMR headset, the voice commands no longer work. It also doesn’t work after building and running the .exe.

I’ve verified that the headset microphone IS working and my voice is being picked up! Windows (I’m on Windows 11) shows in my privacy settings for my microphone, that my app IS using the microphone. I added a script that echos my voice and that works. But my voice commands are no longer being recognized at by the keyword recognizer.

Is there something about that namespace that it won’t run in Windows 11/Desktop WMR??

Hardware: HP Reverb G2

SW Versions:
Windows 11 Pro
Unity 2020.3.16
WMR Runtime 107.2109.10010
Packages Include:
XR Plugin Management 4.07
XR Interaction Toolkit 1.0.0-pre.5
OpenXR Plugin 1.2.8
Microsoft Plugins:
Mixed Reality OpenXR Plugin 1.0.2
Mixed Reality Input 0.9.2006
Microsoft Spatializer 1.0.196

2

Just tested with Oculus Quest 1 using Link cable and the voice commands work just fine with that setup. So this narrows it down to something with the WMR - Either the WMR Portal or the Mixed Reality OpenXR Runtime or perhaps with the UnityEngine.Windows.Speech namespace with is part of the UnityEngine.CoreModule.dll.

3

I’m changing this from Help Wanted to Bug

4

FYI - Tracking this issue here: https://github.com/microsoft/OpenXR-Unity-MixedReality-Samples/issues

5

We have discussed this with Microsoft and will continue to investigate further. Thank you for the report. If there are any updates I will let you know.

6

Hi @the_real_apoxol , Any updates on this? According to MaxWang-MS from Microsoft, this is a Unity bug. Don’t know where a bug report was filed though.
The GrammarRecognizer of UnityEngine.Windows.Speech seems not to work within Unity 2020.3.6f1 and MRTK 2.6.1. · Issue #9836 · microsoft/MixedRealityToolkit-Unity · GitHub
Just pinging to see if there was any update.

7

The best bet at this point is to submit a unity bug so it can get assigned to someone to look into. This is not an OpenXR bug so we need to make sure it gets routed to the right people to look at.

8

@JasonsFreeTime1 did you ever find a workaround for this bug?

I have time same issue with the HP Reverb G2 Omnicept Edition. The weird behavior is it only occurs, when the WMR detects the headset is on my head, else it works fine in play mode. I can even look through the lenses of the headset and use it. As soon I put it on it stops working.