XR Device Simulator does not work properly

Nyox · September 20, 2022, 6:46pm

Hi !
I’m using Unity 2021.3.10f1 LTS
I’m using OpenXR
I’m following this tutorial (Unity VR Development) : 1.1 - VR Project Setup - Unity Learn

I’have set up my XR Device simulator inside unity.
When i hit play i can only move the headset, all rotation and translation work with the headset but nothing seem to work with left hand and right hand. I can’t rotate move and rotate hands. i have tried looking for other bindings thant the natural [T] and [Y] or [Left-Shift] and [Spacebar] but it does not work.

Expected behavior : In play mode i should be able to move hand by pressing [Y] or [T]

TLDR : i can’t move my hand with XR Device simulator in play mode.

Nyox · September 20, 2022, 8:53pm

Well i don’t know why but the reason seem to come from using OpenXR.
I switch for Oculus (On windows platform in Edit > Project Settings > XR Plugin Management) and it’s work fine !

chris-massie · September 29, 2022, 8:55pm

We have a fix coming in an upcoming 2.2.0-pre.1 version that updates the XR Device Simulator to automatically disable other HMD and controller devices that are attached so that they don’t conflict with the virtual devices that are added by the simulator to the Input System.

Before that version, like you said, you will need to modify your Edit > Project Settings > XR Plugin Management settings to disable the plug-in providers that are causing a conflict with the simulator.

unity_t3wnluhJrK3Ptg · November 17, 2022, 12:31am

Hey! Tried 2.2.0 and it’s still not working. Disabling XR plug-ins helped though! I wasted 3 hours to find this suggestion. If it’s hard to fix could you please add a warning or something on Simulator component docs please? Meaning https://docs.unity3d.com/Packages/com.unity.xr.interaction.toolkit@2.2/manual/xr-device-simulator.html

AXGame5 · December 4, 2022, 11:44am

Same issue for me, 2.2 does not fix it, disabling openxr does not fix it. I can control the camera via the the inputs but cant move either hand

chris-massie · December 6, 2022, 9:49pm

If you are on version 1.4 of Input System, try updating to version 1.4.4 or above by opening Window > Package Manager.

The simulator only works with the XR Controller (Action-based) components (ActionBasedController.cs), not the XR Controller (Device-based) components (XRController.cs), so if your interactors are using the latter it won’t work.

Do the controllers still not simulate if you uncheck all plug-in providers in Edit > Project Settings > XR Plug-in Management?

For debugging, you should be able to see three devices in Window > Analysis > Input Debugger under the Devices section. Double-clicking on any should show the simulated input values.

XRSimulatedController - LeftHand (LeftHand)
XRSimulatedController - RightHand (RightHand)
XRSimulatedHMD

Under the Actions section in that same window, you should also see your resolved bindings for the input actions used by your XR Controller (Action-based) components. If you don’t see any bindings under the position and rotation actions, you may not have enabled the actions referenced by those components. If that is the case, add an Input Action Manager component to an enabled GameObject in your scene and add a reference to the Input Action Asset you are using.

AXGame5 · December 7, 2022, 4:51pm

I believe something went wrong on the install since I tried it in a a new project to see if I can port the current project to it. I made a new blank install and now it works perfectly. I think I probably messed something up when trying to reinstalling packages to fix a differen XR issue I had.
So it works great for me now and it’s a big help!

JackKalish · April 17, 2023, 8:33pm

Solution for me was to go to XR Plug-in Management under the Project Settings, and check the Oculus box. After this, the controllers were working, but I started getting some runtime errors:

NullReferenceException: Object reference not set to an instance of an object
UnityEditor.SceneView.CallOnPreSceneGUI () (at /Users/bokken/buildslave/unity/build/Editor/Mono/SceneView/SceneView.cs:3690)
UnityEditor.SceneView.DoOnPreSceneGUICallbacks (UnityEngine.Rect cameraRect) (at /Users/bokken/buildslave/unity/build/Editor/Mono/SceneView/SceneView.cs:2248)
UnityEditor.SceneView.DoOnGUI () (at /Users/bokken/buildslave/unity/build/Editor/Mono/SceneView/SceneView.cs:2330)
UnityEditor.SceneView.OnSceneGUI () (at /Users/bokken/buildslave/unity/build/Editor/Mono/SceneView/SceneView.cs:2255)
UnityEngine.UIElements.IMGUIContainer.DoOnGUI (UnityEngine.Event evt, UnityEngine.Matrix4x4 parentTransform, UnityEngine.Rect clippingRect, System.Boolean isComputingLayout, UnityEngine.Rect layoutSize, System.Action onGUIHandler, System.Boolean canAffectFocus) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/IMGUIContainer.cs:351)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

GUI Error: You are pushing more GUIClips than you are popping. Make sure they are balanced.
UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

NullReferenceException: Object reference not set to an instance of an object
UnityEditor.SceneView.CallOnPreSceneGUI () (at /Users/bokken/buildslave/unity/build/Editor/Mono/SceneView/SceneView.cs:3690)
UnityEditor.SceneView.DoOnPreSceneGUICallbacks (UnityEngine.Rect cameraRect) (at /Users/bokken/buildslave/unity/build/Editor/Mono/SceneView/SceneView.cs:2248)
UnityEditor.SceneView.DoOnGUI () (at /Users/bokken/buildslave/unity/build/Editor/Mono/SceneView/SceneView.cs:2330)
UnityEditor.SceneView.OnSceneGUI () (at /Users/bokken/buildslave/unity/build/Editor/Mono/SceneView/SceneView.cs:2255)
UnityEngine.UIElements.IMGUIContainer.DoOnGUI (UnityEngine.Event evt, UnityEngine.Matrix4x4 parentTransform, UnityEngine.Rect clippingRect, System.Boolean isComputingLayout, UnityEngine.Rect layoutSize, System.Action onGUIHandler, System.Boolean canAffectFocus) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/IMGUIContainer.cs:372)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, UnityEngine.Matrix4x4 worldTransform, UnityEngine.Rect clippingRect, System.Action onGUIHandler, System.Boolean canAffectFocus) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/IMGUIContainer.cs:693)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, System.Action onGUIHandler, System.Boolean canAffectFocus) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/IMGUIContainer.cs:665)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, System.Boolean canAffectFocus) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/IMGUIContainer.cs:658)
UnityEngine.UIElements.IMGUIContainer.SendEventToIMGUIRaw (UnityEngine.UIElements.EventBase evt, System.Boolean canAffectFocus, System.Boolean verifyBounds) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/IMGUIContainer.cs:618)
UnityEngine.UIElements.IMGUIContainer.SendEventToIMGUI (UnityEngine.UIElements.EventBase evt, System.Boolean canAffectFocus, System.Boolean verifyBounds) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/IMGUIContainer.cs:607)
UnityEngine.UIElements.IMGUIContainer.HandleEvent (UnityEngine.UIElements.EventBase evt) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/IMGUIContainer.cs:535)
UnityEngine.UIElements.CallbackEventHandler.HandleEventAtTargetPhase (UnityEngine.UIElements.EventBase evt) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/Events/EventHandler.cs:151)
UnityEngine.UIElements.EventDispatchUtilities.PropagateEvent (UnityEngine.UIElements.EventBase evt) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/Events/IEventDispatchingStrategy.cs:74)
UnityEngine.UIElements.MouseEventDispatchingStrategy.SendEventToRegularTarget (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.BaseVisualElementPanel panel) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/Events/MouseEventDispatchingStrategy.cs:35)
UnityEngine.UIElements.MouseEventDispatchingStrategy.SendEventToTarget (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.BaseVisualElementPanel panel) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/Events/MouseEventDispatchingStrategy.cs:26)
UnityEngine.UIElements.MouseEventDispatchingStrategy.DispatchEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel iPanel) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/Events/MouseEventDispatchingStrategy.cs:19)
UnityEngine.UIElements.EventDispatcher.ApplyDispatchingStrategies (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel, System.Boolean imguiEventIsInitiallyUsed) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/EventDispatcher.cs:381)
UnityEngine.UIElements.EventDispatcher.ProcessEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/EventDispatcher.cs:344)
UnityEngine.UIElements.EventDispatcher.Dispatch (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel, UnityEngine.UIElements.DispatchMode dispatchMode) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/EventDispatcher.cs:222)
UnityEngine.UIElements.BaseVisualElementPanel.SendEvent (UnityEngine.UIElements.EventBase e, UnityEngine.UIElements.DispatchMode dispatchMode) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/Panel.cs:419)
UnityEngine.UIElements.UIElementsUtility.DoDispatch (UnityEngine.UIElements.BaseVisualElementPanel panel) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/UIElementsUtility.cs:468)
UnityEngine.UIElements.UIElementsUtility.UnityEngine.UIElements.IUIElementsUtility.ProcessEvent (System.Int32 instanceID, System.IntPtr nativeEventPtr, System.Boolean& eventHandled) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/UIElementsUtility.cs:211)
UnityEngine.UIElements.UIEventRegistration.ProcessEvent (System.Int32 instanceID, System.IntPtr nativeEventPtr) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/UIElementsUtility.cs:74)
UnityEngine.UIElements.UIEventRegistration+<>c.<.cctor>b__1_2 (System.Int32 i, System.IntPtr ptr) (at /Users/bokken/buildslave/unity/build/ModuleOverrides/com.unity.ui/Core/UIElementsUtility.cs:28)
UnityEngine.GUIUtility.ProcessEvent (System.Int32 instanceID, System.IntPtr nativeEventPtr, System.Boolean& result) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

By the way, it’s insane that this forum does not allow users to upload images, no?

Verne33 · April 25, 2023, 3:02am

Controllers are not working for me, regardless of XR Plugin Management configuration. Very disappointing because this was advertised as a main feature of the new package I was really looking forward to utilizing

confy · April 26, 2023, 11:12am

Oculus Integration v51
Unity 2021.3.21f
I’m constantly getting this error. Tried everything

unity_31130666 · November 3, 2023, 5:58am

Hello It worked for me
I just updated XR above 1.4.4
Thanks

m0nkeybl1tz · November 17, 2023, 9:45pm

Hey all, a couple people I’ve been working with have been having this issue with Unity 2022.3.11. The other common factor seems to be that they’re Mac users. This is with Input System 1.7.0, OpenXR 1.8.2, Unity OpenXR Meta 1.0.0, and XR Interaction Toolkit 2.5.2 with the Starter Assets and XR Device Simulator added.

Is anyone else still having this issue?

manav_shah · May 11, 2024, 10:50am

Issue is solved in Unity version 2022.3.27f1