Low-level plug-in interface - UnityRegisterRenderingPlugin and XR Interfaces

Unity Engine
1

We are working on a WebXR Export package, got the Unity XR SDK, and while implementing it, we noticed that we need a way for Unity to call “UnityPluginLoad” and “UnityPluginUnload” on the native side.

We found a blog post https://blogs.unity3d.com/2017/01/19/low-level-plugins-in-unity-webgl/
which talks about “UnityRegisterRenderingPlugin”.
We tried it, and it only implements the “IUnityGraphics”, but we need a way to register XR Plugin (“IUnityXRInputInterface” and “IUnityXRDisplayInterface”).

As I understand “UnityRegisterRenderingPlugin” is the only hook.

Are there other ways to initiate a call to “UnityPluginLoad” with the needed interfaces?

Is it a bug? feature request? Who should we contact to add a hook for the XR interfaces?

Thanks

2

Even though the function is name UnityRegisterRenderingPlugin, it is actually used to register any type of plugin in WebGL builds. That is, something like

#include "IUnityInterface.h"
#include "XR/IUnityXRTrace.h"
#include "XR/UnitySubsystemTypes.h"

#include "ProviderContext.h"

static ProviderContext* s_Context{};

UnitySubsystemErrorCode Load_Display(ProviderContext&);
UnitySubsystemErrorCode Load_Input(ProviderContext&);

static bool ReportError(const char* name, UnitySubsystemErrorCode err)
{
    if (err != kUnitySubsystemErrorCodeSuccess)
    {
        XR_TRACE_ERROR(s_Context->trace, "Error loading subsystem: %s (%d)\n", name, err);
        return true;
    }
    return false;
}

extern "C" void UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API
UnityPluginLoad(IUnityInterfaces* unityInterfaces)
{
    auto* ctx = s_Context = new ProviderContext;

    ctx->interfaces = unityInterfaces;
    ctx->trace = unityInterfaces->Get<IUnityXRTrace>();

    if (ReportError("Display", Load_Display(*ctx)))
        return;

    if (ReportError("Input", Load_Input(*ctx)))
        return;
}

extern "C" void UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API
UnityPluginUnload()
{
    delete s_Context;
}

Behind the Unity XR SDK Sign Up link you can find the above sample, which adapted to the web would look like

#include "IUnityInterface.h"
#include "XR/IUnityXRTrace.h"
#include "XR/UnitySubsystemTypes.h"

#include "ProviderContext.h"

static ProviderContext* s_Context{};

UnitySubsystemErrorCode Load_Display(ProviderContext&);
UnitySubsystemErrorCode Load_Input(ProviderContext&);

static bool ReportError(const char* name, UnitySubsystemErrorCode err)
{
    if (err != kUnitySubsystemErrorCodeSuccess)
    {
        XR_TRACE_ERROR(s_Context->trace, "Error loading subsystem: %s (%d)\n", name, err);
        return true;
    }
    return false;
}

static void // Changed to static to avoid any linkage collision with function name
UnityPluginLoad(IUnityInterfaces* unityInterfaces)
{
    auto* ctx = s_Context = new ProviderContext;

    ctx->interfaces = unityInterfaces;
    ctx->trace = unityInterfaces->Get<IUnityXRTrace>();

    if (ReportError("Display", Load_Display(*ctx)))
        return;

    if (ReportError("Input", Load_Input(*ctx)))
        return;
}

static void // Changed to static to avoid any linkage collision with function name
UnityPluginUnload()
{
    delete s_Context;
}

#ifdef __EMSCRIPTEN__

extern "C" void UnityRegisterRenderingPlugin(PluginLoadFunc loadPlugin, PluginUnloadFunc unloadPlugin);

extern "C" void UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API RegisterMyWebXRPlugin()
{
    UnityRegisterRenderingPlugin(UnityPluginLoad, UnityPluginUnload);
}
#endif

and then in C# side:

class MyRenderPlugin
{
    #if UNITY_WEBGL && !UNITY_EDITOR
    [DllImport ("__Internal")]
    private static extern void RegisterMyWebXRPlugin();
    #endif

    void Start()
    {
        RegisterMyWebXRPlugin();
    }
}

Basically the function UnityRegisterRenderingPlugin() is a replacement to the lack of ability to do dlsym() on the web, since everything is statically linked to.

3

Thanks,

we tried that in Unity 2019.4.7f1 and it failed.

if we add to main
#include “XR/IUnityXRDisplay.h”
#include “XR/IUnityXRInput.h”

and after
ctx->trace = unityInterfaces->Get();

we add

    s_Graphics = unityInterfaces->Get<IUnityGraphics>();
    printf("Get<IUnityGraphics>\n");
    if (s_Graphics == NULL)
    {
        printf("s_Graphics == NULL\n");
    }
    else
    {
        printf("WE HAVE s_Graphics !!!!\n");
    }
   
    s_XrDisplay = unityInterfaces->Get<IUnityXRDisplayInterface>();
    printf("Get<IUnityXRDisplayInterface>\n");
    if (s_XrDisplay == NULL)
    {
        printf("s_XrDisplay == NULL\n");
    }
    else
    {
        printf("WE HAVE s_XrDisplay !!!!\n");
    }
   
    s_XrInput = unityInterfaces->Get<IUnityXRInputInterface>();
    printf("Get<IUnityXRInputInterface>\n");
    if (s_XrInput == NULL)
    {
        printf("s_XrInput == NULL\n");
    }
    else
    {
        printf("WE HAVE s_XrInput !!!!\n");
    }

will print:
WE HAVE s_Graphics !!!
s_XrDisplay == NULL
s_XrInput == NULL

4

That does look odd. My understanding is that the code should be able to obtain the interfaces (although I have not done any work with the XR API to say for sure)

Does the same code work to obtain XR APIs when you build for native Windows? If so, that does suggest a WebGL bug. If not, that might be a general XR issue.

5

Yes.
Works on windows and fail on WebGL when using the “UnityRegisterRenderingPlugin”.

Should I report a bug and post here the bug number?

6

Reported - Case 1283105

7

Oh hmm, now thinking back I realize the whole XR API might (currently) not be available on the web. But maybe it should be for the exact purposes of implementing XR plugins like this.

Thanks for reporting!

2 Likes
8

Thanks :slight_smile:

If there’s any progress behind the scenes, can you please update about that?

It’s an open source project that we work on in our free time, but it’s a few weeks now that we can’t progress with the implementations.

9

Maybe will also help to mention, that there’s a file “UnitySubsystemsManifest.json” that ask for “libraryName”.
So I set this to “__Internal”, instead of a library file name… like when using DllImport on WebGL.

10

I think the updates should come as a CC to your email inbox. I’ll try to remember to post here for anything that does not.

I believe the UnitySubsystemsManifest.json will be just ignored in WebGL builds (although not 100% sure, have not used that before)

1 Like
11

Thanks

12

The problem is that the XR module subsystems are not available to the WebGL platform. Subsystem requirements are usually declared with UnitySubsystemsManifest.json, but this is not available on WebGL, due to the file system. There will be a lot of work required to get these things to work with the platform limitations WebGL has. Having a WebXR driver for the XR subsystem would be great, I would like to use it, but it will likely be a longer term fix.

2 Likes
13

Does that mean that it won’t be fixed, or just that it’ll take more time to fix it?
Thanks for reviewing the issue.

14

FYI this issue has been fixed in 2020.3.6f1+, 2021.1.4f1+ and 2021.2.0a10+.

1 Like
15

Thanks!
Also just noticed that I got a reply in the bug tracker with a sample package.
The sample package didn’t work, and the C/C++ code from my bug project seems to have errors now when it compiles with Unity.
But managed to test the basic use case of getting the XRDisplay and XRInput, and it works!

I guess that I’ll need to compile a wasm bitcode file/lib and use it in the project, instead of raw C/C++ code.

16

Figured out I can use “-std=c++11” in the emscriptenArgs.
The sample code works :slight_smile:
Thanks!

7294501--882682--upload_2021-7-3_10-58-24.png
7294501--882682--upload_2021-7-3_10-58-24.png1080×676 138 KB

3 Likes
17

2 years later, and I got to work on this again.

For now I focus on LTS 2020.3.11f1 and LTS 2022.3.10f1.

What works:

  • Headset/device tracking - works great!
  • Display rendering - Setting the views and render texture.

What doesn’t work on both versions:

  • Single-Pass and Multiview - I’m not expecting Multiview to work, but I was thinking that single pass might work.
  • The RenderTexture of the display is updated only on the first frame, or if there’s another enabled camera on scene with Target Eye set to None (Main Display). In all other cases, the RenderTexture stays the same.

What fails on 2022.3.10f1:

  • WebGL Warnings on XR mode - “INVALID_VALUE: bufferSubData: buffer overflow”, “INVALID_OPERATION: drawBuffers: BACK or NONE”, “INVALID_ENUM: invalidateFramebuffer: invalid attachment”.
  • Once on XR mode, and enabling one more camera for spectator mode (Target Eye set to None), performance drops a lot. 2 calls to “bufferSubData” that takes 15ms each, on PC. Even after the extra camera is disabled, that performance drop keeps.

9519367--1342516--upload_2023-12-9_6-10-24.png
9519367--1342516--upload_2023-12-9_6-10-24.png875×182 23.5 KB

9519367--1342519--upload_2023-12-9_6-10-42.png
9519367--1342519--upload_2023-12-9_6-10-42.png747×273 17.8 KB

Other than that, this is AWESOME!

1 Like
18

A working branch is at https://github.com/De-Panther/unity-webxr-export/tree/xr_sdk_new_support
MainProject for the 2022.3.10f1 version
DebugProjects/Unity2020.3 for the 2020.3.11f1 version
The repo use symlinks.

1 Like
19

Opened a PR Add support for Unity XR SDK by De-Panther · Pull Request #331 · De-Panther/unity-webxr-export · GitHub
And here are the methods that calls bufferSubData and take lots of time in Unity 2022.3.10f1, in case it helps identifying the issue.

9520129--1342738--upload_2023-12-9_21-57-4.png
9520129--1342738--upload_2023-12-9_21-57-4.png1491×820 79.5 KB

1 Like
20

Done some tests with URP.
There are no issues of display not refreshing/update.
Frame rate is ok.
But there are some artifacts in some cases. On AR it’s randomly modify objects to triangles, and on VR it shows one color for an eye when looking on a place with only skybox.
Also linear color space looks dark when entering XR mode.

1249×559 22.8 KB

1 Like