Display a Unity RenderTexture in an OpenGL plugin

bugmagnet · May 6, 2016, 3:28pm

Hi, my goal is to display a RenderTexture full screen on a separate window from Unity.

I have built a plugin for Unity where I make an OpenGL window and try to draw a RenderTexture on it. If I build my own application around this code, I am able to display my own (dynamic) texture on it. However I am stuck when I try to display a RenderTexture using GetNativeTexturePtr().
In fact, I am not even sure exactly what is coming out of GetNativeTexturePtr():

If I run Unity normally, I can sometimes see garbage pixels (that do not seem related to the RenderTexture.GetNativeTexturePtr() that I sent). These pixels seem to be texture related, but not to the given RenderTexture. My guess is I am looking at random data in the GPU memory. Code below.
If I run Unity as OpenGL, the best I can get is for it to not crash but I presume that this doesn’t work because according to the docs, Unity sends the gl texture “name” in this case, but because I am creating my own OpenGL context my understanding is that each context creates its own list of texture names hence the texture names I get are not meaningful to my external window.

Below is my attempt that at least displays garbage pixels, but sometimes crashes. I know something is definitely wrong but I am at a loss of how to actually USE the data inside GetNativeTexturePtr(). I have looked at the Unity plugin example but that only shows how to write to the texture. Is it even possible to read from it? Or is the pointer only meant for writing?
I am a novice in OpenGL, any help or pointers (har har) are greatly appreciated.

public RenderTexture rTex;

[DllImport("unityCaster")]
public static extern void update(int w, int h, System.IntPtr texPtr); 

//...
void Update()
{
     update(rTex.width, rTex.height, rTex.GetNativeTexturePtr());
}

void unityCaster::update(int w, int h, unsigned char * texData)
{
    if (!instance)
        return;

    if (texData == 0)
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
    else
    {

        glEnable(GL_TEXTURE_2D);
        glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_DECAL);
                  
        glBindTexture(GL_TEXTURE_2D, instance->texHandle);

        glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, w, h, 0, GL_RGBA, GL_UNSIGNED_BYTE, texData);
       
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);

        // Draw a textured quad to the whole screen
        glBegin(GL_QUADS);
        glTexCoord2f(0, 0); glVertex3f(0, 0, 0);
        glTexCoord2f(0, 1); glVertex3f(0, instance->height, 0);
        glTexCoord2f(1, 1); glVertex3f(instance->width, instance->height, 0);
        glTexCoord2f(1, 0); glVertex3f(instance->width, 0, 0);
        glEnd();

        glFlush();
        glDisable(GL_TEXTURE_2D);
    }

    SDL_GL_SwapWindow(instance->window); //im using SDL to get my openGL context
    glFinish(); //draw to screen
}

Tomas1856 · May 6, 2016, 4:12pm

If I am not mistaken, you get texture handle. So basically you don’t need to call glTexImage2D.

void unityCaster::update(int w, int h, unsigned char * texData)
{
glBindTexture(GL_TEXTURE_2D, (GLuint)texData);
...
}

bugmagnet · May 6, 2016, 4:52pm

Hi Tomas, thank you for the reply
In this case, the result is simply a white (blank?) texture.
It does not work, but I don’t know why.
Logging the GLuint handle gives a value such as: “201770208” which seems to be appropriate.
Any other ideas? Does a render texture get cleared at some point during the update process?
Should such a handle be valid across rendering contexts?

I have also tried adding the component to the rendering camera and doing:

void OnPostRender()
{
     update(castedTexture.width, castedTexture.height, castedTexture.GetNativeTexturePtr());
}

Here is my current code:

void unityCaster::update(int w, int h, void * texData)
{
    if (!instance)
        return;

    if (texData == 0)
    {
        glClear(GL_COLOR_BUFFER_BIT);
    }
    else
    {

        glEnable(GL_TEXTURE_2D);
        glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_DECAL);
                  
        //logging this handle appears to give a valid pointer/handle
        //it is consistent and changes appropriately with changes in Unity.
        GLuint handle = (GLuint)texData;
        glBindTexture(GL_TEXTURE_2D, handle);

        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); //alternatively GL_NEAREST or GL_LINEAR
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);

        // Draw a textured quad to the whole screen
        glBegin(GL_QUADS);
        glTexCoord2f(0, 0); glVertex3f(0, 0, 0);
        glTexCoord2f(0, 1); glVertex3f(0, instance->height, 0);
        glTexCoord2f(1, 1); glVertex3f(instance->width, instance->height, 0);
        glTexCoord2f(1, 0); glVertex3f(instance->width, 0, 0);
        glEnd();

        glFlush();
        glDisable(GL_TEXTURE_2D);
    }

    SDL_GL_SwapWindow(instance->window);
    glFinish(); //draw to screen
}

Tomas1856 · May 6, 2016, 5:27pm

You’re probably right about your context not being able to get resources from Unity’s OpenGL context. But you can probably use wglGetCurrentContext (to acquire Unity’s OpenGL context) and wglShareLists to share this data between Unity’s context and your context.

lacucaracha · May 31, 2016, 8:01am

Hello,

I was browsing the forum in order to find useful information.
Your code looks like mine, take a look there :

Maybe this will help !

bugmagnet · May 31, 2016, 5:37pm

We were able to get as far as getting the Unity context, but we were not able to use it to take a texture and use it in our plugin (we could not switch context back to our own window, Unity would take it back). We WERE able to modify textures and windows that Unity itself is using (which is what the Unity plugin example does).

lacucaracha
it looks like you were not able to solve the issue in directx either? Is that correct?

lacucaracha · June 1, 2016, 7:59am

Hello again,

The problem i have is to translate my opengl code into DirectX code, and i have 0 knowledge how to do that.
I looked here : Unity - Manual: Low-level native plug-in interface

There is a demo script in the Example section at the end of the page.
This is the only script i got to use directx / opengl in a native plugin for Unity.

However the code below is never called by my Unity script, so i cannot get the context nor the information about opengl or DirectX form unity.

This function is never called for me at the loading :

void UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API UnityPluginLoad(IUnityInterfaces* unityInterfaces)
    {
        s_UnityInterfaces = unityInterfaces;

        s_Graphics = s_UnityInterfaces->Get<IUnityGraphics>();
        s_Graphics->RegisterDeviceEventCallback(OnGraphicsDeviceEvent);

        // Run OnGraphicsDeviceEvent(initialize) manually on plugin load
        // to not miss the event in case the graphics device is already initialized
        OnGraphicsDeviceEvent(kUnityGfxDeviceEventInitialize);   
    }

But this post is about your problem, i wont ask anything more about mine here, dont hesitate to go at mine if you can help me ^^

Thx !

Oninova · August 22, 2016, 8:54pm

I was wondering if anyone knows how to get a handle to the unity render context. I need to create a texture in a external application and render it in Unity using Texture2D.CreateExternalTexture(). When I create the texture in my external application I get a texture id of 1, which seems wrong. I’m guessing its because it has its own render context and does not share a texture pool with Unity. I could be totally wrong about this because I’m a bit of a OpenGL noob. If anyone could point me in the right direction it would be greatly appreciated.

I tried this and only received a null value from wglGetCurrentContext, I’m guessing there is another way to get the context.

How where you able to get the context?

Oninova · August 24, 2016, 6:47pm

I was able to grab the handle to the opengl unity render context by using a low-level native plugin and fetching the handle with wglGetCurrentContext(). Unity - Manual: Low-level native plug-in interface

Using GetRenderEventFunc() allowed me to call GL.IssuePluginEvent() directly from the render thread, which in turn allowed me to grab the context handle. I am now able to render my textures externally and then display them in Unity using Texture2D.CreateExternalTexture().

unity_qBIeG5jmaYv5QQ · September 11, 2020, 5:54am

Hi, I need the same thing to draw the unity texture on the OpenGl window in C#.Can you please provide some more detail on what you have done to achieve render texture externally. i am doing my code in c# help is greatly appriciated

Varaughe · March 22, 2024, 2:51pm

There is a solution “Render to External Window” but it’s Metal for Mac and DirectX for Windows.