Calling OpenGL in plugins (or how to render a background image)

Unity Engine
,
1

Hi,
I am new to Unity, and I am trying to figure out if it is possible to render a background image via a plugin.

I need to render the incoming images from a camera as the background to my AR app. The target platform is iOS. I have no need of the images as textures inside of unity. I have to have control of the texture handles, and access to the raw image in the plugin.

What I would like to do is have my plugin fully manage the texture, quad, and the associated drawing, so I would do the following:

  • create the texture and quad to render it to in my plugin

  • somewhere in unity (update() in a script attached to the main camera?) call my plugin.

  • have my plugin

  • update the texture

  • set up orthographic projection

  • render the textured quad

  • return to unity to draw the whole 3d scene.

Currently I am doing the following in a script attached to the main camera:

  • creating a plane at the main camera’s far clipping distance
  • applying a material with a texture.
  • then every frame I am sending the texture’s handle to my plugin.
  • In the plugin I am uploading the image to the texture.

It works, but it is slow (or rather, not as fast as I can make it).

Prior to using unity I have done the following:

  • render 3d scene to an offscreen FBO
  • set up orthographic projection
  • render background image to a quad
  • render 3d scene texture to another quad on top of the background quad.
  • display the results on the screen.
2

Prime31’s augmented reality plugin already does this, and it does it better than the system your trying to set up.

You do not need to send your texture’s memory reference to objective c every frame. You only have to send it once. Then you just update that texture reference with direct OpenGL code outside of unity and it will just update in Unity, because unity is referencing the same chunk of memory.

3

Thanks, but Prime31’s AR plugin is not suitable as it is doing essentially the same thing I am doing, but not letting me get to the raw image pixels or control the camera outside of unity in my plugin (where the main bulk of my app exists). Yes I can improve my current code and not pass the GLuint every frame, but it does not make any real performance difference.

I want to use Apple’s CVOpenGLESTextureCache to get a significant performance boost. To use this I have to let apple manage the texture.

I know that it should be possible to display the camera image as a background outside of Unity without passing a texture reference from unity as that is what StringAR does.