Issue with glDrawArrays in plugin

Questions & Answers
1

I am trying to write a plugin for Unity that does the following on iOS using OpenGL ES 2.0:

  1. Read camera image
  2. Draw the camera image into an FBO
  3. FBO is rendered to a Unity texture.
  4. Do stuff with texture in Unity.

The plugin is called in the update function of a script attached to the main camera.

The sticking point comes when I try and draw the textured quad in the offscreen FBO with
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4). If I comment out this line the app runs, my 3d scene is drawn correctly and runs at full framerate. All that is missing is the texture. If I enable the line, nothing draws and everything runs at about 3fps. No errors are reported.

This is the complete draw function:

- (void) drawVideoFrame
{
  GLint nCurrentProg;
  glGetIntegerv(GL_CURRENT_PROGRAM, &nCurrentProg);

  GLint nCurrentFrameBuffer;
  GLint nCurrentRenderBuffer;
  glGetIntegerv(GL_FRAMEBUFFER_BINDING, &nCurrentFrameBuffer );
  glGetIntegerv(GL_RENDERBUFFER_BINDING, &nCurrentRenderBuffer );

  glBindFramebuffer(GL_FRAMEBUFFER, nFBOBuffer);
  glBindRenderbuffer(GL_RENDERBUFFER, nFBORender);

  glViewport(0, 0, videoWidth, videoHeight);
  
  static const GLfloat squareVertices[] = {
     -1.0f, -1.0f,
      1.0f, -1.0f,
      -1.0f,  1.0f,
      1.0f,  1.0f,
    };
    
    static const GLfloat textureVertices[] = {
      0.0f, 1.0f,
      1.0f, 1.0f,
      0.0f, 0.0f,
      1.0f, 0.0f,
    };

  // Use shader program.
  glUseProgram(passThroughProgram);
  
  glBindTexture(GL_TEXTURE_2D, nTexID);
  
  // Update attribute values.
  glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
  glEnableVertexAttribArray(ATTRIB_VERTEX);
  glVertexAttribPointer(ATTRIB_TEXTUREPOSITON, 2, GL_FLOAT, 0, 0, textureVertices);
  glEnableVertexAttribArray(ATTRIB_TEXTUREPOSITON);

  glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

  glUseProgram(nCurrentProg);
  glBindTexture(GL_TEXTURE_2D, 0);
  glBindFramebuffer(GL_FRAMEBUFFER, nCurrentFrameBuffer);
  glBindRenderbuffer(GL_RENDERBUFFER, nCurrentRenderBuffer);
}

The fragment and vertex programs are just simple pass throughs:

Fragment

varying highp vec2 coordinate;
uniform sampler2D videoframe;

void main()
{
	gl_FragColor = texture2D(videoframe, coordinate);
}

Vertex

attribute vec4 position;
attribute mediump vec4 textureCoordinate;
varying mediump vec2 coordinate;

void main()
{
	gl_Position = position;
	coordinate = textureCoordinate.xy;
}

I took my plugin code and implemented it as a stand alone app, doing the same thing, and everything worked.

Note:

  • I am aware that this example is slightly pointless, but it is a minimal example
  • I can read the camera image and upload it directly to a unity owned texture, but that is not what I want to do here.

Does anyone have any idea what is wrong?

2

I think there are 2 things going on that are making that function not work.

First is that Unity uses VBO’s on android, and it’s leaving a VBO bound when you are calling your function. So when you are making the “glVertexAttribPointer” call, it’s interpreting your “textureVertices” pointer as an offset into the bound VBO. See the glVertexAttribPointer docs here for more info on that:
http://www.khronos.org/opengles/sdk/docs/man/xhtml/glVertexAttribPointer.xml

So you need to unbind Unity’s VBO with a call to glBindBuffer with a buffer handle of 0, so like this:

glBindBuffer(GL_ARRAY_BUFFER_BINDING, 0);

The other problem I think you are having is that you are trying to use the existing program handle when rendering the quad. I don’t know what code you did in Unity land to try and set the current program before calling your native function, but when I’ve played around with this stuff, I was using material.SetPass(0), and that function was not setting the current program :frowning:

The only way I could get stuff to render in a native plugin for android was to use a program created in the plugin itself, as Unity wasn’t "glUseProgram"ing for me :stuck_out_tongue:

3

I found upgrading from Unity3 to Unity4 solved a similar issue for me. I am now successfully setting of a FBO to copy from an arbitrary texture ID into a unity managed texture ID.

4

Finally, a solution to this question: https://github.com/robertcastle/UnityFBO

I am not sure what I was doing wrong in this example. I think it was a combination of not quite doing enough storing and reseting of the OpenGL state, and not calling GL.InvalidateState() after every plugin call in Unity.

This demo project I have put together works, and provides a starting block for anyone who wants to use an FBO with Unity. I hope it is helpful.