Read texture data for iOS PlugIn

Questions & Answers
1

Hi,

I’m trying to do a small Augmented Reality App with Unity for iOS. The goal is to capture data from the iOS device camera, put a Unity camera in orthographic projection and a plane in front of it with a texture of the real cameras data (with WebCamTexture for example), process that data with OpenCV on native iOS code and then again in Unity update the position of some 3D object (wich would be between the Unity camera and the plane) with the data processed by OpenCV.

Right now I don’t have a real iOS device so I’m working with the Simulator. That’s why, for the moment, I’m loading a JPG in a texture and trying to process its data for testing. When I have the real device I could use WebCamTexture, but I suppose the overall process would be similar.

The problem I have right now is that I don’t know how to read the data of the texture so I can process it on OpenCV. I can get the NativeTextureID but in OpenGL ES there is no glGetTexImage call that would give me just what I want. I’ve also tried sending the GetPixels32 data to my iOS function but it looks like only the first RGBA pixel is correct, the next one is all 0’s. Also I’m not sure if this would be extremely slow to process, may be it would load too many data on every frame.

In Unity:

Texture2D tex = renderer.material.mainTexture as Texture2D;
data = tex.GetPixels32();
_LoadColor32Array( ref data ); // My iOS native function

In iOS:

struct Color32 {
 Byte r;
 Byte g;
 Byte b;
 Byte a;
};

void _LoadColor32Array( Color32 *pixels[] ) {
    NSLog(@"_LoadColor32Array");
    printf("RGBA(%i,%i,%i,%i)

", pixels[0]->r, pixels[0]->g, pixels[0]->b, pixels[0]->a);
printf(“RGBA(%i,%i,%i,%i)
“, pixels[1]->r, pixels[1]->g, pixels[1]->b, pixels[1]->a);
NSLog(@”-> complete.”);
}

Is there a way to read the texture data in iOS native code? May be with some OpenGL code using the texture ID? I’ve read something about FrameBuffer Objects but don’t know anything about that. Any other approach?

Thanks.

P.D.:I’m new to Unity and OpenGL so may be I’m right now in the wrong direction or there is a simpler way to do what I want.

2

I made it work by not passing the Color32 by reference (as it is an array I suppose it is always passed by reference as in C… or may be not, don’t really know). So just deleting “ref” in the C# code and the “*” in the Objetive-C code did the trick.

I’m still not sure if this approach will be fast enough for an Augmente Reality App. Could somebody explain me how would it be to copy the texture data to a FBO and then read it with glReadPixels? Although this could also not be the fastest solution I don’t know any other one.

Thank you.

3

Seven years later …

nowadays unity has TOTALLY CHANGED all of this:

Enjoy