I am facing some problems in converting an image from YUV to RGB format.
I am using the Google Tango API with Unity. It gives me the bytes of an image in YUV format in the following callback function:
OnTangoImageAvailableEventHandler(TangoEnums.TangoCameraId cameraId, TangoUnityImageData imageBuffer)
Fortunately, they already have a YUV2RGB shader together with a material to convert images in their YUV format to RGB. So my plan was to create the two textures that the shader expects as input from the imageBuffer.data byte array. I then set those to the material from Tango and use Graphics.Blit to render the result to a RenderTexture. This is the function that I use for this task:
public void OnTangoImageAvailableEventHandler(TangoEnums.TangoCameraId cameraId, TangoUnityImageData imageBuffer)
{
// Split buffer into y and uv part
byte[] yDataPart = new byte[imageBuffer.width * imageBuffer.height];
Array.Copy (imageBuffer.data, 0, yDataPart, 0, imageBuffer.width * imageBuffer.height);
byte[] uDataPart = new byte[2 * (imageBuffer.width / 2) * (imageBuffer.height / 2)];
Array.Copy (imageBuffer.data, imageBuffer.width * imageBuffer.height, uDataPart, 0, 2 * (imageBuffer.width / 2) * (imageBuffer.height / 2));
// Create the textures from the data
Texture2D yTex = new Texture2D(width / 4, height, TextureFormat.RGBA32, false);
yTex.LoadRawTextureData (yDataPart);
yTex.Apply ();
Texture2D uTex = new Texture2D((width / 2) / 4 * 2, height / 2, TextureFormat.RGBA32, false);;
uTex.LoadRawTextureData (uDataPart);
uTex.Apply ();
// Set them to shader
yuv2rgbMat.SetTexture ("_YTex", yTex);
yuv2rgbMat.SetTexture ("_UTex", uTex);
// Draw
Graphics.Blit (dummyRenderTex, rgbRenderTex, yuv2rgbMat);
}
In that code snippet, the yuv2rgbMat is set to the ARScreen material from tango, which has the YUV2RGB shader connected.
This leads directly to my first question:
Is it necessary to provide an input render texture to Graphics.Blit if I already set the input textures directly to the material?
Now besides from that question, here is the real problem. When I later what to create a Texture2D from the RenderTexture to work further with it, the texture is completely set to 0. This is the code I use to access the resulting texture:
public IEnumerator drawRGBTexture()
{
while (true) {
yield return new WaitForEndOfFrame ();
if (rgbRenderTex != null) {
RenderTexture.active = rgbRenderTex;
rgbTex.ReadPixels (new Rect (0, 0, width, height), 0, 0);
rgbTex.Apply ();
// At this point rgbTex is completely filled with zeros!!
// Even if there were a lot of non zero entries in the original YUV byte stream
RenderTexture.active = null;
}
}
}
This leads to my second question:
Why is the resulting texture filled with zeros and not with the correct output of the shader?