I’m using a Direct3D11 plugin on Windows to update a Texture2D.
The display colors are fine when I use TextureFormat.ARGB32 on the Unity Scene with ARGB data sent from the plugin with ctx->UpdateSubresource(unityTexture, 0, NULL, dataBuf, width * 4, 0)
If I switch to TextureFormat.YUY2 and use YUY2 data, the image gets a red tint similar to the effect when U and V planes are reversed in YUV/I420. I experience this issue with Media Foundation Media Format YUY2 output and in comparison testing with libyuv::ARGBToYUY2() using ctx->UpdateSubresource(unityTexture, 0, NULL, dataBuf, width * 2, 0).
I would prefer to pass the YUY2 output directly from the image source over to the texture if possible.
Is there a step or configuration I am missing – or is this an issue with Texture2D?
This is primarily a low-level graphics plugin issue using DirectX as I’m trying to directly use the Media Foundation output in YUY2 format instead of applying another transform/conversion to ARGB.
I wasn’t disputing that. Kinect v2 raw color frame is in YUV2 but using the MS provided KinectWrapper, you can’t feed the color frame into a Texture2D.
The only way to do that is through an external DX11 plugin. Hence my question.
Are you sure you’ve selected the correct texture format in the D3D11_TEXTURE2D_DESC for the texture object?
There’s also a possibility you might have set the RenderTargetView with the wrong format.Check out the DXGI enum page under YUV2 for more info.
I’m trying to use output from Media Foundation as a byte array directly with this setup:
OutputMediaType->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);
OutputMediaType->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_YUY2);
and then passing it over to a Unity Texture2D via UpdateSubresource(unityTexture, 0, NULL, dataBuf, width * 2, 0) using the Unity ID3D11DeviceContext in my plugin.
I tested using I420 output and I had to invert U and V references before converting to ARGB to display correctly in Unity. I suspected this is a related issue since doing YUY2 to ARGB conversion also results in a similar color shift and the workaround is to change the Texture2D format in unity to BGRA32 instead of ARGB32
You mentioned having to use an external DX11 plugin – does this mean having to create a d3d texture in the external plugin to receive the byte array and then retrieve its contents to pass to the Unity texture?
Well I haven’t finished the plugin, but yeah the whole point is to create the D3D object in the plugin and update it there, then just feed the texture pointer to Unity to create its own Texture2D using that pointer and basically the plugin is the one managing and updating the texture content.
Unity has only a reference to it and just updates its reference.
So far I have managed to create the D3D texture and also get Kinect to give me the color feed and straight into the D3D texture via UpdateSubResource. But I have yet to test it in Unity.
I think the example provided by Unity for this sort of thing uses it the same way.
Thanks for the suggestion, I’ll test it out to see if the a D3D texture displays YUY2 as expected and if the transfer to the in-scene Texture2D will still match.
The Unity example does byte array manipulation but since the in-scene Texture2D is in ARGB format, it doesn’t manifest the issue.