Is it possible to obtain the view of a camera as a matrix of pixels?

Questions & Answers
1

Hello UA,

I am trying to use the Unity ML-Agents API and for my project, but my question isn’t specific to that.

In my project, I want to use the view of a character in game as my data, so I was wondering if Unity has any built in way to return the ‘view’ (what the camera sees) as a matrix of pixels. I understand that I can probably use screen capture software externally on my screen and use that as data, but does anyone know of any built in functions that Unity has to access this data directly?

2

You can render your camera to a RenderTexture of your target size and then use ReadPixels on a Texture2D to read the rendertexture into a texture2d. Now you can use GetPixels or GetPixels32 to get the color data of the texture. However you should keep in mind that tranferring the rendertexture from GPU to RAM is a quite heavy and slow task.

3

You can make a Camera render to a RenderTexture using Camera.targetTexture.
Then, you can convert your RenderTexture into a Texture2D.
Finally, you can get an array of pixels from a Texture2D by calling Texture2D.GetPixels.

That should do the trick!