Can't use values form StereoLabs Zed camera

Unity Engine
1

I’m using the ZED stereo depth camera in Unity, but I can’t figure out how to get at the depth values. Here is the loop where I’m trying to use the texture:

public Material depthMaterial;
private Texture2D depthTexture;
private float[,] depthValues;
private sl.ZEDCamera zedCam;

void Update()
{
     depthTexture = zedCamera.CreateTextureImageType(sl.VIEW.DEPTH) as Texture2D;
     depthMaterial.mainTexture = depthTexture;

        for (int i = 0; i < height; i++)
        {
            for (int j = 0; j < width; j++)
            {
                depthValues[i,j] = depthTexture.GetPixel(i,j).grayscale;
            }
        }
   
    terrain.terrainData.SetHeights(0, 0, depthValues);

}

But I get the same value for every pixel. Same result with a version that used GetPixels and Color32. The texture does have the depth map in it, I confirmed by applying depthMaterial to a Quad in the scene.

This method gets me values that look like they might be what I’m after, but it is so slow that the program is unresponsive:

void Update()
{
        for (int i = 0; i < height; i++)
        {
            for (int j = 0; j < width; j++)
            {
                depthValues[i,j] = zedCam.GetDepthValue((uint)i, (uint)j);
            }
        }

        terrain.terrainData.SetHeights(0, 0, depthValues);
}
2

Hi,
CreateTextureImageType(sl.VIEW.DEPTH) creates a texture on GPU, to get the values on CPU you should use a computeshader.

1 Like
3

Thanks faren! I think I can figure that out. I also received an e-mail from the developer about:

GetDepthValue((uint)i, (uint)j);

It copies a single pixel of depth data from the GPU also, which is why it is so slow. They say that they plan to include a C# function eventually, that will copy an entire frame of depth data at once, but that the compute shader is the way to go for now.

4

Hi Faren,
I am new to unity and ZED sdk.
Could please explain me the concept of computeshader and how to use it to get depth array.