ARFoundation Black screen when trying to get CPU camera

Unity Engine
,
1

Hey guys

I’ve followed this page to get the picture from the camera using this tutorial. One of the problem I encounter is that whenever I call the function to call the image 2 different situations happen:

  • If Multithreading rendering is enabled, the FPS will drop all the way down to 10 FPS
  • If Multithreading rendering is disabled, I have a full black screen

Is there any other parameter I should change?

Unity 2019.1.14.f1
ARFoundation 2.1.4
Arcore XRPlugin 2.1.2

Tested on a Xiaomi mix 2

The code to get the camera image in CPU is the following

private unsafe void GetImage()
    {
        XRCameraImage image;
        if (m_CameraManager.TryGetLatestImage(out image))
        {
            var conversionParams = new XRCameraImageConversionParams
            {
                // Get the entire image
                inputRect = new RectInt(0, 0, image.width, image.height),

                // Downsample by 2
                outputDimensions = new Vector2Int(image.width / 2, image.height / 2),

                // Choose RGBA format
                outputFormat = TextureFormat.RGBA32,

                // Flip across the vertical axis (mirror image)
                transformation = CameraImageTransformation.MirrorY
            };

            // See how many bytes we need to store the final image.
            int size = image.GetConvertedDataSize(conversionParams);

            // Allocate a buffer to store the image
            var buffer = new NativeArray<byte>(size, Allocator.Temp);

            // Extract the image data
            image.Convert(conversionParams, new IntPtr(buffer.GetUnsafePtr()), buffer.Length);

            // The image was converted to RGBA32 format and written into the provided buffer
            // so we can dispose of the CameraImage. We must do this or it will leak resources.
            image.Dispose();

            // At this point, we could process the image, pass it to a computer vision algorithm, etc.
            // In this example, we'll just apply it to a texture to visualize it.

            // We've got the data; let's put it into a texture so we can visualize it.
            m_Texture = new Texture2D(
                conversionParams.outputDimensions.x,
                conversionParams.outputDimensions.y,
                conversionParams.outputFormat,
                false);

            m_Texture.LoadRawTextureData(buffer);
            m_Texture.Apply();

        }
    }

Thanks!

1 Like
2

I got exactly the same behaviour.

With multithreaded rendering off it works just fine in many phones, but I got a black screen on Pocophone F1, Samsung Galaxy Tab S6 and Sony Xperia XZ2.

As soon as I call TryGetLatestImage for the first time on the frameUpdated callback, the ARCameraBackground scripts stops painting the camera background and draws a black screen. If I rotate the device changing its orientation, the screen starts to paint the camera background again. All the retrieved images are valid, but the camera background keeps black all the time after the first call.

3

Same problem here with ARFoundation 2.2.0-preview3

On Samsung Galaxy Tab S6 and Xperia XZ2 it displays a black screen when I call TryGetLatestImage() method. That black screen keeps there unless I rotate the phone (change its orientation).

In other phones (Pixel 2XL, Samsung Galaxy S9, Samsung Galaxy S9 plus, Asus Zenfone AR, LG G6 …) it just works fine.

1 Like