WebCam preview in UI Toolkit

Unity Engine
, ,
1

Hi all,
I am trying to get a webcam preview inside ui toolkit , to build a simple screen where the user can preview the camera before taking a photo.
I am facing many difficulties, and also the documentation is not very complete on these…
These are the steps I have followed:

  • I am using 2022.3, main targets are Android and iphones
  • i created a uidocument, with a single VisualElement, with a backgroundImage (a simple white rectangle from my assets)
  • I connected the code below, that compiles but doesn’t work

My issues:

  • I am not sure to get a reference to the backgroundImage right (currently I get NULL)
  • how should I apply the webcam texture to the VisualElement?

Any help would be appreciated, thanks!!!
Luigi

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UIElements;

using myDebugger;


public class scr_takephoto : myDebuggerUiParent {

    public override void Start ()
    {
        base.Start();
        Application.RequestUserAuthorization(UserAuthorization.WebCam);

        StartCoroutine(CamStart());
    }

    // https://discussions.unity.com/t/getting-a-web-cam-to-play-on-ui-texture-image/131628/2      
    // https://docs.unity3d.com/2022.3/Documentation/ScriptReference/UIElements.Image.html
  
    public Texture2D Convert_WebCamTexture_To_Texture2d(WebCamTexture _webCamTexture)
        {
        Texture2D _texture2D = new Texture2D(_webCamTexture.width, _webCamTexture.height);
        _texture2D.SetPixels32(_webCamTexture.GetPixels32());
        return _texture2D;
        }
    IEnumerator CamStart(){
        yield return null;

        WebCamDevice[] cam_devices = WebCamTexture.devices;

        myDebuggerDebug("Available cams:",cam_devices);

        VisualElement campreview=rootVisualElement.Q<VisualElement>("campreview");
        myDebuggerDebug("VisualElement done", campreview);
        StyleBackground backgroundImage = campreview.style.backgroundImage;
        myDebuggerDebug("Background image", backgroundImage);

        WebCamTexture webcamTexture = new WebCamTexture();
      
        backgroundImage = Background.FromTexture2D(Convert_WebCamTexture_To_Texture2d(webcamTexture));
  
        myDebuggerDebug("Now play!");
        webcamTexture.Play();
    }
}
2

You should probably blit into a RenderTexture intead of copying the image data into a Texture. You could then use it with an Image element.

3

Hi Alexandre,
thanks for the hint. And then, how can I plug the Image into a VisualElement? If I am not wrong Image is not part of “UI toolkit”
Luigi

4

There’s an Image visual element: https://docs.unity3d.com/ScriptReference/UIElements.Image.html

Not to be confused by the component of the same name from uGUI.

1 Like
5

Hi, this has been asked, and answered recently in another thread. Maybe this will help you:

@AlexandreT-unity It would be nice if the webcam would offer and api to automatically hook it up to a render texture. Seems like that would be a handy feature.

6

Many thanks, with the latest thread hints I was able to do it!

7

Actually the next step for us will likely be to support assigning a WebCamTexture to background/Image. This would avoid the extra blit.

In the meantime, I forgot to mention that you can use MeshGenerationContext.Allocate and provide it with the texture. So if you generate a simple quad it should work. Let us know if it doesn’t, in this case we should process it as a bug.

1 Like
8

Is this available yet..?

feels complicated to hack around rendertextures or something else,
and since old UI just works with webcamtexture (ie. feature parity would be nice..)

thanks!

9

You cannot assign it as a style, but you should be able to assign it to Image.image.

Otherwise, if you want to do some custom mesh with it, the MeshGenerationContext API supports its base class, Texture (see Allocate or AllocateTempMesh+DrawMesh). Here’s an example:

using UnityEngine;
using UnityEngine.UIElements;
using Unity.Collections;

public class WebCamExample : MonoBehaviour
{
    void Start()
    {
        var doc = GetComponent<UIDocument>();
        if (doc == null)
            return;

        var tex = new WebCamTexture();
        if (tex == null)
            return;

        tex.Play();
        int w = tex.width;
        int h = tex.height;

        doc.rootVisualElement.Add(new VisualElement()
        {
            style =
            {
                width = w,
                height = h
            },
            generateVisualContent = mgc =>
            {
                mgc.AllocateTempMesh(4, 6, out NativeSlice<Vertex> vertices, out NativeSlice<ushort> indices);

                float vTop = tex.videoVerticallyMirrored ? 0 : 1;
                float vBottom = 1 - vTop;

                vertices[0] = new Vertex { position = new Vector3(0, h), tint = Color.white, uv = new Vector2(0, vBottom) }; // Bottom Left
                vertices[1] = new Vertex { position = new Vector3(0, 0), tint = Color.white, uv = new Vector2(0, vTop) };    // Top Left
                vertices[2] = new Vertex { position = new Vector3(w, 0), tint = Color.white, uv = new Vector2(1, vTop) };    // Top Right
                vertices[3] = new Vertex { position = new Vector3(w, h), tint = Color.white, uv = new Vector2(1, vBottom) }; // Bottom Right

                indices[0] = 0;
                indices[1] = 1;
                indices[2] = 2;
                indices[3] = 2;
                indices[4] = 3;
                indices[5] = 0;

                mgc.DrawMesh(vertices, indices, tex);
            }
        });
    }
}