Hello,
I was wondering if it’s possible to display a camera view within a UI element in Unity with the UI Toolkit? I’ve been trying to figure out a way to do this, but haven’t been able to find any concrete solutions. If anyone has any experience with this or could point me in the right direction, it would be greatly appreciated. Thank you!
Assign the camera to a render texture, and use the renderTexture as the visual element’s background image. There should be nothing more to do. 
Thanks, I was able to implement it successfully by following your instructions. I created a Render Texture and set the camera to render to it, then used the texture as the background image for the UI element. The camera’s view is now displayed within the UI element as desired. I appreciate your help, it’s greatly appreciated.
Hello,
this link :
@wphd He is asking how to do it with UI Toolkit, not the old Canvas as in your solution.
@SimonDufour can you elaborate more on this. I want the VisualElement to control the size of the render texture, so trying to figure out how best to create the RenderTexture. Creating one and assigning it to the background is a compiler error. assigning it to a new StyleBackground does not have a constructor that takes a render texture.
var mainWindow = root.Q<VisualElement>("main-window");
var windowStyle = mainWindow.style;
Camera camera = new Camera();
RenderTexture cameraTexture = new RenderTexture(160, 90, 24);
camera.targetTexture = cameraTexture;
windowStyle.backgroundImage = new StyleBackground(camera.targetTexture.colorBuffer.GetNativeRenderBufferPtr );
RenderTexture renderTexture = windowStyle.backgroundImage.value.renderTexture; // this is null
renderTexture.autoGenerateMips = false;
Any help would be appreciated. This is just to test. Another scene will either create the camera or change its size to conform to the desired renderTexture.
Also, at what point is the UIDocument “finalized” and the sizes of various elements are initialized?
Humm, I thought GetNativeRenderBufferPtr was ios only. I don’t actually have anything tested, but I don’t see why you would need to re-get a refernce to a render texture you already have. You can just take the reference and assign it in the StyleBackground.
var mainWindow = root.Q<VisualElement>("main-window");
var windowStyle = mainWindow.style;
Camera camera = new Camera();
RenderTexture cameraTexture = new RenderTexture(160, 90, 24);
renderTexture.autoGenerateMips = false;
camera.targetTexture = cameraTexture;
windowStyle.backgroundImage = new StyleBackground(cameraTexture);
In older version of UI toolkit, only the image field supported displaying a renderTexture the process was similar.
item.image = cameraTexture;```Yes, that is what I initially tested. There must be a breaking change in Unity 2022.2. The last statement in your code gives a compiler error:
error CS1503: Argument 1: cannot convert from ‘UnityEngine.RenderTexture’ to ‘UnityEngine.UIElements.Background’
They seem to have changed StyleBackground, but I cannot determine how to tell it to take a RenderTexture.
Rereading some of the documentation, I can get rid of the compiler error, but still not seeing the camera. I can see that the camera is changing the Render Texture, but cannot get it to display as the background image on the VisualElement. Here is what I have been trying:
var mainWindow = root.Q<VisualElement>("main-window");
var windowStyle = mainWindow.style;
//RenderTexture cameraTexture = _gameCamera.activeTexture; // Null
RenderTexture cameraTexture = _gameCamera.targetTexture;
cameraTexture.autoGenerateMips = false;
//camera.targetTexture = cameraTexture;
var background = new Background();
//background.renderTexture = cameraTexture; // Doesn't seem to work.
//background.texture = cameraTexture; // Compiler error.
background.texture = cameraTexture.toTexture2D(); // Extension method that read pixels and sets them. Doesn't seem to work. Even with resizing.
windowStyle.backgroundImage = new StyleBackground(background);
Edited. Most of my problems were on the UXML document. Will post about the structure and colors (opacities) needed next. Here is my sample code:
private void GeometryChangedCallback(GeometryChangedEvent evt)
{
//_root.UnregisterCallback<GeometryChangedEvent>(GeometryChangedCallback);
var mainWindow = _root.Q<VisualElement>("main-window");
Rect size = mainWindow.worldBound;
if (_useRenderTexture)
{
// Todo: Dispose of any old RenderTexture.
RenderTexture cameraTexture = new RenderTexture((int)size.width, (int)size.height, 24);
cameraTexture.autoGenerateMips = false;
var windowStyle = mainWindow.style;
_gameCamera.targetTexture = cameraTexture;
var background = new Background();
background.renderTexture = cameraTexture;
windowStyle.backgroundImage = new StyleBackground(background);
mainWindow.MarkDirtyRepaint();
}
else // Using a viewport
{
var windowSize = _root.contentRect;
// This position is top-left. Camera needs it bottom-left.
// Todo: Check Margin versus Padding (content area?).
size.position = new Vector2(size.position.x / windowSize.width, (windowSize.height - (size.position.y + size.height)) / windowSize.height);
size.width = size.width / windowSize.width;
size.height = size.height / windowSize.height;
_gameCamera.rect = size;
}
}
crawfis, why don’t you go back to the viewport camera by removing background image of the mainWindow?
windowStyle.backgroundImage = = StyleKeyword.None;
Ok. My variant to toggle viewport/texture render camera with UI Toolkit:
public class RenderToggler : MonoBehaviour
{
Camera cam;
RenderTexture targetTexture;
bool renderToTexture;
// how you obtain this VisualElement is up to you
VisualElement visualElement;
void Start()
{
cam = GameObject.Find("YourCamName").GetComponent<Camera>();
// targetTexture is a RenderTexture stored somewhere in resources
targetTexture = Resources.Load<RenderTexture>("targetTexture");
}
void ToggleRenderTarget()
{
if (renderToTexture)
{
cam.targetTexture = targetTexture;
Background background = new() { renderTexture = targetTexture };
visualElement.style.backgroundImage = new StyleBackground(background);
}
else
{
cam.targetTexture = null;
visualElement.style.backgroundImage = StyleKeyword.None;
}
renderToTexture = !renderToTexture;
}
}
@Alex16212 I would have to go back and remember, but I think I still need the viewport, as the main camera is still being rendered within a smaller area of the overall GUI. We have 4 render Textures and a main render area in our games: www.rxgames.com. So I limit the camera to the viewport as below. This is an image for our stroke rehab (or TBI) product (hemi-paresis of the right side).
