Render of just one UI object (and children)

Unity Engine
1

I’ve got a UI Canvas with a couple children.

  1. One of these children covers the screen, and has a bunch of its own children with interesting content (buttons, text, etc.) that collectively make up most of what’s currently visible.
  2. Another top-level child of the Canvas is in front of the first object, and partially occludes it.

I’d like to get a render (to a texture) of just the first object (and its children) without the occluding object in front–i.e. what the entire screen would look like if the front, occluding object were removed.

The ideal solution should also meet the following criteria:
A. Invisible to the user (doesn’t change what actually gets displayed on the screen for the player to see)
B. Does not make any particular assumptions about how object #2 works or how the rest of the object hierarchy is set up (e.g. I’d prefer not to require that object #2 be placed in a special layer)
C. Completes in the same frame (a solution with a 1-frame delay on results would still be useful, but is not ideal)

Some options that have occurred to me:

I could disable object #2, wait a frame, take a screenshot, and re-enable object #2. This accomplishes the main goal but fails ALL of my secondary criteria.

I could place object #2 in a different layer, set up a second camera that ignores that layer, and wait while the second camera renders the screen (without object #2) to a texture. This meets criteria A but fails B and C.

Anyone have a better suggestion?

Presumably there is some code somewhere inside of Unity engine that “knows” how to render a specific object or objects without rendering the rest of the world, but I don’t know if that’s accessible to a game developer in any way that’s useful for this particular problem.

2

I was going to suggest a second camera, how does it fail B and C?
You only need to know what layers are the UI rendered on, not anything special on how it’s rendered on those layers
And why wouldn’t it complete in the same frame?

3

All of the objects are UI. The “second camera” solution requires that the objects you want in the render are on a different layer from the objects you don’t want in the render, and that you know what those layers are.

I’m trying to build a general, reusable component that I can put in all my future projects, so I want using it to require as little extra work/limitations as possible, and thus “you must assign different parts of your UI to these exact layers in this exact pattern or else you can’t use this component” is an undesirable side-effect. Future projects might want to use layers in other ways that might not conveniently line up with this strategy. My coworkers might want to use my component without needing to jump through a bunch of hoops to make it work. I might bite the bullet and do this anyway, but I’d like to do better.

And by “in the same frame” I mean that I want to have a synchronous function launched from Update or something that makes a decision that a render is needed and then gets the output of that render during the same function call with no waits. Or more specifically I guess I mean that I want an opportunity to use that render to modify what the main camera renders (for the user to see) on the same frame that I made the decision to create the render. I suppose WaitForEndOfFrame is technically “in the same frame” but it’s too late for me to modify what the user sees during that frame.

If there’s a way to turn on a camera and force it to render right now, in the middle of Update, instead of waiting for the point in the game loop when rendering normally happens, then I’m not aware of it.

I also suppose I could have the secondary camera turned on and rendering ALL the time, just in case it’s needed, but that sounds massively wasteful for something I expect to need approximately once every 5 minutes.

4

Pretty sure that’s the point of .Render():

You’d use that with its output pointed at a RenderTexture.

How about Instantiate an entire copy of the hierarchy, climb through it to DestroyImmediate() the parts you don’t want, iterate all of it recursively to set all items to an unused Layer, create and point a camera at it, set that camera to ONLY render that layer, take the picture, then DestroyImmediate everything.

Maybe even just ransack the original hierarchy saving all the layers in a Dict, then set the ones you want to see to your layer, take the pic, then restore the layers from the Dict.

Unity is a beast…

1 Like
5

Oh! Thanks, that solves C.

I wonder if I can then solve B by disabling object #2, forcing a render, and then re-enabling object #2 in the same frame? Unfortunately I suspect not–my recollection is that calling SetActive(true) and SetActive(false) on the same object in the same frame doesn’t do what you’d intuitively expect (probably because some effects of that function are delayed until later in the frame?). But might be worth testing just to be sure…

I can’t tell if this is sarcasm, but creating and destroying copies of GameObjects with unknown scripts attached seems likely to have undesirable side-effects (such as invoking any Awake functions on those scripts), and I suspect there would also be a considerable performance cost. And I would be unsurprised if it turned out it doesn’t actually work at all, for reasons similar to the SetActive issue I mentioned above.

Uh…wow. That seems extreme, but I can’t think of any specific reason to rule it out.

Anyone have any idea how I should expect the performance cost of this option to scale with the number of objects? Does Unity do some fancy under-the-covers caching/optimization that will react badly to changing the layer of every object in the universe?

6

You may be right with this. I know at various times UI objects didn’t always update themselves until later in the frame cycle. Not sure what the state of it is now though.

Yeah, that’s true if you put whacky stuff in your UI hierarchy. I try to never do that, which is why I made the Datasacks module, which basically keeps me from directly touching any UI in game code / logic, makes the UI truly just be for presentation and input.

I bet you this is likely to be the least intrusive in the general case, if you anticipate this system being used in unspecified future contexts.

Try it! Are you contemplating keeping this feature going continuously during the game, or only on a one-shot “take a snap” kinda basis? Either way, I think Unity will impress you.

7

You can force the UI to update on some stuff, with text mesh pro Text component you can call ForceMeshUpdate, there’s something called UnityEngine.UI.LayoutRebuilder.ForceRebuildLayoutImmediate() that takes a rect transform as an argument, look it up, I used it force a tooltip to update, this was back in 2017 but I have a memory of using it in 2019 too for something…

1 Like
8

Even if your UI scripts only touch UI, it seems plausible that Awake could do some initialization that might overwrite data that’s currently being displayed, and therefore change the UI’s appearance just as I’m trying to make a record of how it looks.

Expected to be used once each time a moderately-unusual event occurs. Definitely never on consecutive frames.

Basically, I’m making pretty animations for when you leave the current menu/screen, and want those animations to work in certain circumstances where I can’t continue using the original objects.

Interesting, but probably not helpful for my particular problem, which is only about hiding certain objects that would otherwise be in view.

Though, come to think of it…rather than making the unwanted portion of the UI invisible, maybe I could just move them way off-screen? Not quite as reliable, since in principle it could have off-screen elements that I would be coincidentally moving on-screen with the move. Maybe move it off-screen plus apply a temporary RectMask2D to crop anything that might extend back onto the screen?

Guess I’ve got several things to test…

9

I ended up having to move some UI stuff off-screen for one project, it worked fine. I think it was because of click-through issues (it was UI plus regular GameObjects). This was easier than turning child renderers on and off, just move the top-level one. Sounds like an interesting project.

10

Follow-up!

I made a second camera to take the render (and put it on an inactive GameObject, and set a bunch of parameters on it).

A “Screen Space - Overlay” Canvas did not show up in the output of Camera.Render(). Neither did a “Screen Space - Camera” Canvas whose camera was set to null, or whose camera was set to the main camera. So in order to take a picture of the UI, I had to temporarily change the Canvas to point at my secondary camera (and put it in SS-Camera mode, if it wasn’t already).

I discovered that if I change from SS-Overlay to SS-Camera and then immediately change back, my on-screen text gets messed up. (I’m using TextMeshPro.) There’s this sort of blurry box that appears around the letters and stays there until I change screens (probably fixed as a side effect of SetActive).

The details of this are pretty weird. A Canvas in SS-Camera but with no specific camera specified is supposed to (according to Unity inspector) act like an Overlay canvas. If I start out the Canvas in SS-Camera mode but with no Camera specified, then this code causes no problems:

        Camera prevCam = canvas.worldCamera;
        try
        {
            canvas.renderMode = RenderMode.ScreenSpaceCamera;
            canvas.worldCamera = snapCam;
            snapCam.Render();
        }
        finally
        {
            canvas.worldCamera = prevCam;
        }

…but this code causes the visual issues:

        RenderMode prevRenderMode = canvas.renderMode;
        Camera prevCam = canvas.worldCamera;
        try
        {
            canvas.renderMode = RenderMode.ScreenSpaceCamera;
            canvas.worldCamera = snapCam;
            snapCam.Render();
        }
        finally
        {
            canvas.renderMode = prevRenderMode;
            canvas.worldCamera = prevCam;
        }

Setting canvas.renderMode back to its original value causes problems, even if the original value is the same as the new value and this therefore isn’t actually changing anything! I don’t know what’s going on here.

In order to remove the front object from the render, moving it a long distance away and then back worked fine. To my surprise, calling SetActive(false) and then SetActive(true) also worked fine, hiding it from the render but keeping it visible on-screen. Not sure if my memory of issues with this is incorrect, or if Unity fixed it at some point, or maybe even rendering it is what causes it to sort itself out.

I decided to stick with moving the object, though, since that seems less likely to set off side effects (some components could have OnEnable or OnDisable functions defined…)

I did not end up testing out the “change ALL the layers!” idea since I found something simpler that appears to work.

So, seems I have a pretty-good working solution. The visual glitch if I try to restore the Canvas’ original renderMode is bizarre and annoying, but I don’t think there’s any practical problem, since even if I want a SS-Overlay canvas, I can still get that behavior by starting it out in SS-Camera mode with no camera specified, and as long as the code doesn’t TRY to restore the renderMode this seems to work just fine.

11

Hey Anti, I recently learned about this:

https://docs.unity3d.com/ScriptReference/RectTransform.ForceUpdateRectTransforms.html

I’m curious if it fixes your problem by triggering a full UI repaint / remake.

12

Calling ForceUpdateRectTransforms on the canvas after restoring its settings does not seem to affect the visual glitch.

Also, I have noticed that I get a similar glitch if I just use the Unity inspector to change the Canvas from SS-Camera to SS-Overlay while in play mode. So it doesn’t seem to have anything to do with multiple changes in one frame, with my special rendering camera, or with any peculiarities of the exact script I’m using; this is just something that the Canvas does when you make this particular change. Could possibly be a bug in TextMeshPro?

13

Did you try calling ForceMeshUpdate on the tmp text?

1 Like
14

ForceMeshUpdate on the TMP_Text seems to fix the glitch.

However, there’s nothing special about this particular TMP_Text except that it happened to be visible at the time. You’d presumably have to do this for every visible TMP_Text anywhere in the entire Canvas.

On a side note, Looks like basic Unity Text (as opposed to TMP Text) does not have this glitch. So it’s something about TextMeshPro specifically.