The UI team has exciting news for UI Toolkit users! We’ve made significant progress on two highly anticipated features: Custom Shaders via Shader Graph and Post-Processing Filters via USS. These are now available as of version 6000.3.0a5. As we’re in the alpha stage, we’re eager to get your feedback and input on their current state, how they meet your expectations, and to help validate our planned future work.
This is an alpha release, so bugs are expected – please help us by reporting any issues you encounter
Documentation is currently in progress
Custom Shaders in UI Toolkit are currently exclusive to Universal Render Pipeline (URP).
Performance implications:
Assigning a material to a VisualElement creates a new batch, which can affect rendering performance
Using filters involves render textures, which may increase bandwidth and GPU memory consumption
Filters operate in the plane of the VisualElement to which they are applied. If the VisualElement is displayed on-screen with an angle (world-space UI), some loss of visual quality can happen compared to direct on-screen rendering of that VisualElement.
[ Limited support] Previewing runtime shaders in Editor windows may appear somewhat unpolished. Additionally, shaders that rely on “time” will not render accurately in the Editor, as it lacks a runtime concept of time. In the Builder, runtime shaders might display outside their intended boundaries if the container utilizes custom stencils, such as those for rounded corners.
[ Limited support] While direct reading/writing of vertex color from the vertex stage is not possible, a color can be computed in the vertex stage and then interpolated using shader graph interpolators.
[ Limited support] While you cannot directly modify how text is rendered using custom shaders, you can alter the result of the text rendering.
[ Not supported] Custom shader masking as custom shapes is not possible. However, vector image masking is supported.
[ Not supported] Allowing stencil buffer control through the UI Toolkit shader’s master node for masking purposes.
[ Not supported] Modifying vertex position attributes and using vertex colors in UI Toolkit shaders
[ Not supported] Shader nodes referencing USS properties and responding to updates.
[ Not supported] Support of atlas and UV-wrapping
[ Not supported] Background blur and drop-shadow filter
USS Filters apply post-processing effects to a render texture generated from your UI hierarchy. This means blurring will affect the UI itself, not the background. While we include built-in filters like grayscale and opacity, we don’t offer drop-shadow or background blur (or backdrop blur) because USS filters aren’t the ideal solution for these specific use cases.
Both Filters and Custom Shader features simplify the implementation of custom shadows, blurs, and glows. However, our ultimate goal is to provide these functionalities out-of-the-box through standard CSS, specifically box-shadow (including soft shadows) and backdrop-filter.
[ Not supported] Custom vertex data: Users cannot provide custom vertex data, beyond what’s available in UIElements Vertex (position, tint, uv), if they provide their custom mesh with MeshGenerationContext
What we’re looking for from you
Feedback: Please share your experience, expectations, and workflows ideally through our survey, which helps us efficiently review the collected data. Note that the questions are tailored to the scenarios outlined below.
The survey includes an optional section at the end for those who are willing to be contacted for a follow-up call to discuss their feedback further.
Insights: We’re eager to understand your questions so we can ensure robust documentation for the GA release. Please share them via this thread or include them in the survey linked above.
Consider these scenarios
*Note: If you have specific feedback on these scenarios, please consider filling out our survey, which includes questions about them.
Scenario 1: Adapting a uGUI Shader Graph for UITK
Explore how existing uGUI Shader Graphs can be adapted for use in UITK. Identify any modifications needed and share insights on maintaining visual fidelity with minimal complexity increase.
Scenario 2: Creating a Custom UITK Shader
Create a Shader Graph for UI Toolkit (feel free to use the provided UITK Shader Graph template ShaderGraphTemplate.zip (5.3 KB) attached to this post, which will eventually be available in Shader Graph). Focus on incorporating the Render Type Branch node to apply different shader logic to specific Render Types (e.g., Text, Solid, Texture, Gradients). Once created, apply the shader to a VisualElement via assigning a material in the UI Builder. We’re eager to hear about your experience with the template, the node, and any feedback on the overall shader creation and application process.
Scenario 3: Combining Custom Shaders and Post-Processing Filters
Test the interaction between Custom Shaders and Post-Processing Filters in UITK. We’re interested in your experience with layering effects and whether they render correctly without impacting the rendering pipeline.
Scenario 4: Debugging Shader and Filter Performance and Functionality
Investigate how shaders and post-processing filters perform in UITK. Provide feedback on the tools and workflows for identifying and resolving any performance or rendering issues, especially when using multiple shaders or complex setups.
If you have any questions about getting started, please ask them here. For any bugs you encounter, report them using the “Report a Bug” action in the help menu to ensure we receive the necessary information.
Thanks for your involvement and effort as we work towards our 6.3 GA release!
First off - this is so great to see. As-is the aspect ratio feature.
I’ve just been experimenting with this a little as I stumbled across it but in my case, applying a blur filter absolutely destroys performance - to the point where something must be quite seriously wrong. I’m on 6000.3.0a3 not a5 if that might affect things. A simple mobile app running on my desktop 4080 super goes from a render time of ~1.3ms in editor to >12ms and about 20fps as soon as I add a blur of even 5px to one visual element. Albeit one with some depth of hierarchy that covers fullscreen but I’m testing at <1200x1200 px screen size.
Currently gamma space and to output to render texture (though changing this doesn’t make any difference to performance). I assume this is all a symptom of early teething problems but are there any major things I should be watching out for?
Another small comment is that I noticed if I cover this visual element with another opaque full screen visual element, this doesn’t affect performance either. I suppose this makes sense due to the way it breaks batching but am I right in thinking that without a filter applied, the VE underneath usually would get culled/not rendered? I’m perhaps just wrong about that - either way, is there anything in docs explicitly written about this? Good to know if/when we should be calling style.display = DisplayStyle.None on background elements when they’re covered up.
i’d love if someone answers this “With that, what’s the recommended solution approach for drop-shadow or background blur (or backdrop blur) in UI Toolkit?”
what’s the recommended solution approach for drop-shadow or background blur (or backdrop blur) in UI Toolkit?
For drop-shadow, the most efficient approach remains a mesh-based approach. A parent VisualElement can draw a semi-transparent mesh under the bottom of your element. This is most likely how we would implement it at some point. This approach is fully compatible with the uber shader, so no custom shader is required (i.e. doesn’t break the batch). Also, it doesn’t involve render textures so it is bandwidth-friendly. Some users have already implemented a similar approach, as you can see in this thread. A downside is that it generally requires special code to handle irregular shapes.
If bandwidth isn’t an issue (e.g. a small VisualElement), a custom filter can be used to achieve drop-shadow. The VisualElement is rendered to a RT and a custom filter, with extra margins, post-processes it. By sampling the alpha in the source texture at an offset, you can determine if you’re in the shadow area of the UI. Multiple sampling points can achieve a soft shadow.
For backdrop blur, you must have access to a render texture that contains what is rendered behind your UI. In the case where your overlay-UI is being rendered over the world, you need to keep a render texture of your camera pass around. Next, you would post-process it with whatever blur shader you like and use it as the background texture of your UI. If your UI has a small area, you could apply the camera RT to your element and use a custom filter to perform the blur. However if your UI is overlapping most of your scene or there are many, you should probably post-process the camera rt in a single step. If you have UI over UI and both are transparent, you need multiple layers of render textures and blur processing, which is obviously even more expensive.
It might be a problem with the number of samples. From looking quickly at the blur shader, with a sigma of 5, it seems to compute a kernelSize of (3*5+1)=16, but it’s sampled at every unit from -kernelSize to kernelSize, so it’s 32 taps… and done twice because there is a vertical and horizontal pass, so that would be 64 taps. If your element is also large, that’s a heavy amount of samples. It might be lacking some controls. Do you have a simple repro project that you could submit through the bug reporter?
Another possibility is the size. Atm filters use the bounding box of the filtered element (which includes its descendants) and it might be ignoring clipping (overflow:hidden, if applicable).
Thanks for the quick response @AlexandreT-unity and thanks for the documentation link - I hadn’t seen that and its a nice reference. If you don’t mind spending the time - why is it that opacity 0 doesn’t behave the same as visibility:hidden wrt GPU? I think I’m missing some nuance here - in the rare cases I do want to fade out an element it sounds like I should seriously consider setting visibility to hidden upon completion which is a good learning but I don’t really understand what I’m missing out on by doing that - does it break some things that might be stencilled or something? vaguely waving my hands around, not really knowing what I’m talking about
Re the blur it actually might be a mem leak:
First half of profiler is a build of my app @1080x1920 attached to profiler (steady 335mb)
var blur = new FilterFunction(FilterFunctionType.Blur);
blur.AddParameter(new FilterParameter(10.0f));
myVE.style.overflow = Overflow.Hidden;
myVE.style.filter = new List<FilterFunction> { blur };
.. and make a new build, this is what the second half of the profiler looks like - i.e. severe mem leak (5gb < 10 secs with exp growth). No per frame work. DX11 on Windows.
As you mentioned, visibility:hidden doesn’t guarantee that the descendants are invisible as well. A child could override this style, so the parent still needs to push its stencil mask (if applicable). However, besides the mask, all other meshes and render commands are trashed, so they won’t issue draw calls and the GPU will neither process the vertices or the fragments.
Opacity can be anywhere between 0 and 1 and we evaluate it dynamically in the vertex shader. The vertex reads the opacity that applies to its element and multiplies its vertex alpha. This doesn’t prevent the geometry from generating fragments. We will revisit this eventually since we should skip the vertex ranges or at least move them outside the clip space so they don’t generate fragments (equivalent of the Translate outside of the Viewport in the link above).
EDIT: As part of this bugfix, we are now moving vertices outside the clip space so they don’t generate fragments. In the future refactor we intend to skip the vertex ranges entirely. Also ImmediateModeElements will not render when opacity is 0.
Just got a chance to test - that indeed seems to fix the mem leak thank you.
Blur does still feel a little heavy but I can test with it a bit now.
What’s the right way to change the blur via c#? (I don’t think the doc link covers it unless I missed it)
This is the only thing I got to work which seems wrong - modifying the reference to the parameter, filter, marking dirty on update or even reusing the list that’s assigned to .filter doesn’t work - setting a new list every time seems to be the only way I’ve found (I’m sure there’s a separate bind/uss approach) DOTween.To(() => _blurFilter.GetParameter(0).floatValue, x => _blurFilter.SetParameter(0, new FilterParameter(x)), 20f, 5f).SetLoops(-1, LoopType.Yoyo).OnUpdate(() => _boardVE.style.filter = new List<FilterFunction> { _blurFilter });
Other issues:
I’m getting very strange cropping as soon as the filter is applied
This is a valid way to set the filter, however in a real prod I wouldn’t recommend to create a new list every time. You can also assign the filter with a style. In any case, USS transitions should allow smooth transitions between the previous and next value.
After some debugging I found an easy repro - Reference DPI of Panel Settings. Mine was 150 - at the default of 96 things look as expected. Also this bug applies to any filter, not just blur.
Hmm that’s fine but I have reasons for not wanting to use USS transitions so what would you recommend for the c# production approach? Or, if there isn’t a good one - please add that to the feedback list from me. The rest of UITK has c# control in mind.
It’s also very counter-intuitive that references to FilterParameters & FilterFunctions are somewhat dead references so I think that will trip more people up than just me. Thanks for your help, looking forward to fixes and improvements.
Limited support] Previewing runtime shaders in Editor windows may appear somewhat unpolished. Additionally, shaders that rely on “time” will not render accurately in the Editor, as it lacks a runtime concept of time. In the Builder, runtime shaders might display outside their intended boundaries if the container utilizes custom stencils, such as those for rounded corners.
Not supported] Custom shader masking as custom shapes is not possible. However, vector image masking is supported.
.
