Shader Stripping Improvements in URP

[LukasCh]

Shader Stripping Improvements in URP
In recent years, URP has grown and introduced a lot of amazing features. This resulted in a huge amount of shader variants to support those features.

To ensure that these variants don’t add to build times, file size, and runtime memory usage, we have expanded URP’s shader stripping capabilities.

In 2021.2 we are introducing a few new features that will be covered in detail.

Stripping unused feature variants
Many features in URP require “enabled” and “disabled” variants of prebuilt shaders, depending on whether the feature is enabled or disabled. URP already performs some automatic stripping of these variants, but we have improved this from version 2021.2 onwards.

In all versions of URP, if a feature is disabled in all URP Assets included in your build, URP strips the “enabled” variants and compiles only the “disabled” variants.

In versions prior to 2021.2, if a feature is enabled in any URP Asset included in your build, URP compiles both “enabled” and “disabled” shader variants.

From 2021.2 onwards, you can enable Strip Unused Variants in the URP Global Settings. When this setting is enabled, if a feature is enabled in all URP Assets included in your build, URP strips the “disabled” variants and compiles only the “enabled” variants. If this setting is disabled, URP behaves as before.

Here is the list of features that this applies to:

• Light Layers
• Render Pass
• Reflection Probe Blending
• Reflection Probe Box Projection
• Forward+
• SSAO
• Decals
• Main Light Shadows *
• Additional Light Shadows *
• Additional Lights *

• From 2021.2, we have changed some keyword functionality for this feature, and how URP determines whether this feature is enabled in your project. More details below.

Keyword changes
In versions of URP prior to 2021.2, URP enables and disables the shader keywords for Main Light Shadow, Additional Light Shadow and Additional Light based on the configuration of the current scene. This means that URP can switch between the “enabled” and the “disabled” variants for different scenes.

From 2021.2, URP enables and disables these keywords based on settings in the URP Asset. This means that URP uses either the “enabled” or “disabled” shader variant as long as the current URP Asset is in use, rather than switching per-scene.

This change allows URP to strip unneeded variants when Strip Unused Variants is enabled.

Note: if Main Light Shadow or Additional Light Shadow are enabled in the URP Asset but there are no shadows to render (i.e. no visible lights in the scene cast shadows, or the camera has no visible shadow casters), then URP samples a small placeholder shadow map texture.

We felt this was a less common case, and the overall performance change is quite small. Keep in mind that you can disable Strip Unused Variants to retain the previous functionality.

Stripping unused Quality Settings variants
You can assign different URP Assets to different Quality Settings levels. At build time, URP examines these URP Assets to determine which URP features are enabled or disabled in your build, and therefore which variants to compile or strip.

In versions of URP prior to 2021.2, URP examines every URP Asset that is assigned to a Quality Settings level, regardless of whether that Quality Settings level is enabled for the current build target. To avoid including unneeded variants in these versions of URP, you must ensure that unneeded URP Assets are not assigned to a Quality Settings level at build time.

From 2021.2, URP only examines URP Assets that are assigned to Quality Settings levels that are enabled for the current build target.

Stripping unused post-processing variants
In URP 2021.1, we added an option to strip all post-processing shader variants from your build. In URP 2021.2, we introduce the option to strip post-processing shader variants that are not used by any Volume Profiles in your project. For example, if you only use Bloom in your project, all other unneeded post processing variants and shaders will be stripped.

To enable this feature, go to URP Global Settings and enable Strip Post Processing. Note that this feature works by checking all Volume Profiles in the project - for now it does not check if it is actually used in a scene or not.

XR/VR Stripping
If you do not plan to use XR/VR in your project, you can disable XR and VR modules (How to disable a Module). This allows URP to strip XR and VR related shader variants. This functionality has not changed in 2021.2, but it is a useful tip.

“Small” Project
Testing small project with enabled features:

• SSAO
• Decals
• Reflection Probe Blending
• Reflection Probe Box Projection
• Bloom

Results from Windows Player D3D11 build:

--------------------- Before ------- After
Build Time ------ 2min 27s ---- 28s
Variant Count – 4407 --------- 944
Build Size ------- 70.2MB ----- 69.4MB

Template Project
Testing changes with URP Template project.

Results from Windows Player D3D11 build:

--------------------- Before ------- After
Build Time ------ 20min 14s — 7min 12s
Variant Count – 7964 ---------- 2277
Build Size ------- 82.4MB ------ 78.6MB

Boat Attack Project
Testing changes with URP Boat Attack Demo.

Results from Windows Player D3D11 build:

--------------------- Before ------- After
Build Time ------ 58min 57s — 39min 19s
Variant Count – 22890 -------- 11895
Build Size ------- 495MB ------ 492MB

[rz_0lento]

[quote=“LukasCh, post:1, topic: 861031, username:LukasCh”]
In URP 2021.1, we added an option to strip all post-processing shader variants from your build. In URP 2021.2, we introduce the option to strip post-processing shader variants that are not used by any Volume Profiles in your project.
[/quote]Any chance of giving some “always include” override options for the PP stripping? For example right now if you want to use URP PP stripping and custom AO solution that still feeds into URP’s Lit shader, only ways to not have the required parts stripped away is to either disable PP shader stripping, modify the URP package itself for this or enable built-in SSAO but set it’s intensity to 0 and make it run before your real AO solution (this is quite hacky IMHO). Ultimately you’d want this system to be flexible enough to not forcing users to modify the URP package.

As another topic, I’ve mentioned this few times in other related threads: it would be nice if URP and HDRP shader strippers could coexist on same build. I don’t know what exactly changed on 2021 cycle for this because this used to work in past Unity versions (or at least build times used to be very reasonable). Now if I try to make this kind of setup work it either results on one of the SRPs shaders getting fully stripped out or alternatively having one SRP including all it’s shader variants - which obviously can’t work as it would take forever to build anything like this.

[LukasCh]

@rz_0lento

Any chance of giving some “always include” override options for the PP stripping? For example right now if you want to use URP PP stripping and custom AO solution that still feeds into URP’s Lit shader, only ways to not have the required parts stripped away is to either disable PP shader stripping, modify the URP package itself for this or enable built-in SSAO but set it’s intensity to 0 and make it run before your real AO solution (this is quite hacky IMHO). Ultimately you’d want this system to be flexible enough to not forcing users to modify the URP package.

Thanks for feedback. Yes this is something we will look into in the future. Overall If I understood correctly you want SSAO variants in lit shader not to be stripped as your custom SSAO would supply texture there. I am curious if exposing this as option from etc renderer “require SSAO variant” would suffice in your scenario?
As for now I think the other workaround would be just have another renderer in urp asset which uses our builtin ssao. This way you will still able to have all benefits of stripping and also not pay the price of bultin ssao.

@rz_0lento

As another topic, I’ve mentioned this few times in other related threads: it would be nice if URP and HDRP shader strippers could coexist on same build. I don’t know what exactly changed on 2021 cycle for this because this used to work in past Unity versions (or at least build times used to be very reasonable). Now if I try to make this kind of setup work it either results on one of the SRPs shaders getting fully stripped out or alternatively having one SRP including all it’s shader variants - which obviously can’t work as it would take forever to build anything like this.

Yes, we have initiative to address in general URP and HDRP compatability in the same project. Which should also address this issue, sadly I do not have anything concrete about it yet.

[rz_0lento]

[quote=“LukasCh, post:3, topic: 861031, username:LukasCh”]
As for now I think the other workaround would be just have another renderer in urp asset which uses our builtin ssao. This way you will still able to have all benefits of stripping and also not pay the price of bultin ssao.
[/quote]Oh right, didn’t see this alternative at all. I like this workaround more than just having the SSAO on your main data asset.

I don’t have a good idea on what would be the ultimate solution for making this type usage cleaner, but I can imagine as URP grows more there will be other things beyond SSAO that users may want to bypass with some custom solution as well.

[quote=“LukasCh, post:3, topic: 861031, username:LukasCh”]
Yes, we have initiative to address in general URP and HDRP compatability in the same project. Which should also address this issue, sadly I do not have anything concrete about it yet.
[/quote]I don’t expect it to happen over night but it’s nice to hear that it’s at least on some backlog.

As another note on the shader stripping: I’d love to have some kind of visual indicator on the unity inspector per shader if it was actually included or stripped away on previous build. Like just small note when you look at the shader telling it wasn’t included on previous build etc. This would mainly be useful for custom shaders that may get stripped accidentally so wouldn’t need to add them to that always included shaders -list “just in case they might get axed”.

[Thermos]

Alongside automatic stripping, IPreprocessShader interface is a powerful tool to strip unwanted shader variations.

Would love to see a built in tool (or user interface) that could show all shader variations in previous build and let users to decided which variations are unwanted so the get stripped away in the after builds.

[fherbst]

Just want to chime in and note that “stripping is based on project content” is kinda dangerous, since often Packages or AssetStore content ships with scenes, post processing volumes, …

Would be great if there was a visual post-build overview of “we included this and that variant because we think you’ll need it based on this (pingable) asset”.
Additionally, would be great if an alternative to this would be to specify an explicit list of assets to be used (e.g. just because I’m using post processing in a non-build test scene doesn’t mean I want it to be in my build).

[LukasCh]

@fherbst

Just want to chime in and note that “stripping is based on project content” is kinda dangerous, since often Packages or AssetStore content ships with scenes, post processing volumes, …

Yes, this is first step. We plan to have in future to do it based on which profiles are referenced in scene.

[burningmime]

This is probably a similar complain t to @rz_0lento 's but I have a custom shadow solution (stencil shadows), which I am injecting into the screen space shadow map and then enabling in a render pass by calling cmd.EnableShaderKeyword("_MAIN_LIGHT_SHADOWS_SCREEN";);. I struggled for a bit to get this working in a build because the shader was being stripped. The solution was to create a separate URP Renderer called “DO NOT USE” with the normal ScreenSpaceShadowmapFeature, include it in the build under Quality Settings, and then never use it. But that seems very hacky.

I’d love for there to be some way to force shader variants with a certain keyword to be included even if the shader stripping thing does not detect that it will be used.

[tatoforever]

Unity 2021.2.2f1 and URP12.1 Bug case: 1381759

Build fails after hours trying to compile a custom URP Lit shader (of 900 000 variants).
Worth mention the memory leak is umbelivably high (~25GB of ram) while compiling variants. Btw nothing gets cached, if I start a new build the whole variant compilation process start from zero.

[tatoforever]

Still happens in Unity 2021.2.4f1:

Still can’t build build always fail, editor crashes on a 64GB ram system. Doesn’t crash on 128GB ram system but give a bunch of IL2CPP errors and all shaders contain compile errors too.
Worth mention that in the 128GB ram system the shader compilation is started again from zero (nothing gets cached) despite successfully compiling all shader variants for hours.

[aleksandrk]

@tatoforever

nothing gets cached

Please check the Editor log, this is the source of truth for what has been cached and what hasn’t been. The progress bar always says “Compiling shader variants” and always starts over, but that doesn’t mean the compilation results haven’t been taken from the cache.

[tatoforever]

@aleksandrk

Please check the Editor log, this is the source of truth for what has been cached and what hasn’t been. The progress bar always says “Compiling shader variants” and always starts over, but that doesn’t mean the compilation results haven’t been taken from the cache.

In the 128GB system, after few hours compiling my shader variants the build just fails with a bunch of IL2CPP errors. I then tried to restart the build and the whole shder variant compilation, starts from zero once again.
The 64GB system can’t event get pass half of my shader variants compilation build because Unity crashes after reaching 60-61GB of ram trying to compile my shader.

[momings]

We also met build crash after upgrading to 2021.2.6 from 2020.3. It seems unity ran out of memory.
OnProcessShader and ShaderVariantCollection is useless because too many variants be prepared.

Editor logs:

Compiling shader "Universal Render Pipeline/Lit" pass "ForwardLit" (fp)
[6.75s] 100M / ~503M prepared
[13.66s] 200M / ~503M prepared
[18.39s] 300M / ~503M prepared
[27.64s] 400M / ~503M prepared
[32.40s] 500M / ~503M prepared
94371840 variants, prepared in 32.56 seconds, starting stripping...
Obtained 0 stack frames.

[burningmime]

It should at least warn you if you’re going over even like 1000 (maybe 10k) variants of one shader. There’s no call for that.

[aleksandrk]

I have a fix for OOM in the queue: Unity Issue Tracker - Shader stripping crashes when it gets too many variants

[Kevin-VFX]

@LukasCh

Here is the list of features that this applies to:

• Light Layers
• Render Pass
• Reflection Probe Blending
• Reflection Probe Box Projection
• Forward+
• SSAO
• Decals
• Main Light Shadows *
• Additional Light Shadows *
• Additional Lights *

Did I miss something, when did Forward+ get added?

[burningmime]

@Kevin-VFX

Did I miss something, when did Forward+ get added?

You didn’t; it’s still in progress.

[Anthelmed]

Hello,
I just did a quick test and I build an empty URP project and it seems that the post processing Film Grain textures are included by default even if I don’t use the post processing pass.

Unity v 2021.2.8f1, URP v 12.1.3

[jimmikaelkael]

@LukasCh

Thanks for feedback. Yes this is something we will look into in the future. Overall If I understood correctly you want SSAO variants in lit shader not to be stripped as your custom SSAO would supply texture there. I am curious if exposing this as option from etc renderer “require SSAO variant” would suffice in your scenario?
As for now I think the other workaround would be just have another renderer in urp asset which uses our builtin ssao. This way you will still able to have all benefits of stripping and also not pay the price of bultin ssao.

Any news about this?
Now that URP does deferred the workaround only works at the condition that rendering path is the same on both renderers.
Also not having such option makes it hard to distribute custom SSAO solution on URP because many customers requiring support for no AO in builds…

[victorhs_studio]

What is the point in compiling shader variantes that will be striped later?

if Variante.goingToBeStriped
Variante = DontEvenCompile_NextPlease();