Improvements to shader build time and runtime memory usage

Unity Engine
,
21

Variant prefiltering will be available in 2022.2.0b15. 2021.3 to follow :slight_smile:

7 Likes
22

This appears on 2021.3.13. Is this it?

8587414--1150993--upload_2022-11-15_7-20-29.png

Also, will this setting allows us to use multiple HDRP assets without expecting several gb of memory usage for HDRP Lit?

23

No, thatā€™s dynamic variant loading :slight_smile:

Highly likely, yes.

2 Likes
24

why canā€™t unity come with precompiled shaders for the most common usages?
seems like reinventing the wheel each time anyone does a build

25

It will also be available in 2021.3.15f1.

6 Likes
26

Any ideas about a potential ETA?:slight_smile:

1 Like
27

I no serious issues are found, it will be available around Nov 30.

6 Likes
28

So exciting to hear.

1 Like
29

Any news @aleksandrk ? :sweat_smile:

30

Looks like 2021.3.15f1 is available since yesterday.

3 Likes
31

Just installed the update (2021.3.15f1) and so far it has been an improvement.

Cold build which was basically the first ever build done on the new version was an expected 6 hours, somewhat of a large game and itā€™s expected for a new version to be doing a large build on first go. If I remember correctly, the version I was on before this one took me a total 8-12 hours to do my first build.

Second build it was also an improvement, from my usual 15+ minutes, itā€™s gone down to 7 minutes. Again, Iā€™m unsure if this was an expected result the engineers wanted, but this is me not really fine tuning everything, it was just an installation and seeing how things went.

Iā€™m unsure if I was really supposed to change much, but to me itā€™s a decent improvement anyway. Thanks again for listening to everyone here, and I appreciate the support.

2 Likes
32

It seems like 2021.3.15f1 is back to the pre-crazy build time levels. Back from 1.5-2hrs to 37m for cold build, and for warm builds, went from 36m to 8m.

8 Likes
33

2021.3.15f1 brings warm builds down to ~14 minutes for us. This is still significantly slower than 2020 LTS (currently at ~4 minutes), however we have no choice but to take it.

StripUnusedMeshComponents option has a larger impact in 2021 than 2020. Disabling it further reduces warm builds down to under 7 minutes.

3 Likes
34

Unity 2020.3.40 + HDRP

  • With one referenced HDRP Asset, most features unticked: 150mb runtime memory usage

  • With one referenced HDRP Asset, most features on: 500mb

  • With three referenced HDRP Assets (low, medium, high): 2gb+

  • Initial build time took a long night, more than 10 hours

  • More details in this thread

Unity 2021.3.15 + HDRP

  • With one referenced HDRP Asset, most features unticked: 150mb

  • With one referenced HDRP Asset, most features on: 150mb

  • With three referenced HDRP Assets (low, medium, high): 320mb

  • Initial build time took around 3 hours

Not sure when exactly the steps of progression occured, as we did notice runtime memory gains when jumping from Unity 2020 to 2021, but eh! Thatā€™s some major progress anyway.

Thanks to everyone involved in this at Unity, this is really nice.

12 Likes
35

Just to add: Iā€™m also seeing slightly shorter build times.

2 Likes
36

Btw. since ā€œcoldā€ times are so huge, would it not be worth it to memory-dump all that thatā€™s computed and save it on drive for reload?

37

@DragonCoder How would that help with cold build times?
Cold build = no shader variants have been compiled. When we compile a variant, we store it to the disk.

1 Like
38

After upgrading from 2021 3.6f1 LTS to 3.15f1 with the latest URP version and deleting the Library folder, the cold build (I assume it means the build into an empty folder without cashed shaders?) now takes more than 2 hours, creating from 1,5k to 3k variants for each custom shader (and the default Lit shaders for URP, which I shouldnā€™t even use in the project, really), in a project with quite a few custom shaders. Is this normal? Iā€™m using Amplify Shader Editor and Addressables, wonder if it could affect the build times. I donā€™t really notice any speed gain in builds, is there something wrong with my project?

The next build without any changes is lighting fast, less than a minute.

Then I slightly edited 2 shaders made in Amplify Shader Editor and the next build took 25 minutes (worked mainly on those 2 shaders, each needed 3k variants)

Then I slightly edited just 1 shader made in ASE, but also unticked some checkboxes for different platforms that I donā€™t use (DX9, Switch, PS4, XBoxes, OpenGL, etc, just left DX11 for now). After that the build took ~12,5 minutes, which is roughly twice as fast as the previous build where I edited 2 shaders. So it looks like ASE is not the culprit, as unchecking the boxes didnā€™t speed up the shader compilation time (I havenā€™t upgraded ASE for the most recent version yet, though, because they disabled the Undo feature for some reason and Iā€™m waiting for it to be resolved).

Only after that I decided to check if there any new settings for variant prefiltering, and I found none.

This is quite a long buildup for each new or edited shader, and it promotes testing the game in the editor instead of in builds and itā€™s quite dangerous in my experience.

Currently the source project is 4,5GB with the Library folder and 1,5GB without it. The build is 253MB.

2 Likes
39

Hey, I already saw a few forum posts about Variant Keyword Prefiltering, but no examples/doc on how we can use it, or maybe I missed it, or donā€™t fully understand it :slight_smile: Is it applicable for keywords in the shader directly? Can we use something like this?

#pragma shader_feature AWESOME_FEATURES

#if AWESOME_FEATURES
   #pragma multi_compile _ RED GREEN BLUE WHITE
#else
   #pragma shader_feature RED
#endif
40

Hi there,

We got some really bad build times here too, over four days to build the entire game, but since 2021.3.15f1 with Variant Keyword Prefiltering, for the second build itā€™s very similar to before on built-in.

For those who are still wondering how the Variant Keywork Prefiltering works, here is the keynote for the Unity 2021.3.15f1 version that explained it for me:

Editor: Enabling shader keyword pre-filtering so that build process does not have to enumerate through full shader variant space. This fixes the issue of URP builds even with warm shader cache taking really long time. This fix does not reduce the time spent on actually compiling shaders.

I also had a confirmation from Unity that this is automatically done and there is no need to do any setting about it.

Iā€™m still looking how we can reduce the first build time because we need to build for PC, Android and PS4 (soon for PS5) and this is painfull. Also, updating the URP Package or Unity version makes the cache go away and the build starts from scratch again. A minor change on a shader also takes hours to build it again, with about 40k variants on PC.

Iā€™m looking forward to some understandable ways to reduce this variants if already possible.

Thanks,

2 Likes