[Official] Feedback wanted: How to make shader loading faster / less hiccups

Unity Engine
,
1

In addition to today’s “changing compiled shader format” thread, here’s another one with ideas / thoughts about shader loading performance.

Basically, shader loading is slow. And sometimes instantiating something at runtime with a new shader causes hiccups as well. A good question is, how to solve that?

We’ve looked at shader loading, and it seems that on the platforms where the problem is the worst (mobile), almost all time is spent not in our code. We can change our own shader serialization format (binary instead of text) and optimize loading code, but the truth is, 95% of time is spent in just glCompileShader + glLinkProgram. There’s not much we can do here, except to “load less shaders” or “load shaders later”.

So some ideas:

  • Reduce number of shader variants that are included into game to begin with (some UI somewhere to specify what kinds of lights/shadows/… you’ll need etc.). For 5.0 we’ve started doing this with lightmapping modes; soon with fog types; could also add more options like that.

  • This saves both game building time, game data size, makes loading faster and runtime memory usage smaller. The downside is, adds new settings somewhere, and if you get them wrong then some head scratching can occur.

  • Delay loading of shader variants until they are actually needed. Currently, typically 50-80% of shader variants are never needed in the game.

  • Can be a net win if there are many actually unused variants.

  • However, for variants that are actually used, this delays “load time” until “rendering time”, i.e. you just changed longer load time into a hiccup on first render.

  • Shader.WarmupAllShaders could probably be extended into “something better”. Good question is - to what?

  • Spawn some background thread and/or timesliced shader loading? Or some other approaches.

What’s your shader loading performance?

To understand the problem better, it would help to know what are your shader loading problems, and especially to know the numbers. For example:

  • Which platforms you have shader loading problems on? Hiccups?
  • Is it more important to load shaders faster, or to avoid hiccups at runtime?
  • How many shaders do you typically have, and how many variants they have internally?
  • Do you use Shader.WarmupAllShaders, and is the functionality ok for you? What you would like instead?
  • When you have hiccups at runtime - is that because an object with a whole new shader came into view? Or just one of internal variants that was not used before? Does it happen on first play of the game, or on later runs as well?
  • etc.

Feedback welcome!

2

  • Hiccups on iOS. Avoiding them by warmup if possible. (I don’t care about load times) if not possible to warmup (crash) then I force them to render “hidden” at startup manually.

  • I use custom shaders only, mostly IBL shaders with many textures. Variants are usually set for quality per device only and does not change after startup except those that create the environment (sky, water). I prefer to give an object it’s own slightly customized shader variant (even for the LODs) if necessary instead of writing uber-shaders and I don’t use cgincludes to keep control over optimization. I would say something like 50 customized shaders (maybe more) and if then maybe 6-10 variants. I don’t use any instances after startup at all (pool everything) and I don’t load scenes.

  • Shader.WarmupAllShaders currently crashes the game (black screen without recover) on anything below iPad Air (but I still have some deactivated old rubbish in my scene) - a warmup of specified Shaders and Materials would be good instead of doing this kind of global.

  • Hiccups usually only occur when the material comes into view for the first time - happens sometimes even if it is the same shader but used on a different material. Switching variants only happens in my project when the user completely switches “screen content” so if there is any hiccup then I haven’t noticed it yet.

3

Great :slight_smile: I’m working on Ori and the blind Forest. We have our own shader permutation system (with our possible permutation space, billions, multi compile just wasn’t viable). A full game build today contains about ~1500 shaders. Almost all shaders have 3 variants in them (multi_compile)

The are 3 main problems we’ve encountered:

  1. Loading shaders at runtime causes hiccups, on almost all platforms, we’re always close to 16 ms. We of course do shader prewarming shaders, but:

  2. Shader.WarmupAllShaders only warms up the shaders that are currently loaded… That means that we have to do some voodoo build magic to actually load all included shaders in the first scene, warm them all up, and then keep them loaded. That leads to:

  3. Loading all shaders in memory, especially in the current format (as you mentioned in the other post), becomes quite huge. In the orders of ~50MB, which, on 360, is as much as our graphics budget. Additionally, warming up 1500 shaders really bloats game loading times

Here are my thoughts on a direction to take here

  1. (option to strip out variants manually)

Reducing the number of variants is crucial to making shader_feature useful for more than simple stuff. The exponential nature of variants means that it will always explode the amount of variants. Additionally, often some combinations aren’t even valid, as in, not compiling. Having some kind of matrix similair to the physics layers would solve that. How that would look is tricky - per shader, stored in the .meta file could work. Either way, it’s the only way to keep shader_feature managable.

2 / 3: (Delay loading of variants, rework Shader.WarmupAllShaders, timesliced loading)

I would propose a system where, when a material gets loaded (so not when it’s rendered, but when say a scene gets loaded) it should warm it’s current shader variant (or, current + a few light modes). When someone changes keywords at runtime it would mean warming a variant at render time, but I feel that that is much less frequent.

When calling LoadSceneAdditiveAsync, it would mean warming the shaders time sliced as the materials come in (I realize that this isn’t how it actually works, but conceptually speaking). This would be on the loading thread so it wouldn’t interrupt gameplay.

Ideally shaders would not just unload when they’re no longer referenced, but also unload variants that haven’t been used in a while.

  1. (the other forum topic)

Yes, please change that compiled shader format :slight_smile: I’m 100% for it, no real downsides, major upsides for us. Less memory, less parsing.

Also, it seems that a build for a platform only should have the compiled version for the target platform. Or is there some deeper reason for that? (fallbacks maybe?)

Really excited to see these problems starting to get traction, hopefully they can be resolved :slight_smile: If you need any more info let me know!

4

But (in 5.0) #pragma shader_feature is already only including variants that are used by materials into the build.

(shader_feature does not exist in 4.x)

Yes, that’s what should be happening right now (or well, in any Unity version). If you build a game for XboxOne, it only includes XboxOne shaders. If you see otherwise, that’s a bug.

5

A heads up on the current plan/thinking:

  • User option for shader loading behavior: a) no preloading, b) automatic profile-guided, c) full preloading. (Unity 4.x is c; default should be b).

  • Shader variants that are not preloaded will be loaded when they are needed for rendering. So best to have everything that you need to be preloaded, but nothing “catastrophically bad” happens if they load just in time.

  • Add asset type, something like ShaderPreloader. Basically a list of shaders & subshader+pass+keywords in each.

  • Scripting API to preload everything in the ShaderPreloader asset, at two possible levels: 1) load shader variants, and 2) load them and issue dummy draw calls to actually make sure they are fully loaded by the driver. This is like Shader.WarmupAllShaders, done better.

  • Simplest option is to have it be synchronous.

  • Could also be timesliced and exposed as coroutine - you tell it how many milliseconds per frame to use up, and then can yield on it.

  • Editor tracks “built-in shader preloading list” automatically, and that’s used by default in the “automatic profile-guided” case.

  • Button somewhere to turn so-far-recorded preloading list into a new ShaderPreloader asset.

  • Some sort of inspector for these shader preloading assets.

  • Do not unload actual hardware shaders (recreating them is fairly expensive - but do unload Unity side of shader stuff, since reloading that is fairly cheap).

  • Do provide some scripting API to unload unused hardware shaders, if needed.

  • Or perhaps manual call to Resources.UnloadUnusedAssets should also unload hardware shaders?

6

Don’t mean to hijack thread but:
How does #pragma shader_feature work and how does it differ from #pragma multi_compile ?

1 Like
7

They work very similarly, but unused (by any materials) shader_feature variants are not included into the game build at all.

So pretty much, shader_feature: for things that will be controlled by materials; multi_compile: for things that will be controlled outside of materials (global shader keywords, etc.).

8

Thanks for the explanation and looking forward to using shader_feature in 5.0

9

Shader.WarmupAllShaders makes the game to pause for sometime…Any Help???

10

You’d have to wait until I implement the plan outlined above. Which I have done for upcoming Unity 5.0.

In the meantime, either do not use WarmupAllShaders, or manually make shaders smaller. E.g. if you know you don’t use lightmaps in surface shaders, add “nolightmap” to the #pragma surface line; same for directional lightmaps, deferred shading, etc. etc. A problem with WarmupAllShaders, is that it does exactly that - it warms up all shaders, and all variants in them.

11

Thanks Aras
But i didnt get the idea of making the shaders small.can you post an example?

12

This mostly applies to surface shaders. By default they generate code for “everything that could possibly be needed”. Lightmaps, directional lightmaps, deferred lighting etc.

If you do not use directional lightmaps in your game, for example, then there’s no point in generating shader variants for them (and then loading them during game play). So in the surface shader, you could add “nodirlightmap” to the #pragma surface line to not do that - see Unity - Manual: Writing Surface Shaders - (and again, in 5.0 most of that will be done automatically).

13

Thanks again. Aras
Can you suggest me tutorials to start with shader scripting.Wll be a great help.

14

http://en.wikibooks.org/wiki/Cg_Programming/Unity

15

Hi, I am trying to build an uber shader in a whole game project, with many shader_feature control them.
But I when I use about 18 different shader_feature, the unity become almost unuseable, and when it loading, it take about more than 8G memory.
Am I use the wrong way?

16

It’s probably just a ton of different shader variants.
Following the example below, a shader with 18 multi_compile features with two options each feature, should generate 2^18 = 262144 variants in total.

Copied from the documentation:

17

I think unity can optimise this behaviour by create actual shader when it needed, and cache them in the Library folder.
Just like the Gamebryo shader tree or something like, keep an hash map with the compile keyword in the memory, each time need a shader, lookup the table, it there is nothing existed, try load from cache on the disk, if the disk cache is not exist, then create a new one.

18

Hi, I’m currently looking into this exact issue, laid out by Aras here:

Is this the case? Do shaders get unbound from gpu when calling UnloadUnusedAssets?
(Using Unity 5.4.1p4 atm).

Thanks!