Unique Value - Generating a hash for UnityEngine.Shader object

Hello everyone,

I have a modding system that allows people to upload anything they want through asset-bundles. Most of this I can sanitize or whitelist at runtime while importing, but Shaders have been troublesome.

I need a way to uniquely identify a shader by a hash value so that I can keep a white list of allowed shaders. Information on how to do this for built-in or URP is fine. Ideally the hash should be calculated from the shaders contents and not the name.

As far as I understand the UnityEngine.Shader Object is a proxy object that does not contain enough information to uniquely identify it beyond its name.

For now my solution is to create a hash based off the asset-bundle itself, but if anyone knows anything about uniquely identifying shader objects, help on this would be greatly appreciated.

Thanks!

I figured out you could create a new Material and call. Although I wonder how reliable this is for my purposes, the docs on this are lacking. Anyone know anything about this?

Material.ComputeCRC

I don’t know about the method specifically, but i just checked the code and part of the CRC is the instance ID of the shader, which I think will be different on each load/run. Which would mean this wouldn’t be suitable for your needs, I think.

Apart from that, the method looked pretty decent for your use case :frowning:

@richardkettlewell Hey thanks for looking into this!

I figured something out for the built-in render pipeline. I start CgBatch process that compiles my shader into streaming assets. I load that at runtime and pass the string source to new Material(shaderFile). Now I can compute a hash on user loaded shaders. Although this looks like it could be a problem in future versions of unity.

[Obsolete("Creating materials from shader source string is no longer supported. Use Shader assets instead.", false)]
        public Material(string contents);

This material-ctor is not listed in the documentation and in code its marked as obsolete. Is this being removed in unity 2020.1? If it is being removed does this mean you will expect that we load shaders from Asset-Bundles only? I really hope that this is not the case.

I’m looking into how SRP works https://github.com/Unity-Technologies/Graphics/tree/master/com.unity.render-pipelines.universal/Runtime I have thought about creating a pass that adds support for legacy shaders or maybe just a way to load user supplied shaders and compile the source at runtime. Although I am still very new to the custom render pipeline. Any advice on this or small pointers in the right direction would be appreciated.

If I can be assured that unity will not be removing public Material(string contents) ctor in the near future I might stick with built-in pipeline if it is being removed in unity 2020.1 I will need to figure out how to do this in URP.

Thanks.

Compiling shaders at runtime isn’t supported on a bunch of platforms, and is quite an old-fashioned way to do things, which is probably why it’s obsolete. It could in theory disappear in a future release, but I don’t know which one.

Do you have the shader source then? Can you just hash that?

@richardkettlewell Unfortunately its a user modding system that allows you to supply the shader at runtime, so I do not know what the source will be ahead of time. The compilation will be done in the SDK (unity editor with tools).

Its to my understanding the the material-ctor doesn’t do the compilation but its the precompiled shader string that is given to it as a param.

“Unity’s default behaviour is to postpone the compilation / optimization of a shader loaded at runtime until it appears on screen” here

I am a little confused why loading shaders in at runtime always need compilation / optimization. I figured that Unity compiles them before loading them into a asset-bundle and thus all necessary variants for the target platforms are already cached or maybe unity player creates a UnityShaderCompiler process when loading asset-bundles that have custom shaders. here

Do asset-bundles even contain compiled shaders or is it plain source code text encrypted?

Shaders are almost universally compiled in the editor (only old OpenGL is an exception).

Most platforms don’t even support a way of compiling at runtime - it’s impossible. But I guess we support it on OpenGL and DirectX 11 (Just a guess)

The hitch in that linked thread is a bunch of Unity work to supply the compiled shader object to the platform backend. It isn’t (usually) compiling the shader.

If the modder compiles the shader in the editor “with tools” … does that mean with your tools? If so, perhaps add something to your tools that computes the hash of the shader source code, and stores as a label in the shader asset: Unity - Scripting API: AssetDatabase.SetLabels Then you could get the hash when you load the shader?

1 Like

@richardkettlewell Thanks! that makes a lot more since now.

I guess what I really need is low level access. the labels is a good idea and I would be able to use Asset-Bundles, but anyone could change that to push malicious shaders to a server, so the server is what really needs to be computing the hash, because its hash validation.

I might be left with having to let go of hash validation and just let users push what ever shader code they want… Well that’s not good. It would be very easy to crash people.

Just ripping out the binary shader component in the asset-bundle from file and computing a hash on that would work fine, but asset-bundles are proprietary. Maybe a feature request? I feel that it would makes since there is enough left open on the UnityEngine API to re-implement the loading of an asset-bundle (if you really wanted to), because there are times when you might need specific features that asset-bundle does not provide like computing hashes of each individual resource or component, and for some it’s worth re-implementing it all. I think that requiring to do everything through asset-bundles is simplistic but its also limiting.

I am also supporting android (crap). I need to check if I can do new Material(string) ctor on that. If this is really all that can be done, then I guess I will have to make do with what I have and hope that a new version of unity adds support.

I do appreciate all the help, I understand this is a unusual thing to be doing in unity-engine.

You probably can, because it will be OpenGL ES. But not if you use Vulkan for Android.

Yep you could do that. I guess if/when we make a change to the shader format, the hash would change though. Really, hashing the original source code is the only robust way I can think of.

Even if we let you compute a hash on a shader object at runtime, it has the same problem - if a new Unity version adds a new shader property, it would become part of the hash, meaning your hash would change between versions. The source code of the shader seems like the only thing you can truly depend on.

1 Like