GLTFast shaders in PolySpatial

Unity Engine
, , ,
1

I’m trying to add GTLFast to my basic Unity PolySpatial project.

GLTFast provides URP Shader Graph shaders. I’ve integrated GLTFast using unity package manager. I added a new node to my scene, added the Gltf Asset MonoBehaviour and set the Url to https://github.com/KhronosGroup/glTF-Sample-Models/raw/master/2.0/2CylinderEngine/glTF-Binary/2CylinderEngine.glb. I also set the scale of the GameObject to 0.001 so that the model displayed at a reasonable scale.

This works just fine in play mode, rendering correctly.

I then went to Project Settings → Graphics and then under Currently tracked: clicked Save to asset.... I then added that asset to the list of preloaded shaders.

When I build for VisionOS and run in the simulator the model geometry shows, but is rendered in hot pink. There is logging in the Xcode console:

[Platform] [PolySpatialUnitySimulation] Could not load ShaderGraph material 'Shader Graphs/glTF-pbrMetallicRoughness' from file:///Users/Casey/Library/Developer/CoreSimulator/Devices/01679CEB-B0B6-4301-921F-CCF8963C6C4A/data/Containers/Data/Application/F482849A-EB69-46A3-BED8-1359A58516C1/tmp/F48EF0DC-E6FC-4343-AE6F-E56803F02734.usda: invalidTypeFound

I then went back to Unity’s Project Settings → Graphics and added gtlf-pbrMetallicRoughness to the list of Always Included Shaders.

I then build for VisionOS and see this warning in Unity’s build log:

Shader warning in ‘Shader Graphs/glTF-pbrMetallicRoughness’: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them at /Applications/Unity/Hub/Editor/2022.3.6f1/Unity.app/Contents/CGIncludes/UnityCG.cginc(513) (on metal)

Compiling Subshader: 1, Pass: BuiltIn Forward, Fragment program with DIRECTIONAL SHADOWS_SCREEN STEREO_INSTANCING_ON _ADDITIONAL_LIGHTS _MAIN_LIGHT_SHADOWS_CASCADE _SCREEN_SPACE_OCCLUSION _SHADOWS_SOFT
Platform defines: UNITY_ENABLE_DETAIL_NORMALMAP UNITY_ENABLE_REFLECTION_BUFFERS UNITY_FRAMEBUFFER_FETCH_AVAILABLE UNITY_LIGHTMAP_DLDR_ENCODING UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_NO_DXT5nm UNITY_PASS_FORWARDBASE UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BLENDING UNITY_SPECCUBE_BOX_PROJECTION UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS
Disabled keywords: DIRLIGHTMAP_COMBINED DYNAMICLIGHTMAP_ON FOG_EXP FOG_EXP2 FOG_LINEAR INSTANCING_ON LIGHTMAP_ON LIGHTMAP_SHADOW_MIXING LIGHTPROBE_SH SHADER_API_GLES30 SHADOWS_SHADOWMASK UNITY_ASTC_NORMALMAP_ENCODING UNITY_COLORSPACE_GAMMA UNITY_HALF_PRECISION_FRAGMENT_SHADER_REGISTERS UNITY_HARDWARE_TIER1 UNITY_HARDWARE_TIER2 UNITY_HARDWARE_TIER3 UNITY_LIGHTMAP_FULL_HDR UNITY_LIGHTMAP_RGBM_ENCODING UNITY_METAL_SHADOWS_USE_POINT_FILTERING UNITY_NO_FULL_STANDARD_SHADER UNITY_NO_SCREENSPACE_SHADOWS UNITY_PBS_USE_BRDF2 UNITY_PBS_USE_BRDF3 UNITY_PRETRANSFORM_TO_DISPLAY_ORIENTATION UNITY_UNIFIED_SHADER_PRECISION_MODEL UNITY_VIRTUAL_TEXTURING _ADDITIONAL_LIGHTS_VERTEX _ADDITIONAL_LIGHT_SHADOWS _ADDITIONAL_OFF _BUILTIN_ALPHAPREMULTIPLY_ON _BUILTIN_ALPHATEST_ON _BUILTIN_AlphaClip _BUILTIN_SURFACE_TYPE_TRANSPARENT _EMISSIVE _MAIN_LIGHT_SHADOWS _MAIN_LIGHT_SHADOWS_SCREEN _OCCLUSION _TEXTURE_TRANSFORM _TRANSMISSION

In the VisionPro simulator the results are the same as before: pink geometry with a log about unable to load the ShaderGraph material.

How do I go about debugging this problem? Is there a tool which will check whether the ShaderGraph shader is compatible with VisionOS?

I have saved copies of the .usda files mentioned by the logs if those can help for debugging.

–
Casey

2

I dug into gltf-pbrMetallicRoughness in the ShaderGraph editor and found there were a few custom script nodes. Fortunately they were easy to remove:

BaseColor has one custom function:

  1. ProjectColorSpace it appears to be a simple switch statement. The node can be removed and one of its inputs can be wired to the output (I took the linear color space input and wired it through to output)

Emission has two custom functions:

  1. glTFast_HDRP_GetEmmisionHDRColor (big long script in include/Emmision)
  2. RPSwitch that is a simple ifdef that chooses either the HDRP path or another path.

I removed the entire HDRP path and switch supporting it and hardwired the other path.

This resulted in correct rendering in playmode and still having the same error about loading the shader when running in the simulator.

The following is a list of node types now found in the entire shader graph:

  • Property
  • UV
  • Screen Position
  • Sample Texture 2D
  • Scene Color
  • Multiply
  • Lerp
  • Out_Vector3
  • Step
  • Subtract
  • Add
  • Split
  • Vector2
  • TEXTURE_TRANSFORM
  • Out_Vector2
  • Vertex Color
  • Out_Vector1
  • Normal Strength
  • Colorspace Conversion
  • EMMISIVE
  • Out_Vector4
  • OCCLUSION
  • TRANSMISSION
3

I’ve stripped down the shader further so the Scene Color node isn’t present anymore. I’m still getting the same cannot load shader warning and the pink color in the simulator.

The all-caps nodes are boolean keyword nodes, which I assume are supported. All other nodes appear to be supported according to Unity PolySpatial ShaderGraph Support | PolySpatial visionOS | 0.1.0

Any thoughts on what I can try next?

4

Working with @ccrabb I was able to make some progress there but it is far from fully working:

PlayModevsSimulator_Rebuilt_GLTFastPBRShader
PlayModevsSimulator_Rebuilt_GLTFastPBRShader3590×1032 487 KB

so I have few questions:

  • can you confirm which URP package version we need to use to be compatible with VisionOS ?
  • can we use precision half for the node input ?
  • the documentation says: " Within the ShaderGraph editor, unsupported nodes will be indicated by the presence of a # symbol." => I look around I haven’t seen any of that when creating new node or from existing nodes in subgraphs or at the subgraph level. Where are those symbols supposed to appear?
  • In project settings/Polyspatial there is an option “Enable Mac Reality Kit Preview” what is that supposed to do ? Will it simulate same rendering you have in simulator in play mode (with MaterialX conversion) ?
  • +1 on question from Casey on debugging,
5

We’ve been testing with 14.0.1.

I wouldn’t expect this to make a difference. Currently, for purposes of conversion to MaterialX, we ignore the precision (and use the default MaterialX precision, which should be single).

They should appear when you connect unsupported node directly or indirectly to a shader graph output (e.g., “Base Color”).

No; I’m not aware if “Enable Mac RealityKit Preview” currently works, but the idea with it is to show a window that renders the scene using RealityKit for macOS, which does not support MaterialX shaders (ShaderGraphMaterial is only supported on visionOS).

6

This indicates that there was an error in the generated MaterialX (usda) file, and it should be accompanied by additional information in the Xcode console about the precise problem (e.g., an unsupported node).

If you can send me the USDA and/or the original shadergraph, I can help debug.

Our support for keyword nodes is limited at present. Basically, they will reflect whatever setting the keyword has when the shader graph was imported/saved–not the actual runtime keyword setting. The exception to this is the built-in “MaterialX” keyword, which allows one to use different values for MaterialX export versus Unity rendering.

7

I found that shaders in prefrontalcortex/UnityGLTF works in VisionOS. So I made a wrapper to apply UnityGLTF materials to glTFast object.

I also made a Visual Scripting Node to load glTF at runtime and apply the compatible shaders.

Screenshot 2024-01-05 at 20.56.46
Screenshot 2024-01-05 at 20.56.462758×2228 660 KB

3 Likes
8

Nice! It looks like the gltf materials are mapped to Standard, URP/Lit, or URP/Unlit, which we then convert to RealityKit’s PhysicallyBasedMaterial.

1 Like
9

Note on this topic, also discussed in : Apple Vision OS support · Issue #666 · atteneder/glTFast · GitHub

1 Like
10

Hi @from2001vr !

I’m running into the black material issue when downloading a GLTF with the GLTFast package but I’m not using visual scripting.

Can you explain more about the process for applying the prefrontalcortext/UnityGLTF shaders/materials to the downloaded object? I’m using C# and wondering if there’s a way to accomplish it that way.

11

Just fyi, the next version of UnityGltf will include direct support for PolySpatial for its shaders.

3 Likes
12

This is great news! Thank you for the heads up! :pray:

13

Great. I look for the visionOS support of the package.

@dave-mona
This is the C# code of my visual scripting node, by the way.

1 Like
14

Unity glTFast 6.4.0 now supports visionOS!!

2 Likes
15

We are using the Unity glTFast pbrMetallicRoughness shader and noticed a couple of issues. Texture tiling via the shader doesn’t seem to do anything. We don’t see Base Color Tex ST changing tiling for either the albedo or normal map. We also try different UV channels.