Hi Guys
The list of raytracing limitations does not seem to get updated from 1 version to the next. For example the HDRP 13 Raytracing docs still talk about Unity 2020.2 in the limitations:
So then does that mean the raytacing limitations between HDRP 10 and HDRP 13 hasnt changed at all?!?
That cant be right!!!
Many of these limitations, such as vertex animation and tessellation, are due to those effects being generated directly by the GPU and as such canāt be represented by the BVH acceleration structure used in raytracing. HDRP 13 does however have newer āMixedā modes for quite a few raytraced affects that get around these limitations by using hybrid of raytraced and rasterized techniques. For example, the mixed raytraced reflections uses traditional SSR and blends it in with the raytraced results to account for that missing information in the BVH.
The reason this is necessary is because getting the BVH to include the geometry generated by the GPU would require sending it to the CPU, generating the BVH, and then sending all that information back to the GPU which in more complex scenes is prohibitively expensive. However, this may eventually change as in the future it could potentially be possible to move BVH generation to the GPU via compute shaders, which would allow it to access all the information it needs to include these GPU driven effects.
Does HDRP 13 have a mixed mode for shadows also?
HDRP 12 does not, so its either raytraced shadows, or its not. Would be good if we could choose which objects get raytraced shadows and which dont. For example a large amount of foilage (static FBX meshes) would not require raytraced shadows in most cases, but currently there doesnt seem to be a way to exclude these from raytraced shadows and give them normal shadows, while at the same time have other objects like buildings cast raytraced shadows.
There is unfortunately no mixed mode for shadows. Rasterized shadows are a very complex beast which can make it difficult to combine with raytracing, at least in the way you are describing. NVIDIA actually published a paper back in 2015 on hybrid raytraced shadows however the technique they describe is used for generating realistic soft shadows using traditional rasterized shadows as a basis rather than having some objects use raytraced shadows while others using standard shadows.
While Iām not familiar with your project and its performance target, one potential work around could be to use spotlights on the specific object that you need to have traditional shadows. Another option might be to try and adapt your foliage to use the HDRP SpeedTree shader as I believe that is compatible with raytraced shadows. If I am remembering correctly and that is the case it might actually work out in your favor performance wise as raytraced shadows can actually be both faster and higher quality than traditional shadows when used in scenes with a very high number of very small objects like leaves.
How would I go about doing that?
For example, currently I have Nature Manufacturer Trees, and since they dont support beta versions, their shader doesnt work, so Iām just using a regular HDRP/Lit Shader instead, which looks the same as theirās but of course without any wind animation (which Iām fine with). Is it simply a case of swopping the shader to Speedtree and thats it? Or its a bit more complex than that? I donāt mind not having wind.
Remember, Iām doing a very large exterior render which will likely have thousands of trees, shrubs etc in view. This is for film so Iām not too worried about actual realtime performance, but more about ārealtimeā rendering. The whole point of all this stress and struggles is to find a solution to move our pipeline from traditional rendering to ārealtimeā or at least near realtime and not wait overnight for a single image and days on a renderfarm for an animation.
I believe it should be mostly straight forward to transition to the SpeedTree shader as it uses a fairly standard PBR approach to materials however the wind and subsurface scattering features probably wonāt work properly as they require data generated by the SpeedTree modeler program.
Getting access to the SpeedTree shader will require getting a SpeedTree asset which is actually pretty easy as they offer a free version of the modeler program, which you can grab here, that allows you to export a handful of example trees that come with latest version of the SpeedTree shader as well as textures so you can see how those are set up and adjust your own if need be.
If the SpeedTree shader ends up not working out Iād highly recommend taking a look at the second room in the HDRP sample scene as that has some great examples of folliage, both bamboo trees and ferns/bushes, that use the standard HDRP/Lit shader.
OK, I think I have a pretty solid foundational workflow and setup now in HDRP 12 and raytracing.
Now if I can only get a parallax effect of any kind or displacement of any kind going, that would be sweet and we can start moving our pipeline over to Unity (normal map isnt enough, we need displacement or parallax)
Any idea if something is in the works in that regard?
There is the HDRP/LitTesselation shader which does triangle based displacement. Though, like mentioned in the picture from your first post, the raytracing will fall back to the non-tessellated base mesh. The mixed modes for reflections and global illumination will still allow the tessellated geometry to represented for those effects because of their screen space fallbacks, however, the feature that will have the most glaring issues will be raytraced shadows as the displaced geometry will not be able to cast shadows and will cause inconsistent results. If you really need those shadows for a tessellated object your best bet would be to bake the displacement out as the actual mesh in your DCC of choice.
Hi @INedelcu ,
Iāve noticed that you are working on raytracing in Unity and replying to questions about similar stuff like gpu instancing/vertex animation with raytracing. I have a question related to this thread.
Do you think that itās possible that raytraced shadows will work with vertex displacement like tessellation in the future?
Any answer will be much appreciated.
Thank you.
Hi!
Unfortunately no. To build an acceleration structure, the GPU needs the source mesh (vertex/index buffers) in main GPU memory. If you have some vertex animation, geometry shaders or GPU tessellation, this data is kept in local GPU registers only and is not possible to automatically capture it. There will be options to add a Mesh to the acceleration structure instead of MeshRenderers. You could refactor your animation for example and use compute shaders + Mesh bindings, compute the vertex animation and output the result into main GPU memory and then use the result Mesh data as input to build the acceleration structure.
SkinnedMeshRenderer does this internally when GPU skinning is used.
These are not Unity engine limitation but itās how DXR works right now.
You can further imagine a forest or jungle + wind animation. This is the worst case possible for ray tracing. Since wind noise generates unique animation / vegetation instance, each vegetation instance (tree, bush, flower or grass) needs to output its own Mesh. This can mean a lot of extra GPU memory and bandwidth. Each of these meshes will have an associated acceleration structure (BLAS in DXR) which means a lot of GPU time and memory to generate them every frame. Additionally since vegetation uses alpha texture, anyhit shaders will be active (to perform alpha testing when rays hit triangles) and depending on how dense the vegetation is, the GPU cost for traversing the acceleration structure will be prohibitive.
I have noticed that vegetation renders quite slow with raytracing. Is that why? What can I do to make it faster?
Check https://developer.nvidia.com/blog/rtx-best-practices/
Q. Is there guidance for how much alpha/transparency should be used? Whatās the cost of anyhit vs closest hit?
A. Any-hit is expensive and should be used minimally. Preferably mark geometry (or instances) as OPAQUE, which will allow ray traversal to happen in fixed-function hardware. When AH is needed (e.g. to evaluate transparency etc), keep it as simple as possible. Donāt evaluate huge shading networks just to execute what amounts to an alpha tex lookup and an if-statement.
I see, thank you for the reference. Is there a HDRP setting that will make the raytracer ignore cutout-opacity on foliage?
Hi @INedelcu
When do you estimate the work youāre doing on this, to make it into an alpha/beta?
Hi @newguy123
Which work are you talking about?
Hmmmmm apologies @INedelcu
I seemed to have replied in the wrong thread. The comment is for you though, and should have gone here:
First of all thank you for the detailed responses in this thread. They are very helpful in understanding the systems.
Iāve noticed that there is a new position in the rendering roadmap related to raytracing:
https://portal.productboard.com/8ufdwj59ehtmsvxenjumxo82/c/1785-raytracing-api-out-of-experimental
Does it mean that you have some plan to support instancing APIs like Graphics.RenderMeshIndirect?
As it mentions foliage Iām wondering if you have some plan to ātranslateā shader based vertex animation to the raytracing acceleration structure?
Itās very interesting. Especially after reading your posts about raytracing, vertex animation and tessellation:
