Critias SpeedTree Grass System

Hello!

! UPDATE !

For those interested in a complete grass/tree, easy to setup system based on this tree and the grass system check out ‘** Critias Foliage System **’.

! UPDATE !

A full project with all the code included can be found here.

One of Unity’s ‘big’ problems is the lack of a decent vegetation system. Developing an open world quest I had to fix that issues. For me none of the current asset store or unity sollutions (sorry guys) proved to be effective and to both look good and work well.

My goal was the performance and looks of certain AAA games out there, that sucesfully used SpeedTree for everything from grass to trees.

The current system uses SpeedTree grass with it’s wind animation that are directly exported from the Unity’s SpeedTree editor.

Implementation:

Grass mesh data counts per patch: 570 verts, 234 tris with uv/uv2/uv3/uv4/colors, just like good ol’ SpeedTree exports them.
Grass blade count: 10 milion - Well the count doesn’t really matter, only the density per patch matters.
Terrain size: 2250 meters x 2250 meters
Big cell size: 90 meters, 25 cells in length, 625 total cells
Small cell count: 5 subdivisions for a large cell, 25 small cells per large cell, 18M each, at 15625 total count
Grass density: 16000 blades per large patch, 640 blades per small patch

Take one terrain and divide it into large cells, creating a grid.


(Overview)


(Grid large cell)


(Grid small cells)

Don’t worry, we don’t need to keep all those references and ‘BoxCollider’ objects, we can safely delete them. All we need is their world bounds that is stored with the grass.

Having that determining in which cell the player is, is trivial based on terrain’s local space player position using interpolation.

Vector3 pos = ; // - get your player's position -
Vector3 size = terrain.terrainData.size;

int row = Mathf.Clamp(Mathf.FloorToInt(pos.x / size.x * m_Cells), 0, m_Cells - 1);
int col = Mathf.Clamp(Mathf.FloorToInt(pos.z / size.z * m_Cells), 0, m_Cells - 1);

After we determine in which large cell we are, take all the sourounding cells of that cell and test them for visibility using ‘GeometryUtility’ class.

if (GeometryUtility.TestPlanesAABB(planes, bigGrassBox.m_Bounds))
{
   // If it is visible display grass
  totalUpdated += Render(bigGrassBox, ref cameraPosition, planes);
}

We only need to test for 9 visibility bounds. Ussualy there are something like a 3 big grids mostly visibile, and out of the rest only a few small cells are visibile.

If a big cell is visible we do the following:

  • Iterate through it’s small grids
  • If the small grid within the grass distance AND it is visible proceed
for(int cellIndex = 0; cellIndex < data.m_Cells.Length; cellIndex++)
{
   ...

   if (distanceToSmallerGrid < m_VisibilityRange + box.m_Bounds.size.x && GeometryUtility.TestPlanesAABB(clipPlanes, box.m_Bounds))
   {
     ...
   }
}

We test the distance first since since it the ‘TestPlanesAABB’ is more ‘computationally expensive’ :p.

After we determine that a small grid is within range AND visible we iterate through it’s grass patches and start collecting them if we have the ‘DrawMeshInstanced’ API or simply draw them if we don’t. We only collect a grass patch if it is withing viewing range. We don’t test it for visibility, that is why we have that ton of grids, to mitigate the need of testing the visibility.

And how much grass we draw? Well of course not all of it for distance cells. We chose it using a dynamic lod function:

private int DynamicLOD(float cameraDistance, float cameraNearZ, float cameraFarZ, int maxBladeCount)
{
   return (int)((1 - 7 * (cameraDistance - cameraNearZ) / (8 * cameraFarZ)) * maxBladeCount);
}

// 625 is the maximum grass blade count per small patch, can be variable based on your density
int grassCount = DynamicLOD(distanceToSmallerGrid, 0.3,  m_VisibilityRange, 625);

for (int i = index; i < grassCount; i++)
{
   ...
   bool castShadow = m_CastReceiveShadows ? (distance < m_ShadowDistance ? true : false) : false;

   if (distance < m_VisibilityRange)
  {
     ...
     if(hasInstanced)
     {
       if(castShadow)
         grassBatchMatrixShadow.Add(grass.m_WorldMatrix);
       else
         grassBatchMatrixNoShadow.Add(grass.m_WorldMatrix);
     }
     else
     {
       if (distance < m_AnimationRange)
       {
         Grahpics.DrawMesh(grass, .... whatever animated, castShadow);
       }
       else
       {
         Grahpics.DrawMesh(grass, .... whatever animated not animated, castShadow);
       }
     }
   }
}

if(hasInstanced)
{
   Grahpics.DrawMeshInstanced(grassBatchMatrixShadow, ... whatever, true);
   Grahpics.DrawMeshInstanced(grassBatchMatrixNoShadow, ... whatever, false);
}

This can be further optimized so that we don’t test with ‘if’ each blade count but have before we enter a small cell a function chosen based on the hardware capabilities. But these are details.

Performance:

  • 10 milion blades of grass, 2250M terrain, 90M cell size, 5 cell subdivisions *

Unity 5.4 - GTX 970:
60 FPS at 50M of grass distance, 30M animation, 10M shadows - DrawMesh API (we don’t really have an alternative…)

Unity 5.5 - GTX 970:
300 FPS at 50M of grass distance, 50m animation, no shadows - DrawMeshInstanced API
150 FPS at 50M of grass distance, 50m animation, full shadows - DrawMeshInstanced API
37 FPS at 50M of grass distance, 30M animation, 10M shadows - DrawMesh API

Limitations:

  • No player interaction. But with some smart shaders, that also becomes possible, by activating only the grass that is very close around the player and bend
    it accordingly when it collides. We can also optimize the creation of colliders in a ton of ways that I will not cover now.
  • Large memory footprint since it uses grass with positions. We can mitigate that with:
  • Procedural grass based on ground texture, generated at runtime
  • Holding in the list only the grass type and position, modifying the scale at runtime procedurally
  • Loading grass cell data in memory only when required, unloading data that we don’t require, based on active grid
  • No fading. But that can be fixed with some SpeedTree shader modifications. It’s still work in progress
  • Grass popping when changing distance. Can be fixed so that DynamicLod is only applied to grass that is less visible. For example let us not apply it to flowers…

It seems that ‘DrawMesh’ is slower on 5.5 than in 5.4 at least in the beta. I’m sure that’s going to ruin somebody’s game :)).

Of course as a optimization we can draw only partly the shadows, something like 20-30 meters and the rest with no shadows in order to keep the FPS at 200-250.

Tested on a “GTX 750Ti Golden Sample” too, with good results.

A GTX970 is pretty high end so, of course, I hope that as soon as possible I can provide a demo in order to let you all test the system on your system’s configuration and provide with feedback and suggestions. I also hope that for our game (The Unwritten Critias, A full open-world game) we can have some in-game footage soon, in order to see the system applied in a real game! The game is not only grass, but a ton of other objects and effects, afterall.

And it seems that we’re forced to wait for 5.5 due to it’s DrawMeshInstanced API. We can’t let the grass to eat up everything else.

Some screenshots:


(90 meters of pure powergrass… With flowers.)


(50M all animated)

References:

! UPDATE !

You can now find the demo here.

! Update !

Is someone is interesting on the tree system also, feel free to check it out here.

Update 1:

  • Camera distance based scaling so that distant grass does not pop any more
  • Possibility to set batch size sent to GPU

Update2:

  • Removed DX9 support, only DX11/DX12
25 Likes
  • Reserved for a future post is the building of the structured grass. Where do we get the grass from in order to render it? Stay tuned for an answer to that! -

[Update]

As a quick tip, all the grass I’ve placed is generated with Gaia. After that it is extracted from the terrain (the grass is added to the terrain as trees) and added to my system. It can also be manually placed with the tree brush.

In the future I have a plan to add it procedurally based on a density parameter and the underlying texture that the terrain has at that location.

1 Like
  • Reserved for a future technical art tutorial on how to create very well optimized grass in SpeedTree, ready for runtime usage -

Amazing write up, really looking forward to the next part. I came up with a similar solution in my game but using only large patches (100 x 100) with are activated or deactivated as the camera moves around, I fade in and out in the shader directly.

It’s been quite effective the only issue is it’s very memory intensive since I keep a reference to every blade position since the grass needs to be removed (and the patch combined mesh needs to be rebuilt when it happens) but so far performance has been really good. It only takes 1.7 seconds to generate 100 grass patches on a 2048 x 2048 and it can later be removed by placing roads or buildings.

That was my first solution but, unfortunately, I would have lost the awesome graphics and wind that SpeedTree offered me. I can’t wait to put the demo or some video of it used in something ‘real’ to see it’s ‘real’ power :slight_smile:

This is awesome, I’m going to give it a shot this weekend and stalk the thread in the mean time :slight_smile:

It’s true…it’s awesome…and with the new 5.5 planned for the next month I think that your approach will give us a good solution for these situations. Congratulations!!!

@Assembler-Maze I have two question for you, one related about the use of the instances and one specific to your solution:
1- I’m a 3d artist used to work with non-realtime rendering systems and in that world the instances of a mesh is a huge saver when your scene become full of detail, both in the scene viewport management and in the image rendering phase.
For the scene management they’re useful because with few 3d models you can quickly populate environments duplicating them and using rotations, scaling and so on.
For the rendering phase, using instances saves memory.
So when Unity introduces instances in the latest releases I was expecting to have finally a good tool for our realtime envitonment but, at the moment, I haven’t seen too many working examples using this option from the Unity users base…Yours seems the first.
What do you think about the use of instances in Unity?
What are the difficulties to implement this?

2- If I’m not wrong, you did a grass patch and after instanced many times…Do you think it’s possible to use with trees too? If not, why?

Thanks in advance for your reply.

I’m impatient to see your progress and your demo.

1 Like

Hello!

I will try to tackle each problem. Please, bear in mind that it might be quite subjective, based on my experience and not on others :).

  1. Instancing is something that we should have had quite a long time ago (again, my question, why don’t we have Unity open source like other engines today in 2016? This things would have come a lot faster…) . However we need quite ‘new’ hardware for desktops and it is not supported on mobile.

Quote from the Unity manual:
"GPU instancing is available on the following platforms:

Windows: DX11 / DX12 with SM 4.0 and above; OpenGL 4.1 and above
OS X & Linux: OpenGL 4.1 and above
PlayStation 4
Xbox One"

However, from what I knew, instancing was available from DX10, but I guess Unity ppl wanted the best way to have instancing for us.

DX release dates:

DX 11 October 22, 2009
DX10 November 30, 2006

So, one thing, if you plan to use instancing with Unity you probably need to target hardware from 2009 and up. Of course older cards can run on DX11 mode but it will probably not be the best thing, performance wise.

So keep that in mind when developing :).

Also, we had instancing from Unity 5.4 but it was done automatically by the engine, so we can say that was equal to zero divided in four. If you’re not doing it manually it is kind of useless. The difference between 5.5 and 5.4 is that you now have an API called ‘DrawMeshInstanced’ that when you call you are absolutely sure that instancing will be used, if available on the hardware. But again with only access to the API through C# scripts, for systems it is not the best way to do. C# is slower than C++ no matter how much you optimize it. But it is better than nothing. As a practical example my drawing routines were improved by an order of almost 5-10 when using instancing.

The thing is that the CPU is bound to something like 3000-4000 (please correct me if i’m wrong) draw calls per frame, probably it will grow in the future to something like 4000-6000 with 8 cores etc… So, imagine 24000 pieces of grass, it means that almost all the CPU time is eat up by the grass only, with 24000 individual draw calls. But with instancing you basically have one draw call for each patch of grass multiplied by the count of grass types in that patch. And if you have about 40 visible small patches with let’s say 600 grass blades each(see the post above) you only need 40 draw calls for 24000 grass pieces.

And even for you, when you buy a pizza it is a lot faster to grab the whole box from the delivery guy rather than taking it slice by slice :). The same applies for PC’s.

References:
http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter03.html
http://http.developer.nvidia.com/GPUGems3/gpugems3_ch02.html

So, having that said, it makes the perfect system for grass, ground details, trees. Quote from GPU Gems:

“An outdoor scene with lots of vegetation and trees, many of which tend to modify their attributes (for example, trees and grass swaying in the wind), and lots of particle systems. This is probably the best scenario for the Geometry Instancing API.”

  1. Yes, it is possible to be used with trees, the shader for instanced trees already exists in unity. For performance, I do not know yet, we need to wait for a more stable version of Unity 5.5. But I hope for a awesome improvement, performance just like Valley Benchmark. But I didn’t had time to test if for the moment. And again it’s instanced dependent. Before knowing that I made another system (that I’ll make another post when I have time) specialized for SpeedTree trees. It basically adds all tree the billboards to a static batch when the trees become billboards. You can basically have millions of billboard trees with little impact on performance. I think I tried it with something like 100k trees.

There is some data at:
https://forum.unity3d.com/threads/speed-tree-optimisations.317585/

The system is older there, used with Unity’s particle system. I managed to get a decent 150FPS with 50k trees. But with manually batched trees the performance is something like 700-800FPS. They’re just some billboards with all the data packed in the the uv3/uv4 channel packed, that is the pivot, size, and world position per-vertex. Nothing fancy.

For even more data and a better system check out Uncharted 4’s approach with baked pivots:
http://advances.realtimerendering.com/other/2016/naughty_dog/NaughtyDog_TechArt_Final.pdf

Hope I’ll have time to write about all that, but we’re only 2 guys and I have to also do art, level design and programming. I can’t promise but I hope to put the demo soon, in a week or so.

If I was unclear or went of-course feel free to ask me for clarifications.

6 Likes

I have just updated the demo on Dropbox.

There are several problems though:

  • No fading at the moment
  • In some camera angles the animation gets stuck
  • (Not my fault, probs Unity 5.5 related) some weird artefacts when not using instancing


(Max Settings Screenshot, 90M view distance)

6 Likes

Thanks for the demo. I’m getting pretty good results on my GTX950m, still above 30 FPS at full density and view distance, albeit no post processing, shadows and on 1600x900. The only really bad thing about this is the “popping” of grass very close to the camera, not sure if that is one of the drawbacks of the system. Speaking of popping: Would fading in the grass defeat the purpose of the system? If so, scaling grass smaller with increasing distance could also help to mitigate grass popping into the view distance.

Also, I got a couple of errors in the log:

The referenced script on this Behaviour is missing!
(Filename: C:/buildslave/unity/build/Runtime/Mono/MonoBehaviour.cpp Line: 1526)

The referenced script on this Behaviour is missing!
(Filename: C:/buildslave/unity/build/Runtime/Mono/MonoBehaviour.cpp Line: 1526)

The referenced script on this Behaviour (Game Object 'GrassSystem_V2') is missing!
(Filename: C:/buildslave/unity/build/Runtime/Mono/MonoBehaviour.cpp Line: 1754)

A script behaviour (probably Grassifier?) has a different serialization layout when loading. (Read 32 bytes but expected 76 bytes)
Did you #ifdef UNITY_EDITOR a section of your serialized properties in any of your scripts?
(Filename: C:/buildslave/unity/build/Runtime/Serialize/SerializedFile.cpp Line: 1803)

The referenced script on this Behaviour (Game Object 'GrassSystem_V1') is missing!
(Filename: C:/buildslave/unity/build/Runtime/Mono/MonoBehaviour.cpp Line: 1754)

A script behaviour (probably Grassifier?) has a different serialization layout when loading. (Read 32 bytes but expected 76 bytes)
Did you #ifdef UNITY_EDITOR a section of your serialized properties in any of your scripts?
(Filename: C:/buildslave/unity/build/Runtime/Serialize/SerializedFile.cpp Line: 1803)

Yes, it is absolutely still [WIP] :).

And as I have mentioned:
“No fading at the moment”, so the grass does not fade when coming in from the distance at the moment. The solution would be to pass in another variable to the SpeedTree default shader and modify the shader a little so that the grass can fadein at the distance.

A. For the grass popping in at the furthest range:

There are three solutions for that:

  1. Use the ValleyBenchmark type of fading in, the grass has a very small scale in the distance and it ‘grows’ when you get closer
  2. Use the Wow type of fading in, the grass is very transparent in the distance and it gets more opaque when getting closer
  3. Use the SpeedTree dither mode, but modifying it so that you manually calculate in the shader it’s value based on distance to camera

B. For the grass popping in at the closer range:

For the grass popping in at closer range, that is because of the DynamicLOD function. I am afraid that even if modifying the shader the grass will pop in very close and cause that nasty effect. A workaround for that would be to only apply the DynamicLOD further away from the player so that the popping is only visible at a farther range, distracting the player from the effect. Of course in a real scene you also have trees, objects in order to further mitigate that effect. I see that even the guys at ‘Valley Benchmark’ could not completely mitigate that effect. Made less worst yes, completely remove, that’s more difficult.

For the complete 100% correct solution, I am thinking of having a list of data for each blade of grass and when a blade becomes visible with the DynamicLOD I would have some special data for it, like an opacity value that increases over time, or something like that. But before going in the complicated solution I would first try to see how much I cna mitigate the effect without huge headaches. Another thing is that this is C# so we lack the performance of C++ based systems. So, if my complete solution would be, maybe, awesome for C++ I don’t know if holing that ton of data and access it on C# would be cool and fast.

For the performance, from what I know, a GTX950m is about x2-x2.5 slower than a GTX 970 (like my card) so I guess that if I get 60FPS at ultra, you should get 30FPS at those settings. Of course, there are the performance settings. I think you can reach a decent 60FPS with half view distance and less density. With medium settings and no post-proc I can reach about 200-250FPS. But again this is a reason why these systems are made optimized in C++ and not in C# :). And without something like ‘DrawMeshInstancedPersistent’ like we should have it, having to re-submit all the draw commands each frame. For max performance I’m afraid we have to wait a few more months till the Unity guys implement their system as per the new terrain system forum.

Don’t worry about the log errors, they’re harmless.

I tried your demo, but I’m confused. When GPU instancing is turned on I get about half the frame rate. Shouldn’t I get better performance? I tested on Windows 7 AMD FirePro W5100.

Wow, it works worst with GPU instancing than it does without it? That is very weird indeed. Can you post a little more FPS info with your settings?

I’ve looked up your GPU, and it is SM 5 with dx 11/12 so it shouldn’t be a hardware problem I think. But if there really is a problem I’ll make another demo with less grass batched together. At the moment there are 500 blades batched, so maybe we could try with 250, maybe some buffers overflow on your video card.

For the next demo I plan that you can change the count of blades that get batched together (range from 100 to 500) and the fading in the distance.

If this will not fix your problem I’ll contact the Unity guys asking for some clarifications with that video card. Since my testing on different video cards give at least x5 more performance.

Updated with a some new features:

  1. Valley benchmark like grass scaling in the distance (maybe I’ll try with dithering/transparency in the future too). Of course, don’t expect to go completely unnoticeable because in a real scene there is a lot more than grass. There are a lot of ground textures, rocks, trees, structures etc… In the future maybe I’ll also update with the fast SpeedTree system I’ve been working on.
  2. Possibility to set batch size that is sent to the GPU

Still problematic:
Grass popping at smaller distances

@Assembler-Maze thank you for your reply.
About the demo…the link seems not valid…just me?

There are 2 links that were posted and the second one definitely is broken. The one I used that work is as follows:

https://www.dropbox.com/s/fk8ce0wv2yjxn1i/GrassSystem.zip?dl=0

Sorry, I forgot to update the second link. I have removed it so that it doesn’t trick people any more :). Just refer to the link in the original post.

I’m testing on a PC with integrated graphics (DX12), and I also get lower framerate when instancing is on. Haven’t tried it on my dev PC yet, so I will let you know if I see the same results there.

I understand, so it is a very problematic issue since it doesn’t happen to only one person. On my nvidia gtx970 it works just fine. And by the way it’s no matter that you have dx12 since the app’s API is only dx9 and dx11. Maybe it automatically chooses dx9 instead of dx11? I’ll try make an executable that is only dx11 enabled with no possibility of having dx9 since there’s no instancing there.

Could I also have some info about your hardware setup? If the problem persists even after I’ll contact the Unity guys with the sample project telling them the problem appears on non-nvidia hardware.

P.S: Now I am making an app that is only dx11/dx12 enabled. If that doesn’t solve the issue, I’ll be very grateful if you reply here again.

I downloaded the demo this morning. On the default settings I see practically no difference between gpu vs non-gpu, both at around 110fps. However when I raise the grass distance to the max 90 level, non-gpu runs around 40fps, and gpu runs around 75fps. (Windows 8 64bit, NVidia GTX 580)