[WIP][Sept 16th] Terrafirma - An Auxiliary Terrain Engine Replacement and Extraction Toolset

Community Showcases
1

Entire Post below has been rebuilt and updated.
Update
TerraFirma is moving to a new home… Though i will still be heavily involved in it’s development from here on in, me and the brilliant designer, Lennart Johansen have forged a bit of a partnership, and the system is now officially, an “Awesome Technologies” production! :wink:
What this essentially means is as a whole, TerraFirma will be far more complete product, with many great features planned between the both of us that should give developers a bigger better suite of tools to enable more possibilities than one coder such as much so could hope to do in a timely fashion.
With amazing vegetation, terrain and in a sense, scene management, at your disposal… Well the future is looking very bright from our perspectives. :slight_smile:
We are working on this now and will soon start another forum thread and website with updates as the development continues.
Thank you everyone for your support, and many great updates to come with this as well. !
*** A good majority of the images found through out this post at the time of writing are no longer visible. Moved image hosting my own domain.

This project is re-imagining of the terrain system built in with Unity3D designed from the ground up with intention of being performant, lightweight and feature rich, while also delivering a feeling of familiarity in-terms of how both the Terrain objects interact and are interfaced.

Below I’ve outlined a mostly complete feature list, as well as image sets comparing mesh quality and performance.

Mesh Comparison
Top Image Set
Left - Unity
Right - TerraFirma (All patches set to the highest level of detail)

Bottom Image Set
Top - TerraFirma
Bottom - Unity

-Can generate from Unity terrain data. (Convert unity terrain to a terrafirma terrain.)

-Is completely compatible with the unity terrain vegetation system. (Utilized for now and later as a fall back, vegetation system preview below.)

-Mesh topology in terms of geometry is identical to the unity terrain object it is generated from.

-When using the built-in shaders that come with texture splatting appearance is maintained as well.

Performance Comparison
Below is comparison of both TerraFirma and Unity handling the terrain mesh.

The first image is in-land where only a small portion of the terrain is exposed.
The bottom image shows the same scene running with entire terrain object expose in the viewport.

In both image sets :
Top - TerraFirma
Bottom - Unity3D

Middle of the Terrain.

Whole Terrain in Camera Perspective

4K Heightmap Based Terrain

This scenario is much the same as the above examples, the only difference is rather than handling a 1048 height map, the terrain here is utilizing a 4098 height map as it’s data set.

Top - TerraFirma
Bottom - Unity

-Like unity, TerraFirma supports terrains up to a maximum size of 4096 pixels.

Texture Array Support (Handle all texture layers in one pass)

-With texture array support every terrain texture layer can be drawn all in a single pass.

Tessellation Support

-Tessellation supported across all texture layers.

Texture Array Performance Comparison
Because all painted textures are handled in one pass, draw calls are reduced. This leads to an increase in

Top-Unity
Middle-TerraFirma (Non-Array Based)
Last-TerraFirma (Array Base)

Triplaner Projections

Left-Triplaner Projections ON
Right-Triplaner Projections OFF

-Supports Triplaner Projection removing texture tile stretching at sharp/long feature changes.

Batching And Frustrum Culling

-Comes with a custom batching method for reducing mesh instances/draw calls.

Height Based Culling as well as User Specified patch culling/exculsion.

-Supports culling of patches below a certain height.
-Supports culling of patches specified by the user through script. (Create your own culling rules!)

Real-Time Shadow Support

-Supports Real-Time Shadows across all built-in shaders.

Multi-Terrain Support

-Supports stitching and LOD transitions across multiple terrains.

-Supports modifying neighboring terrain connections in real-time, enabling the user to build infinite terrain
generation schemes, as well as systems to load/offload data from disc, the web, ect.

LOD Transition Methods

-Currently Supports two transition schemes.

Left Error Metric Based
-Detail is placed optimally, where the terrain changes occur the most.
-Partition “pop-in” on transition is reduced.
-Works in a similar fashion to unity’s method.

Right Radial Based
-Detail is set higher to camera and progressive degrades away naively based on distance.
-Partition pop-in is noticeable on large terrains.
-Overall requires less draw calls and resources to handle the mesh.

Integrated Editor
Comes with a built in mesh editor designed to mimic as well as add to what is currently offered with the unity terrain editor.

-Also support layer-able brushes. Interface visible below.

Full Object Interface

Future Features

Vegetation Handling
As of now Terrafirma uses the unity vegetation system as a fallback with it’s vegetation system in the works.

-Custom Vegetation system. (Uses unity’s as of now and later as a fall back.)
-Wind simulation supported.
-Custom billboard generator build-in.
-Built to mimic the Unity vegetation system.

Things Left to Do :
-Support for different grass types.
-Better tree handling along with other optimizations.
-Voxel Patch Support/Paint on any mesh.
-Add in option to handle grass via a GPU.

Voxel Support (Tunneling / Mesh Over-Hang support)

-Marching cube meshing integration then later surfaces nets, first naive then with the option to use a qef solver and Hermite data (“Dual Contouring”).
-3D data is utilized only when the terrain data in any given partition calls for it.
-Stitches seamlessly with terrain patches driven by a 2D data set.
-Faster processing when editing volumetric data. (Flood filling, ect)

Things Left to Do For Starters :
-Material handling for volume patch edges.
-Optimize/finish the volume patch manager.
-Optimize the volume partition structure.
-Integrate the voxel editor. (Video below)

Things to do Later
-Island removal. (floating chunks).
-Surface nets/dual methods.

Mesh Terrain Intersection Blending (Just needs to be interfaced)

Top - Two Objects Blended with the Terrain
Bottom - Two Objects Blending with each other and with the Terrain

Things Left to Do :
-Interfacing

Full GPU Driven LOD Transitions (via tessellation for now compute shader handling in the works well)

-All LOD transitions handled via the gpu.
-Dramatic reduction in draw calls.

Things Left to Do :
-Interfacing

Other Planned Features
-Sphere mapping a cube with 6 terrains as the faces. (Planets)
-Texture Coverage System for both meshes and the Terrain (procedural snow/leaf covering system handled via shader.)
-Threading support.

22 Likes
Creating big multi-tile maps with streaming, new veg tools, etc.
[RELEASED] Vegetation Studio
Unity for flightsim
2

Looks great, I can not wait to try.
Will you include an API like the unity terrain where you can sample height, normal’s etc at points?

3

Terrafirma may be great to integrate into MegaSplat .
Looks really cool.

1 Like
4

Looks interesting. Will be watching this :slight_smile:

5

Yes, There will be an API with a sampleHeight (3 calculation methods) and getNormal option (4 calculation methods), i’ll update the main post to go a little more in-depth on this later tonight. But there will be api for accessing materials, normals, heights, ect. Thanks you.

6

Hmmm… Considering the assets popularity i’ll be definitely purchasing MegaSplats and working towards that. Personally however I think integration would be rather trivial myself. :smile: Thanks for pointing this out to me however, I’ll add an advanced option with parameters for the number of splats per layer/pass aloud when generating materials and inputs for giving the generator the proper tags, floats, ect for the shader.

7

Thanks !

8

Update January 4th and 7th:

**Demo and Video previews planned for release sometime between 7th (Tonight) and the 9th. (90% ish complete)

Technical Bits and Showcase (Updated January 4th)
-Images showing the difference between rendering a scenes terrain using Unity’s Terrain mesh vs Terrafirmas. Visual shows overall mesh memory usage and has a write up detailing what’s going in terms of performance differences between the two systems. Terrain object featured consists of 9 unity Terrain objects, 2000 x 2000 units making up a landscape that is 6000 x 6000 units total. Terrafirma can currently represent the terrain using 55.4 mb’s at 1700 fps vs Unity’s mesh at roughly 250 fps.

(Updated January 7th)
-Mesh generator has received a few major extensions in it’s functionality summarized below and detailed a bit more extensively across the intial post in this thread.

Mesh generators settings have been accommodated to support :
-Merging of Control Maps
-Processing of level mesh generation across frames. (Works for all LOD options)
-Support for specific meshing parameters for individual levels. (Works for LOD options)

Link to updated mesh generation settings interface :

386×587 30.5 KB

**Image in initial post updated to reflect changes as well.

The demo is almost ready as well, and will come in two parts; Both parts will essentially be the same. Currently I have a large terrain with a camera fixed along a path moving across the landscape. There’s a toggle for switching between the Terrafirma Object and the Unity Terrain “Draw Base Mesh” Option as well as controls for LOD transition distances and a speed slider for controlling how fast the camera is moving with a rendering statistics window for contrasting the two systems performances.

Part one is practical demo using the most efficient method in terms of speed and memory for mesh generation on the terrafirma object, and part two is more of stress test for comparing Terrafirma using the worst LOD option in terms of performance vs the Unity Terrain mesh with no settings changed. (Note; Terrafirma object was still faster by quite a bit when running the stress test) I will also be releasing videos of both demos in action as well showing the Terrafirma mesh running alongside the Unity3d Terrain Mesh.

Also added a fourth LOD handling scheme which saves pre generated instances of Terrain Patches to disk instead of memory. Acts exactly like all memory saving schemes with the exception that it may be a lil slower when meshes switch levels but should run around the same overall in terms of speed (Slower is reallllly subjective especially when comparing memory/storage managed methods vs simply generating meshings on the fly ). No real trade of aside from that. Storing meshes in ram vs on disk will always result in the overall size of your game being bigger. It should however relieve your system of a lot of ram, having only the mesh instances needed being held in memory. (Basically uses the same memory as not storing any mesh information at all, LOD option 2). This all still remains to be interface at this moment however.

Last note, new layer for prioritizing LODs based on complexity is under way. Currently finished functions for computation for assigning complexity scores to any given partition of terrain. Relies somewhat on some rudimentary concepts from differential calculus. This new additional layer will attempt to prioritize the best mesh quality based on patch complexity, distance, as well as weather or not a patch is currently in the view frustum. (Note; scores are precomputed so functions aren’t ran at runtime)

1 Like
9

EDIT; Though Terrafirma is faster in both examples, the system has seem alot of optimizations since this has been posted. Feel free to view these, however be aware, the demo scene, at the top of my opening post of this thread is a more updated example of what my systems is now currently capable of and to be frank, puts my past effort seen below, to shame. :cry: whines

-Added a small series of videos comparing Terrafirma to Unity’s Terrain under a new section labeled, “Performance Comparison Videos (Comparing only the Terrain Object Meshing Performance Between Systems in this Example) (Updated January 8th)”. Located at the top of the initial post along with Terrain Object specifics and some information briefly detailing a few of the things going on in the scene used.

Unity Terrain (Slowest) :

Terrafirma Realtime Meshing Scheme (Second Slowest) :

Terrafirma Storage Scheme (Fastest) :

-Currently finishing up the demo scene. Should be live anytime over the next 48 hours.

Stay tuned and have a great day!

10

Update! The stitching algorithm has been optimized to use half the number of calculations the old method required. Memory Storage Schemes received a decent boost from this while the Transition Scheme received a negligible one.

Also system can now offload meshing to the GPU… :

More on GPU vs the traditional method after I complete some more stress testing. So far looks extremely encouraging though !! :slight_smile:

Demo will posted tomorrow around this time, and the initial thread post will be updated to reflect these changes.

Thank you.

11

Very interesting asset, have you thought about talking with mapmagic dev to create a mesh output generator to bypass the unity terrain generation altogether ?

This asset + mapmagic + megasplat could really become the terrain system unity has been waiting for…

12

Not exactly sure if this will answer your question. The way my system was designed was with the initial intentions of being a replacement for the terrain Mesh produced when the “Draw” option is selected under “Base Terrain” on Unity3D Terrain Objects so as of now it act’s as a light weight layer to free resources from Unity’s Terrain Handling System and to also give users more control over… well a lot of things… haha. At some point though when I get around to vegetation handling I would like the system to stand alone, no longer dependent on the Unity Terrain asset it was converted from. At that point I may get in touch with a few developers as integration should be trivial if the asset was designed to work with Unity Terrain. Would also like to mention that when the system is enabled to run Unity Terrain free, aside from the case of infinite map generation; once an asset like gaia, or map magic generates a Terrain, terrafirma will have no problem converting it and running with out the need to fall back on the Engines default Terrain Object (Aside From two calculations, it already does. :wink: ).

Would also like to apologize for any seeming lack of punctuality in regards to getting back to you. I tend to get really caught up in my work, haha. Thanks for the interest in the project though. :slight_smile: Maybe at some point in the future, it will be ready to replace the default solution, until I finish up a few more things however, I’ll keep opinions in that regard to myself. ;D

Major Updates

-Rather then making two post as well, I figure I’ll mention here the demo will be ready, hopefully tomorrow. Some set backs integrating gpu drawing methods. As of now though they are ready and support light weight culling options. (Frustum, Near Clip Plane).

  • Managed to find some major optimization areas however as well, depending on how long it takes to set up, I should be able to remove every “get” call in relation to Mesh data (vertices, triangles, ect) which is the only thing really causing garbage collection allocations. Removing them should speed things up tremendously in some case, in others, modestly.

-Memory Schemes mainly use resources referencing mesh data, especially reading it. The actual stitching process barely dent’s performance in contrast to simply reading the meshes vertices does.

-I also re-implemented a lot of features in such a way that the system doesn’t need as many objects to operate, and is able to manipulate the mesh data with out having to reference a meshfilter’s sharedmesh directly. (When drawing on GPU is disabled).

I would update the main thread post to relfect these changes but as of now I haven’t slept for around 48 hours and feel it can wait till tonight. Thank you !

13

Demo is now live; Currently updating the post to reflect all alterations and new additions added to the system. In regards to the demo you can however grab a copy from the link below or at the top of the initial thread post which also details the demo build a bit.
Demo :

https://drive.google.com/file/d/0B5L941lta-F2TDliYnludmhRT28/view?usp=sharing

14

Hi, I am quite familiar with MapMagic (and other terrain tools too, I have most of them) basically what I was asking is if your system could create a mesh terrain from a bitmap heightmap instead of using terraindata. I should also create a mesh collider but this is easy.

If this is the case then it should be relatively easy to make a generator for Mapmagic to create mesh terrains directly. Also there are a lot of grass/tree placement tools on the asset store or as free community project that will place grass/tree/objects over a mesh… or the same could be created in Mapmagic (or Terraincomposer, or Gaia)

15

Sorry, that’s what I’d figured you had meant, just wanted to make sure we were on the same page. Either tonight or sometime tomorrow, i’ll add in functionality for the system to run completely independent of any default Unity Terrain objects (Terrain, TerrainData). With that in mind however, the only reason I opted to leave that portion till later on was because I have yet to set up any interface to actually manipulate said data from Terrafirma’s side, though the api is there to do it (methods, datatypes, ect) , and aside from the built-in getHeight (3 options total) and getNormal (4 options total) options, the rest of the system is already independent of “TerrainData” Objects, and is actually referenced similarly, so you are correct in saying that it should be trivial to integrate support. And yes, of course, the option is actually already there. ;D You can enable Mesh Collider generation instead of using the Unity3D Terrain collider. And yah, they’re would be no issue supplementing Unitys vegetation system with another, a few things would need to be considered though. Something that relies on the same height map along with an accompanying detail map as the ones used by Terrafirma would be ideal, rather than using the mesh it self for referring to heights and storing data in complex data structures.

16

If you want I can test the mesh generation over heightmap and try integrating MapMagic maybe you could send me an alpha version … I have some free time this weekend…

17

I’m still a few week’s away from a stable build ready to publish, and I figure it’ll be a “last minute” thing getting the generate from height map portion of the editor up and running. Thanks for the interest however, I’ll definitely keep you in mind if/when I need testers. I am planning on releasing a “lite” version before release, though. I’ll leave in the methods that get height samples from .pngs Terrafirma, and add i’ll add in function for injecting the height map .pngs so you can mess around with that. Can’t promise thought any of that will be implemented visually however.

18

Almost Finished Interfacing the extractors, exporters, converters and utilities. As these are still a work in progress and a set of “injectors” are in the works as well, to compliment them, I decided to post a preview for the time being to give an idea of what the system offers in terms of utilities. (Note** these are all intended to operate on default Unity Terrain Objects**)

1024×491 280 KB

Other tools and Functions; 1 A - Control Extract (Not-Merged); 1 B Control Extract (Merged); 2 Height Extract; [u] __[/u]3 Detail Extract; 4 Mesh Converter; 5 GameObjectSwitcher; 6 Tree to GameObject;__
Also managed to take even more of a load off of the rendering loop finalizing the merge splatmap function. (Yayy!!!) Along with a major boost in load times if the object is built in the editor first.
Also posted two videos, a demonstration of 4 Unity Terrain Objects generated with MapMagic being converted and ran with Terrafirma in play mode along with an example video of the demo scene, available in the first post of this thread, along with brief descriptions. Links can be found below :
Demo
__https://www.youtube.com/watch?v=OeEAcSOAqMo*_
* __
[u]Conversion Example[/u]
__*https://www.youtube.com/watch?v=kulu7roTEb0*___

19

This looks fantastic and very promising. I am developing a couple of games for the Wii U and performance is always a concern. Is there any reason that this might not work on the Wii U? Any special requirements? I am super anxious to give this a try.

20

Let me know if you want to do an integration with MegaSplat. I already work with Meshes and Unity Terrains, support converting from any standard Unity Terrain, and the quality of my texturing is second to none on the Asset Store. It also runs at a consistent pixel shader cost no matter how many textures you use, so it can outperform Unity’s shaders on the low end.

6 Likes