Seamless Open World in Unity

Unity Engine
1

So, I’m switching over one of my games from Gamemaker to Unity 3D. In the Gamemaker prototype, I used chunk based loading to create a seamless exterior world, but I have no idea how to go about doing this in Unity Free (if possible).

The way I did it in Gamemaker was:
-Place all objects into one scene
-Programmed another application to parse the scene, and split all objects into different chunk files based on their x and y positions (ie. chunkX = floor(objectX / chunkSize)). Each file held 9 chunks.
-In runtime, I would then detect which chunks needed to be loaded, open the correct file, and load all objects for that chunk

It was fairly simple, although not the best way to go about it, I’m sure. What would be the best way to implement chunk based loading in Unity? I realize that there is the loadleveladditiveasync function, but that requires Unity Pro, and I don’t really have the money to purchase Pro…

2

Sadly, without the asynch loading, your game will “hang” every times you load something, unless you load very small chunks.

I would ask, why do you split your world in smaller chunk that way? Is it really huge, or made for a very limited platform such as mobile?

1 Like
3

Thanks for the reply.
So there is no way to do chunk loading without Pro version?

The world is intended to be very large, yes. I would like to make sure that the game runs smoothly for as many users as possible.

4

Not smoothly. If you break your chunks into smaller parts, and spread the loading of those parts over time, you can decrease how big a hitch there is at once. But to avoid the hitch entirely, you do need asynchronous loading.

5

That’s fine. Gamemaker didn’t have asynchronous functions, so I managed to imitate it so that it ran without any hitches. I think I know how to go about that in Unity as well, I just don’t know how to go about the rest of it.

Does anyone know?

6

Pretty much the same way as in gamemaker. You can add objects to your scene by using Instantiate and remove them with Destroy. As others have said, there may be a slight bit of lag as new cells are added and destroyed.

7

Thanks for the response.

Yeah, the problem I’m having is that I don’t know how to convert the Unity scene into chunk files. With Gamemaker, it was easy, since the scene files were just XML, I just coded a parser for that.

I also don’t know what kind of functions Unity has for dealing with text files. Is there an easy way to write an array’s data to a file?

8

You could look into solutions for version control with Unity scene files. You can’t easily merge Unity scene files as they are binary blobs, but you can build a tool to convert it to a text or XML representation and use version control on that (or parse through it to calculate your scene chunks).

I can’t direct you to a specific library, but it’s possible someone released some tools that do this already. Or you could always attempt to build one yourself.

9

Thanks for the reply.
I actually figured out a way to export all objects in the scene in XML, so I’ll probably use that.

I’m kind of worried about spending so much time making this though. If/when I do sell the game, and I purchase Unity Pro, this will pretty much be pointless, won’t it?

10

LTTP here, but I recently released SECTR STREAM, which handles a lot of the plumbing you were discussing here. I also just submitted a new version to the asset store with some new terrain/open-world specific tools. Feel free to post here or email support@makecodenow.com if you want more info.

11

Just watched basic SECTR STREAM-videos and used methods are pretty interesting. How would it translate to multiterrain scenes? What kind of terrain/open-world specific system new update is going to have? Are you making introduction video for these new features?

12

A multi-terrain scene uses the same plumbing, just with one sector per terrain and pass-through portals connecting the terrain sectors to one another.

The 1.0.3 Terrain beta includes a GUI tool that will build a grid of sectors and portals for a terrain. The sectors can be independent of the terrain, in the case where you want to stream the things on the terrain but not the terrain itself, or the tool can split the terrain for you into multiple terrain objects, preserving things like splats, grass, trees, etc. If your level is already split into multiple terrains, you can just add sectors to them, and then manually hook up the Sector neighbor attributes. Speaking of neighbors, the STREAM specific terrain logic knows about terrain neighbors, and will auto-connect and disconnect terrains as they stream in, making sure that the built in terrain LOD works correctly even with multiple terrains. All of these elements are in 1.0.3, as soon as it’s approved by the asset store team.

These tools will be official in 1.1 (coming in 2-3 weeks) and I’ll make custom docs, demos, and videos for that release. For now, I’m just ahead of schedule and wanted to share my work with folks and get feedback before the official release. The terrain sectorizing tool is in CORE (the free module), so anyone can download it, check it out, even use it without buying my other packages.

13

I was actually just about to bump this thread, and get some more suggestions on crafting an Open World.

Here is a screenshot of just one area of my game (that’s not even complete)

1550050--90836--$SceneSelected.png
1550050--90836--$SceneSelected.png1149×635 733 KB

I was planning on just crafting each area of the game inside this one scene (each “area” parented to a different game object, that is in turn parented to a “world” game object), and working out what to do with them later, but I figured that I should start worrying now.
I don’t even know how many thousands of objects there are (all of them are quads) already. At the moment, I’m not sure how many areas there will be total, but most likely at least 10 more, all larger than this one. When running the game, it takes forever to start up in editor (although it’s instant when compiled, oddly)

I was toying with the idea of actually using this scene as the gameworld (ie. not streaming in areas) but just having everything disabled by default. The player would then have a large trigger box around him that enables all objects it collides with. How feasible is this? Do disabled objects use memory?

SECTR STREAM looks pretty neat, but I’m not sure how useful it would be in my scenario. I don’t have Unity Pro, I’m not using Navmesh (I’m using A* Pathfinding Project), and I’m not using terrain. If I had Unity Pro, I could just split up my world into separate scenes, and then use the LoadLevelAdditiveAsynch function, couldn’t I?

What does everyone think the best method is?

14

If you’re making an open world game, you definitely want to start thinking about how you will handle scale as soon as possible. It’ll be your number one enemy throughout development. In particular, you want to think about three things:

  1. Rendering
  2. Memory
  3. Gameplay and AI overhead.

The common solutions are:

  1. Tons of LOD
  2. Streaming
  3. It depends on the game, but usually some sort of system that creates and destroys game objects in a bubble around the player, creating the illusion of a fully populated world.

For #1, I think the techniques are pretty straightforward, and you can use geometric LOD, distance culling, etc. One thing that you might consider is if your terrain is composed of thousands of individual quad-shaped game objects, you will probably want to have a system that can take all of the game data and build a single (or small number) of mesh objects and colliders out of that data. It’ll be much more efficient than thousands of individual batches.

For #2, Unity will, by default, load everything that is referenced by your scene when the scene is opened, regardless of whether its visible or not, enabled or not, etc. That can be a lot of memory, which is where streaming comes in. In Unity, the generally best way to stream is using LoadLevelAdditive (AssetBundles have a lot of problems). SECTR STREAM is basically built on top of LoadLevelAdditive, with tools to make import and export easy, runtime components to request loads, and runtime book keeping to handle things like lightmap streaming, multiple load requests, etc. Like any asset store plugin, you could write it yourself. The question is just if you want to :slight_smile:

For #3 that will really depend on the specifics of your game. The idea of a “big trigger that enables things” is a fine idea. One thing you should consider is if you need to specifically place all of the gameplay objects, or if they can be spawned semi-randomly around the player based on rules. Most open world games (Skyrim, GTA, Brutal Legend, etc) take the latter approach. Things like buildings and important characters are placed by hand, but most of the rest of the world is populated dynamically, based on rules like “in this area, there is a one in one hundred chance of a dragon appearing at night”.

Hope that helps. Good luck!

15

Thanks a lot for the response!

Well, the camera doesn’t render anything outside of view, and I do have distance culling enabled, so at the moment, rendering isn’t a big issue. I was originally using a texture atlas for the terrain, and then modifying the UVs to keep the draw calls down, but I stopped doing that for two reasons:
-I didn’t like the idea of having each terrain square having to call a function on startup to set proper UVs (that would be a lot of calls)
-I couldn’t get the UVs to change in editor (using an editor script) without either the material becoming instanced or it giving me an error, which made crafting areas a pain, since each terrain square had the graphic of the squished down texture atlas.

Now, I have a material for each individual piece of terrain (ie, sand, dirt, grass, cliff wall, etc). It’s kind of annoying to make prefab for each type of square, but I’m only at about 200 draw calls, even with all the lighting and soft shadows enabled (and normal/spec calculations), and it allows me to see what the terrain looks like in editor.

I was thinking about having a system that built a single mesh object from all the individual pieces of terrain, but I have no idea how to go about that in a way that won’t be pretty CPU intensive. I googled it a lot, but every conversation I found about it was doing this for procedurally generated worlds, which is much different than what I’m doing.
Wouldn’t I have to loop through each terrain object in the scene? And if I’m using a streaming approach, it would be best to grab them as they get loaded in and create a mesh from them, but as far as I’m aware, there is no easy way to return the objects loaded this frame.

Yeah, my plan was to basically have all terrain/environment prebuilt, then have it generate random places for spawning monsters, placing resource nodes, etc.

All in all, I don’t really like the idea of having everything loaded up when the game starts, so I would like to come up with a system that streams in areas, but I don’t have the money to purchase any assets (Or else I would have also bought NGUI, some dialog system, etc). I’m just trying to find a balance between ease-to-make and easy to run.

Once again, thanks for the responses. It’s great to have people to discuss techniques with / bounce ideas off of.

16

Yeah, discussing this stuff is fun. Of course, I like it when people buy my plugins, but the real reason I’m in this business is to help people make more awesome games, so always happy to help discuss ideas.

Regarding the mesh stuff, if you’re only at 200 draw calls, I wouldn’t worry about it, unless you’re targeting mobile (and of course that number will go up over time). If you ever do need to go the mesh route, I think you’d want to keep a few things in mind:

  • Even in editor, you would probably have a single “terrain” object with some custom GUI for placing and modifying tiles. Basically, like Unity Terrain. Or, you might make an in-editor “bake” button that builds a mesh from all of the tiles and makes a new scene that has that baked mesh instead of all the instanced tiles.
  • You can save Mesh assets to resources, even ones you create dynamically. This would get around the save and startup issues.
  • If you need to modify the mesh at runtime, you’d want to have a custom Terrain game object that manages the mesh, and it should be ok to update that mesh, provided you only do it when something changes, not every frame. You can also keep total costs down by “chunking” the world so that a change in one corner doesn’t force the entire map to rebuild. There are also some very smart mesh data structures out there that can help you very quickly figure out which verts and uvs to change based on which tile was changed.

For streaming stuff, basic usage of LoadLevelAddiitve is easy. Just know that w/ out Pro it’ll hitch/stall as new pieces come in, and there’s a lot of complexity if you want to handle lightmaps, arbitrary numbers of load requests, etc. A custom solution that’s just for your game might be very tractable, though.

17

Well, if it makes you feel any better, I probably would have bought your plugin if I had the money as it does look like it would have made things easier! Although, I’m not sure how well it would have worked with a Skyrim type approach to a world - are those portals necessary? What do you do when its not a corridor to the next section, it’s just a wide open plain?

I’m only targeting PC at the moment, and I don’t plan to ever port to mobile (It would be way too cumbersome to play this game on a mobile device).

The baking idea is actually really brilliant, I never even thought of it. Essentially it would loop through all terrain objects in the scene, merge them, save the new mesh to resources, then create a new scene that uses that mesh, correct?
Do you happen to know of any sources that would help me learn how to get started on that? I’m still pretty new to Unity, although I’ve learned a lot over these past few weeks.

There are times where I may need to modify the mesh at runtime. Although, since the whole way the game world works would be changing, I haven’t the slightest clue of how to go about this. Let me briefly go over what I was planning to do, and maybe you’ll be able to see a way I’d be able to do it with this method.

The main town of the game is dynamic, meaning that there is only a few buildings at the beginning of the game, but as you progress, more buildings are added. I was originally planning to have “gaps” in the terrain; holes where the buildings and their land would fit in. I would then create a “landmanager” prefab, and place one at a specific point in each gap (like the top left corner). In the manager for that area (Each area has a manager to keep track of resources and dynamics for that area) I would add each land manager to an array of all land plots in the town. Then, I would have a prefab for each building (made out of the same tiles that the rest of the game is - not 3D objects), and when a new one is added to the town, loop through the array to find the first open spot, and instantiate the prefab in that location. Open spots that have no building in them would just spawn regular grass tiles to fill up the terrain.

The part I’m worried most about not being able to do is the paths to each land plot. It would have been possible for each “landManager” prefab to have an array of tiles outside the plot to change to make a path. When no building is there, the tiles in the array would stay grass, but when a building is there, loop through all these tiles and change them to the path tiles. It would have even been possible to have different types of paths, depending on the building, such as dirt or brick. If everything is one mesh though, how would I loop through these tile locations to change the tiles?

EDIT: I forgot to mention: by Lightmaps do you mean baked ones? If I’m using dynamic light, it’s constantly generating a lightmap, isn’t it?

18

No worries on (not) buying SECTR. If it helps you at all, feel free to grab SECTR CORE. It’s free and has some useful tools and codez.

I think you get the core of how baking should work. For API, most of what you need to know is in the Mesh and AssetDatabase Unity docs. The rest is just figuring out how to compute the right data for your game.

I think your basic ideas about how to organize the world seem sound. If you’re going to modify the mesh dynamically, you’ll need a data structure that makes it easy to go from tiles to vertices (or vertex indices). There are lots of ways to do this, but start simply, maybe a Dictionary where the key is the tile (ID or index or object) and the value is a List of integers, where those integers are indexes into the Mesh vertices/uvs/normals arrays. You can get more complex from there.

Lastly, as with all programming problems, it’s good to think about the big picture, but don’t get overwhelmed. Pick a problem to start with, break it down into the smallest individual problems that you can, and then just work your way through them one at a time. You’ll get there :slight_smile:

19

PS - Regarding SECTR and a game like Skyrim, check out the Terrain tool in SECTR CORE. It shows how you can use the Sector/Portal graph even outdoors.

20

Awesome, thanks a lot for the help and advice, I really appreciate it!

Also, do you happen to know of any information I could look at to make the “custom GUI” you mentioned for placing and modifying tiles? I want to explore both possibilities so that I know which is best to go with. I assume this way would actually be slightly better performance-wise, as I wouldn’t have to save it into the resources file (I read that anything in resources is actually kept in memory).

I think for calculating the path to buildings, if I stick with the baking method, I’ll have the landManager prefab take in UV coordinates of the new mesh instead of using tiles. I’m sure that there is probably a way that will allow me to “paint” the UVs of the mesh that I want to edit, and have those stored in an array (although I haven’t the slightest clue how to go about that). That way I can just build only the terrain in one scene, bake it, then in the new baked scene, I will have all the objects, etc.
Assuming it’s possible to change the texture at the specified UV coords to a completely new texture? I have no idea how merging multiple textures and meshes into one actually works…

Which also brings up another question (Sorry, hopefully this will be the last!). Right now I have each tile with it’s own material, allowing me to adjust the shininess and Fresnel values for each type, so that rocks will have different specular lighting than grass, for example. When I merge all of these into one mesh, won’t it just have one material for the whole mesh? Wouldn’t this mean I would lose all the individual values for each type of texture?
It’s not a huge deal, of course, but it would be nice to maintain the different looks for different types of ground/walls.