How to handle truly open world maps with Unity, due to floating point limitations?

Unity Engine
1

Hello guys, I posted this at the feedback website, if you have tried making a large world, you know how hard it is with Unity today:
https://feedback.unity3d.com/suggestions/native-world-shifting-floating-origin-to-overcome-problems-due-to-7-digit-limitation-for-floats-on-transforms

I’m trying for years (!) to see if this works, but it is still hard, there is no solution for this problem… In details:

I’m trying to make a driving simulator with large routes (20km+), and this is almost impossible in Unity right now. Tiling, loading and unloading scenes work really well (async), but…
We need a native origin shifter feature to move everything keeping the camera at 0,0,0 when the player moves far than 999 units! (or 9999 in some cases). This is because the floating point used has only 7 digits. After 999 units (1000 to 9999) we get only three decimal digits… Then the camera shakes, shadow blinks, smaller objects appear out of place… That’s so bad :frowning:
In this video I explain all the problems, showing a sample of it working in a game (not mine) made with Unreal Engine:

The game shown in this video is Fernbus Simulator from TML Studios. I could not find any example of a game like this made with Unity yet, all mobile games have smaller static scenes with boring routes, or divided with loading screen between scenes (which I do not want and I will not make, that’s not natural). I want making some real routes from Brazil someday and they must be large, with real scale…
I heard that 64-bit float is not the right thing to do due to performance problems. But we need at least some API to move everything in an optimized way, where sound and other engine features continue working, like the new Navmesh… It’s dynamic but does not work if we move the world with moving agents. I’m almost giving up after some years trying to do this since Unity 3.x, but I don’t want switching engine just because of this little step… :frowning: I spent many years learning Unity as an indie, I like its interface, features, services, the company vision etc…
Is it really almost impossible making open world in this engine?! What would you recommend?
I’m using a floating origin like this, with some optimizations:
http://wiki.unity3d.com/index.php?title=Floating_Origin
In essence: we should not rely on global world coordinates, we should have an API to move the entire world seamlessly, where everything continues working (light, audio, physics, navmesh, etc). I think that it is not that hard making this in Unity, since floating origin almost does it. Almost…
Without this it is almost impossible making an open world game with Unity, at least a decent one, without many limitations on other features (be it on audio, physics or navmesh, for example…).
If you have any information about this feature, or other things that I could try, please send me an e-mail:
marcos@viamep.com
marquinhosxp@gmail.com
My game, made with Unity (beta, where the problem exists): https://play.google.com/store/apps/details?id=com.viamep.pbsu
I love Unity for everything that it has and also I am a Plus subscriber, but this issue is almost killing my project… There are many feature requests related to 64-bit floating point for transforms or origin shifting, but no official position from the devs yet… For years… :frowning:
Please if you can fix this huge problem, or at least give us some information on how to handle these large maps! The floating origin available at wiki is not perfect.

6 Likes
2

I’m pretty sure Kerbel Space Program uses a floating origin, and that was definitely made in Unity.

Switching to doubles (64-bit floats) would not solve this problem. Because of how IEEE floats work, the additional space does not really increase granularity all that much.\

I found this wiki page in a KSP forum: http://wiki.unity3d.com/index.php?title=Floating_Origin

1 Like
3

Thanks newjerseyrunner, but I posted it in my text, it works but with lots of drawbacks… I think that the engine itself should have something to help us with this “problem” in a more optimized way. Moving everything in a single frame for some games may cause a little delay in audio running (less than 1 second, but it is annoying for players), navmesh currently does not work, and many other things also still rely on global world coordinates (like static batching…). Open world was never the focus on Unity, I hope that it can get some features to facilitate this development someday…

1 Like
4
1 Like
5

It’s too complex to setup, I’m doing the same thing but in a simplified way (my scene streamer is working well, this is not the problem).

Native Unity features that require static positions stil won’t work with World Streamer… This is a move that the engine should take, everything that we can code is limited by its APIs…

By the way I read there that it supports dynamic navmesh, I’ll give it a try later… (although this is not the perfect solution, this asset changes the way we make the world). I still think/vote/ask for this as a native feature someday hehe

3 Likes
6

Plus, PhysX does not seem to support doubles either, according to following NVIDIA post from 2015:

https://devtalk.nvidia.com/default/topic/627742/physx-and-physics-modeling/can-i-change-pxreal-to-use-double-precision-in-the-free-version-of-physx-/post/4469496/#4469496

1 Like
7

This topic came up a few times in the forum in the past and I remember somebody suggested to put the entire world chunk under a root node and move that root node around.
3494471--278354--rootnode.png

If you use Unity 2018, you could try to use the C# Job System and the TransformAccess to move those chunks around in a multi-threaded way.

However, I believe it won’t work with static objects. On the other hand, you most likely can’t build a big world and mark everything as static object anyway. Unity combines static meshes for example and if your scene has a gazillion static trees for example, you probably run out of memory really fast.

I guess you want to implement a system that uses a predefined amount of objects and moves them around at runtime, basically reuses the same pool of objects and relocates them depending on the view. This could be implemented via C# Jobs as well. And you probably want to draw them via GPU instancing.

I don’t know how physics behave if you move the root node around though. I imagine physics doesn’t really like that, but I never tried, so not sure.

2 Likes
8

If we move everything in FixedUpdate physics seems not to be a problem… At least for now… It may vary depending on the script execution order for each project. The floating origin (script from wiki) moves things on LateUpdate, this breaks physics since it is not in sync with FixedUpdate. Changing it to FixedUpdate solved lots of problems for me. I think that Unity could create a native way to do it without interfering with user scripts, like moving everything always first, before calculating the next frame…

Actually I’m already doing this… My project uses blocks of 3km², all objects are positioned as a child of a root game object, and once player passes 999m it moves everything back to 0,0,0 considering the camera position. By getting to the borders other tiles are loaded (async) and most of the time it works really well. By checking the distance the system unloads scenes left far away. Most of the time there is only one or two scenes loaded (plus a manager in a persistent level, to control everything).

I spent many months making/testing the tiling/level streaming, while I could do the artistic or other parts… Every other person who wants to create something like this must reinvent the wheel…

This is a feature that ideally the engine should provide us some day. Checking if a scene is already loaded, positioning the gameobject close to the other, etc… It makes the level creation much harder than it should be, it is a very important feature supporting large worlds these days.

Even after I created the level streaming, the main problem is that many of the engine features does not work well with this kind of games, being the Navmesh one of the most frustrating one… Also I wish a native world streamer and multiple terrain editing (like Unreal), since coding everything from scratch everytime is very hard… It seems that better terrain tools are coming soon, that’s good thing, a step in the right direction.

Of course I’m considering it if Unity wants to be used for more open world projects in the future. Lots of new games could be made if this were integrated with it.

One last notice about moving everything once a while in FixedUpdate for me, is the sound… Depending on the speed that we are and the amount of objects in the scene, it gets an annoying “sound spike” for less than a second in all my projects. I still don’t know how to fix this. For a driving simulator in my tests we need to move everything after 999 meters driven (4 digits after the dot), if we go by 9999 (3 digits after the dot) there are problems with shadows and lower near clip planes. So each 1km driven we get a bad user experience due to this, even if the game is running at more than 90fps it shows a little “stop” in the experience… There must be a better way of doing it.

4 Likes
9

What’s causing the hitch, according to the Profiler?

10

Unfortunately I haven’t found it yet on the Profiler. It may be the act of moving everything, but apparently floating origin itself does not show high cpu overhead. Sometimes this does not occur, it’s strange (but most of the time it appears just in the exact moment that everything is moved). It never happens if we don’t move the entire world. I’ll try debugging it with Profiler again with more time.

11

That would be the first step to make sure to understand what’s going on and what’s exactly affecting performance. Once you have the results, you can see if it’s something you can further optimize or is an engine flaw / missing feature.

12

I tried it a lot, of course, but there is no visual thing there indicating the problem… It seems related to updating lots of colliders at the same time (moving everything is a bit heavy operation).
Some audios have the pitch changed in Update/FixedUpdate, first I thought that this was causing the problem due to differences in script execution order (some before, others after Floating Origin)… But even if I completely disable/remove them, and put a simple audio running (a single AudioSource with a looped AudioClip, no script changing it) the problem persists, so it is not any of my scripts that are causing that… Another time I’ll try recording a detailed video showing Profiler, or even better, sharing a sample small project where the problem appears.
The project runs fine at 250+ fps (while on low graphics, but with all simulation code running), but the little audio problem is there.

13

Unfortunately i’m not sure you’ll see soon some open world features in Unity.
Each open world game made in Unity uses it’s own system and it works pretty good like Pine game for example.
https://www.twirlbound.com/pine/blog/devblog70_asyncloads_acacias
This is the only way to make open world instead of waiting and never see the features coming.

14

I’m going a bit off topic, yet as an audio guy this comment makes me think of a few things…

Are the audio files in MP3 format? If so, does a WAV file with the simple audio experiment make the hiccup go away? Secondly, could using Time.unscaledDeltaTime with those specific audio files during the translation operation help? And finally, could adjusting the Audio DSP Buffer Settings slightly up or down help smooth the playback out? This last one is generally not recommended by Unity, yet with other pro audio applications and hardware, adjusting the buffer can sometimes quickly solve audio spikes and loss of sample rate sync.

These are all shots in the dark, yet might spark an idea and worth looking into.

1 Like
15

I’m not aware of any commercial games that don’t rely on floating origin. Games like Breath of the Wild, The Elder Scrolls, and so on are all built on engines and frameworks that according to you make it impossible to create a decent open world game.

It definitely wouldn’t solve every problem because it isn’t just Unity that is restricted to single precision. Both of the major physics engines on the market - Havok and PhysX - are restricted to and optimized for single precision too.

16

There is a move in this way coming, slowly, but it is happening… Multi scene editing in 5.x (without this it would be much harder making the seamless level borders), Dynamic Navmesh in 2017.x (although it is still not perfect)…

I’m not waiting, I’m making it with all those drawbacks, but at the same time considering other options… That’s the point of my message, there is a huge demand for features like these, soon or later Unity should take some decision if they will embrace or not open world by default, as they do really well for 2D, VR/AR and many other things…

17

There is UNIGINE that has this feature, but I don’t know how much performance it drops by going 64-bit for positions on everything. Would love to see some benchmarks with 32 vs 64-bit on popular desktop GPUs and devices. But paying $10k to discover is not a good option hehe (by the way there is a trial).

18

Most are wav but there are some mp3, I’ll check that later, thanks for the tips!

19

Well open world features are already available in UE4, sometimes it’s a matter of choice to pick up the right tool that will allow you to make what you want in a smooth way without needing to re invent yourself the wheel again.
Anyway good luck at your open world.

20

UE4 floating origin support is very limited, doesn’t support most of what people on these forums think it does. It’s not just flip a switch and navigation and physics just work. The hard problems are still yours to solve.

Most of the challenges aren’t really specific to floating origin. They are rather about what is the most efficient way to move things. Floating origin is simply a context that pushes some things pretty hard. So there is no magic bullet unity can even do here, floating origin ‘support’ is a misunderstanding of the core problem.

Yes they could allow setting an origin for things like navmesh. That kind of thing would be nice, but it also wouldn’t solve the really hard problems.

1 Like