Fastest/best method for random map generation?

Unity Engine
,
1

Hi, I’m generating a random map of 10,000 x 10,000 tiles for a crafting game, however it takes about 240 seconds (~4 minutes) minutes to complete.

The problem is when the user loads their game, it generates the map using a saved seed, and then applies all their changes to the map, but that’s a 4+ minute load each time! I thought of using jobs but have a few questions.

  1. The generation also draws pixels on a texture for the minimap, and as far as I’m aware you cannot pass textures to/from jobs. So how could this be done?

  2. It uses a 2D array for the tiles (Tiles[,]), but can this even be used in jobs, or can they only use 1D NativeArrays?

  3. The resulting 2D array being 10k x 10k of tile structs is massive, will this be a problem on modern hardware?

  4. Is there any other good method of generating maps? I’m also using the FastNoise plugin which has a SIMD feature but I’m not sure if that makes a difference.

Thanks!

2

First question, are you doing any chunking? If not, you should probably do that.

Also don’t use such large 2D arrays. Use one dimensional arrays. Just takes a bit of math to go from 2D coordinates to 1D and back.

As I already mentioned in your last thread, you’re restricted in the types that you can use for jobs. I already linked to the resource for that it so I won’t link it again. Again, jobs is best used for small, quick, number crunching work.

Most importantly though, have you profiled to see where the majority of the overhead is? 240 seconds sounds like an absurd amount of time. Surely there are some simple but big gains to be made by a bit of profiling and working out what is taking the longest amount of time.

1 Like
3

Chunking for the generation no, as the mini map will need to be fully visible by the player and thus needs the full generation all at once. However it does load the 3D blocks in chunks as they move through the world.

Oh are 2D arrays that bad? I could definitely change it to a larger 1D array if it makes that big of a difference?

Yeah I already moved all the noise generation into the jobs with the idea that each job can calculate sections of the map. The only thing I can’t figure out is how to return texture info for the minimap, without just returning an array and looping over the whole thing – thus defeating the purpose of the jobs

I have not profiled yet no, but the code is pretty simple. It loops through the 2D array, checks 3 diff noises for the 3 different biomes, and just sets a texture pixel color. The 240s is just from generating the pixel colors, nothing else. I’ll try profiling tomorrow but don’t see any way to reduce that code really.

I think the main issue is just the 100,000,000 loop lol. I thought splitting it into many different jobs could’ve worked well.

4

Generation can still be chunked. I wouldn’t even be touching the minimap until the entire generation is done. I would be doing a low fidelity pass over the map to build it (depends on its style of course). Or, represent the minimap data as simple data that is used throughout the jobs. Do not even touch the texture side of things until generation is done, though, as you want to write to the texture all in one go.

If you were writing individual pixels at a time, that is going to be a huge source of slowdown.

Then profile it. There is genuinely no point coming to us to ask for help how to optimise things when you don’t even where the performance issue is.

5

Okay reading this again, yes, writing tons of individual pixels is incredibly slow.

Prepare all the data, and write to the texture in one go.

But also still get in the habit of profiling yourself.

1 Like
6

At that kind of scale you want to go all-in with Jobs, Mathematics, Collections. Value types and native collections only. And as Spiney said: chunking.

In its naive form your 100,000,000 (one hundred million!) tiles array will consume no less than one hundred times sizeof(Tile) megabytes of memory. If Tiles contains four int that will be 16x100 = 1.6 GB of memory.

Do you at some point intend to save that as a file on the user‘s system?

7

Looks like the big slowdowns are:

  • FastNoise plugin
  • Dictionary lookup to get the proper biome noise (ex: Noises[“hills”].GetNoise())
  • Dictionary lookup to get the proper tile color (ex: SetPixel(x,y, GetItem(“Grass”).Color))

So for now, while testing map generation in editor, I will separate it into jobs, followed minimap creation. With the obvious removal of dictionaries and changing it into a 1D array.

Later I’ll optimize gameplay to use chunking & further avoid the long generation times and giant memory footprint.

And no lol. I tried it on my VERY first attempt months ago and noticed that a 20x20 map was like 6mb.
So I’m just saving modified tiles, and then will apply them to a generated world when loading them back up.

Also that FastNoise plugin also contains a SIMD version which is supposed to be faster, but is also much harder to use. I can’t find much info, do you know about SIMD noise? Is it worth it?

Edit:
I did as I suggested > Transfering all noise & generation algorithms into jobs, then building the minimap after (so I can see the results). a 100mil tile map went from 240s to 30s. and a 20mil map went from 86 seconds to 7 seconds. Massive improvement.

1 Like