Hey!
Since I’ve been in the same position some years ago when my mates and I started working on Northern Lights which is also open world, I can give you some advice in regards to your game setup (our world is similar - 12k x 12k terrain as island, locations with dense forest, easily surpassing 100k nature objects in world per total)
First things first, make sure your prefabs are set up correctly. If you use colliders on trees and rocks, make sure they are either primitive colliders (box/sphere/capsule collider components etc - i.e capsules work great for tree trunks) or mesh colliders set to convex. You can’t have non-convex mesh colliders on thousands of objects as it will kill the CPU performance - it’s been a while since I digged into this and I might remember it wrong, but long story short, if you use regular mesh colliders on your objects, they won’t get baked physics at runtime and each collider will get updated every physics tick which will kill your cpu performance.
Also don’t exagerate with LODs on 3d objects, as modern GPUs can normally handle lots of polygons on screen and extra LODs are just going to increase the drawcalls and add extra popping when you move around. One high quality lod up close and a low quality distance lod seems to be enough in our case. Our trees also have billboards at very long distances.
These two are only a part of the problem however.
A big issue we hit with unity, you just can’t render that many objects without hitting some kind of renderer limitation, and you probably shouldn’t anyway. Now, the issue is that Static Batching is out of question (with that many objects it just crashes or runs like ass due to insufficient memory), and the GPU Instancing thats built in the surface shaders will just get overwhelmed with that many objects and results in very bad performance. So we ended up using GPU Instancer from asset store: https://assetstore.unity.com/packages/tools/utilities/gpu-instancer-117566. It’s a bit pricy for a newbie, but if you are serious about making the game at that scale, its worth every penny. This alone boosted our framerate from ~20fps on a gtx1080 to around 100. It also handles occlusion culling and other neat optimizations at a GPU level for you so you don’t have to worry about objects being rendered behind walls etc. Or you can learn about the rendering/mesh drawing API of unity and make your own instance manager but thats rather complex and require some knowledge in how graphics rendering works, would take you a while and its just not worth the time if you are just trying to make an indie game IMO.
P.S. using Deferred rendering instead of Forward with thousands of vegetation objects seems to run better (at least in our case)
So to sum it up so far, prefabs set up correctly, and finding a better way to handle rendering your objects via instancing would be the first steps to make a performant open world game in unity.
Now whats left is handling the scene’s data. With that many objects, the scene can be a memory and cpu hog on its own, since it has to store data for each object of your scene in ram (which will also result in slow loading times in builds). You will have to find a way to chunk your world into smaller scenes that you will have to stream (via LoadLevelAsync or some third party plugin) as the player travels the world. Now this also depends on how far you can see in the game. Since our game has a winter setup, it’s easy to mask the world chunks loading at runtime since the fog and snow particles cover them up at far distances. If your player can see far in the world (being on top of a mountain etc.) I would suggest experimenting with keeping bigger objects such as trees and big cliffs in the main scene and only chunk the small detail objects such as rocks, bushes, decor as they won’t be visible far anyways.
Oh and Unity’s Profiler window is your friend! Whenever the game runs bad, you can check out whats killing your performance by looking at what takes the most miliseconds to execute.
Now, some real talk advice, if this is your first game and you are serious about it (i.e. from a financial standpoint), or you haven’t built any game seriously before, I suggest you should start with a smaller project. I don’t want to demoralize or scare you about game development in any way or something, I’m just talking from my personal experience with our game. When we started our project, we had little idea about pc game development - we had a background in software development and 2d mobile games - and looking back at it I wish there was someone to give us all this advice. We spent at least one year to get a game this size to work good in unity and it has already eaten 4 years of our lives in total, and the game itself is still not done nor released (but getting there this summer!). I honestly think that if I was to go back in time and start again we wouldn’t pick on a project this big again .
Anyways I hope this post will help you or anyone else looking for these kind of answers. I normally don’t get involved in this side of the forums giving advice, but I saw your post and remembered I was facing the same issue a few years ago so I had to help in some way