I am having trouble implementing fog of war in my game and I hope someone can give me some pointers or ideas.
The way I have it set up now is by creating a mask texture covering the whole terrain and sampling it in the shaders to hide things.
The mask is 1024x1024 texture and I have max 20 vision sources which have a range of about 50x50 mask points.
The mask has 3 states - black for areas not visited, grey for areas visited but not currently in range and white for visible.
I also transition between these states to give it a bit of smoothness but that is done in the shader.
I update the mask in a parallel thread because it takes too long and when it’s done update the texture.
In the parallel thread I do this:
Stage 1: I go through all the vision sources and update the mask pixels in range. This is no problem
Stage 2: Check if the point is obstructed by terrain (a hill or something) via a terrain height map of the same resolution as the mask by walking all the pixels to the current one and checking if the terrain is higher than my line of sight. This works but has already slowed it down to about 4 updates per second.
Stage 3: This is the big problem. Check in real time if there is an object blocking the view.
Now for stage 3 I have been breaking my head for days now and everything I try is just to slow or fails. I could just update the height map but as it is only two dimensional that gives me the problem of an object the shape of a letter T casting a shadow of a box. Same thing happens if I just check the objects in range by bounding boxes. I really need a way to test visibility in 3D. Ray casting is out of the questing as that would be 50X50X20 ray casts and even if I limit it only to vision sources and objects that moved it would still take too long. And I can’t ray cast in a parallel thread. I tried with a custom octree and ray casting script that loads the objects at start but it slowed down to about one update per 10 seconds. Also there is a problem of updating the octree when objects move.
So that’s it.
Is there another way to go about it? Maybe a complete solution in a shader is possible (not really an expert here)? Is there a way I can do shadow volume with GL class to a texture if that is actually viable?
I welcome any and all ideas, suggestions, comments, …