ECS and Voxel engine

Unity Engine
,
1

Hello. I’m starting to learn ecs approach. I have a problem with making a voxel engine in ecs style.

First problem:
I want to have entities called chunks(not ecs chunks - When I refer to chunk I mean entity with voxel component on it) on which I will have position component and VoxelsComponent. But my first problem is with VoxelComponent. How can I store array in it? I know the limitations on IComponentData and I know why it should have fixed bittable size but if I want create array with predefined const size this should be possible. If I know size at the compile time int[16] should be no different than 16 ints. How can I achive this?

Second problem:
Chunk entities are not independant of eachother. In most parts they are but for rendering purposes I need to access neighbours chunks(for face culling). What is the best approach for it? Iterating over all chunks and checking their positions seems as not very optimized way to do it. Storing entities in dictionary <position,chunk> would work but if I understand correctly this defeats the purpose of ecs and linear memory access. Theoreticaly if I have 16x16 chunks iterating over all of them should not be that much of a bottleneck but for 128x128 it starts to be unefficient.

I’m new to the ecs concept so if I don’t see something obvious I apologize :slight_smile:

1 Like
2

You can use
fixed int data[16];
if you know the size of array but I think this only works for primitive types.
For everything else you can use IBufferElementData which is a component type that can hold an array.

As for second problem, SharedComponent might be a good candidate for spatially grouping chunks. I don’t know much about voxel systems but there has been several discussions on it in this forum so I’d do a search and see what others have come up with already.

1 Like
3

Thanks for answer. Fixed mean unsafe. I guess in this case it isn’t that big of a deal. I will look up forum and shared component. To clarify lets say we have chunk that have 10 voxels in a row and it is placed on position (0,0). Next chunk have the same size and it is placed on (10,0). To render first chunk I can do it without the second one but voxels on edge will be renderer fully because I don’t know what is next so I can’t decide if it will be visible. For example if on (10,0) voxel is air then voxel on (9,0) should have right face but if (10,0) is solid, I don’t need to render (9,0) right face because it will not be visible. This example scale to x,y,z dimensions so every chunk needs to know its neighbours. I have problem with this not because I don’t know how to achive this, the problem is with how to achive this in good terms with ecs in which I don’t feel comfortable yet so I don’t know in which direction to go. Key points are - there can be many chunks in my game so iteration is not the option. In oop implementation I stored chunks in dictionary so no matter the size of the world I could get chunks in the same time but dictionary with references to different chunks from what I understand is againts DOD. I will look up other threads about voxel engines but google didn’t provided me with much informations so any help would be great.

Edit. I need to store only primitive types in array so it is actually a great solution. Thanks :slight_smile:

4

You can have component per each voxel block (entity), which holds 6 entities in component data, left, right, front, back, up, down.
Now you can rapidly access neighbour entities.

Or If you store your chunk of blocks in an array, you can easily access neighbours by knowing index offset. Which probably is much cheaper, than having left right etc, per each block.

1 Like
5

The problem isn’t about accessing neighbour voxel in the same chunk(entity) but across multiple entities. Each enity has array of voxels and edge voxels needs to know other entity edge voxels. I can’t store entities in an array because it is multiplayer game server. Multiple players can be in completly different locations. If I render 4 chunk(entities) for each player I can have chunks loaded for position (-1000,0,0) and for (xx,xx,xx) position. You can say that I should then treat each user chunks separatly and store them separatly but this won’t work - what if 2 players have the same positions but each of them have his own representation of terrain under him? Like I said before. I could iterate over all chunks. Check their positions. If positions suggest that chunks are next to eachother then proceed with accessing this chunk voxel array but I need ecs way of handling this neighbour chunk searching. Each entity as a voxel is not optimal solution. It is too slow and memory consuming.

6

Doesn’t matter with chunk with blocks you are accessing, when you store them in an array. Each chunk holds same structure of blocks. Weather is 2d or 3d. You select correct chunk, then calculate offset index to get block at the edge, then get adjacent chunk and calculate offset index of corresponding near block. And you can show hide faces if you like. In the end, you need store blocks in serialized form as array/database.

7

But it would not be efficient. Chunks(entities) in array is impossible or completly overkill when we don’t have a “smooth” world. We have for example 16 chunks at player position (-100,-100,0) and another 16 chunks at position (200,200,0). In beetwen them we dont have chunks because there is no player there so it is not needed. What good does it give me to store chunks in array if position in array doesnt mean they are next to eachother. The only scenario when they would be next to eachother in array like that is if I would create array big enough to store the whole world into it from position -10000 to 10000 on every axis flatten to 1d array which mean insanely huge array in memory. I have voxels in array so if I want to serialize it i save it as a chunk/part of the world into file. I need to have separate entities(chunks) with arrays to store my data because I want to create “infinite” terrain so I need to split world data into smaller chunks. The problem is again about finding neighbour chunk to another chunk - neighbour voxel array to another neighbour voxel array. I can have multiple groups of this voxel arrays and they can be as far away of eachother as possible. So in array we would have one chunk[100] here and another chunk[100000000] and the number increase depending on how far away players are from eachother. This would not scale to many players and world sizes. I don’t have constant number of chunks. I have constant number of voxels in chunk. Chunk/entities are created dynamicly depending on players count and their view range. I had this implemented in oop. All chunk were stored in dictionary<position, chunk> so if chunk wanted another chunk it would simple do world.get(position + offset).

8

What I mean is, having voxel blocks in a chunk, stored as 1d array.

But if you want strictly find near chunks only, I would add neighbour component, as I mentioned earlier. But in this case, probably only 4 neighbours, left, right, front, back.
Having 10k scattered chunks all over the world, won’t break the memory bank at all for that reason. You pay significantly more in memory, for storing blocks (voxels) in each chunk.

Then simply knowing in which chunk player is, you can get cheaply 9, 25 etc surrounding chunks in blink of eye.

And will work for any player in world.

9

I don’t know if we understand eachother correctly. I will put this in non programming way. I’m looking for best approach in ecs to handle given case. I have book library that potentialy can store infinite number of books. I need them to be sorted by name the most efficent way. You gave my an array as a potential solution. There is a problem with it. If I already put books with letter K in my array and someone give me a book with letter A I need to shift the whole array to insert it. Also if my array can hold 1000 books and I just got 1001 what should I do? In oop I would create dictionary for fast access to books in different positions.

Edit. Didn’t see your next response. So having reference to neighbours is acceptable in ecs? And about array. I acctualy have 3d array flatten to 1d in my IComponentData attached to entity(chunk). What do you mean that entity + array of voxel is more memory consuming than entity + voxel. If I understand correctly in one case we have memory for one entity + voxel_array[100] and in the other one we have (memory for one entity + voxel) * 100. In numbers the example difference would be 1 + 1*100 or (1 + 1) * 100? Or I don’t get it correctly?

10

Why not? Is just solution to the problem, like many others.

If that are voxels blocks that is correct. You probably want fixed size of an array for simplicity.
In the end you have blocks like air. Minecraft probably have some form of compression, of each chunk, where blocks groups are the same. I don’t mean mesh. Just indexing. But otherwise, if you have fixed array size per chunk, you can find easy any block in that chunk.

If you consider vertical chunks as well, then air blocks, or same blocks per chunk, could be significantly optimized, in terms of memory, as instead holding array of x voxels blocks. you could have single block type. At least until chunk’s blocks are changed.

What I meant, is that array of voxels blocks in each chunk takes significantly more memory, than memory taken, by chunk neighbors. You can have for example (3D) 10x10x10 = 1000 voxels blocks per chunk. While only 4 (or 6 with up/down) neighbors chunk (entities) per chunk entity. Hence cost is insignificant of storing neighbors. If that makes sense?

Just to be aware, if storing entity, you store two ints. Entity index and version. Hence in fact, 4 (or 6) become 8 (or 12) ints.

You could even storing neighbors optimize more, in terms of memory, but I don’t want to get into that at this point, until above is clarified.

11

I know it is solution but because I’m new with ecs I’m beeing careful not to borrow solutions from oop into ecs. Mental shift from one to other may push me into using oop techniques into ecs which would not be good. Mostly that is why I’m here. I could implement it in many ways but I want the “ecs friendly” way. It was like that when I was learning rust-lang. I was constantly trying to structure my code as oop because my only programming experience was with oop.

Ok now I understand what you meant. Stroing neighbours per chunk = max of 6 neighbours per entity. I can survive with that. Thanks for helping. I have few more questions but I will ask them after I implement what I already know. Maybe my questions will answer themselfs after I write some code.

2 Likes
12

Hi, I’m trying to port my voxel engine to ecs and I’d like some advice on how to structure things.
I store voxels in a flat array and access adjacent through their index.

public struct Chunk : ISharedComponentData
{
    public int Index;
    public NativeArray<Voxel> Voxels;
}

Jobs that only requires one chunk works great but I’m having trouble on how to write jobs that require multiple chunks. Jobs that need multiple chunks are for example:
Occlusion Job, that hides faces that aren’t seen.
Pathfinding Job, that traverses the world.

The Occlusion Job only requires that I know the six adjacent chunk of a given chunk. In this thread, Antypodish suggested that each chunk has a Neighbour Component.

public struct ChunkNeighbour : ISharedComponentData
{
    public Chunk Back;
    public Chunk Front;
    public Chunk Right;
    public Chunk Left;
    public Chunk Up;
    public Chunk Down;
}

This works for the Occlusion Job as I can access the adjacent chunks. Now my problem is on how to scale this solution. The Pathfinding Job would require more than just the six adjacent chunks and I’m lost on how to access them.
My thought was to use this Neighbour Component to traverse the world like you would do in a linked list or a tree - but I can’t access a ISharedComponentData in a job.

public struct PathfindingJob : IJob
{
    public Entity Entity;
    // N number of chunks that I work with
    // (doesn't work)
    public NativeArray<Voxel>[] SearchSpace;
}

I’ve tried with collections like DynamicBuffer<NativeArray> but it doesn’t work since it’s not blittable.
My first thougt was to store the entire world inside a NativeArray and access chunks by an offset. But this must be terrible for job access and scheduling.

One solution would be to copy all the worlds (or relevant chunks) voxels to a new NativeArray before the job. This could require me to copy an enormous amount of data each job.

I also thought that it might be possible - to inside the job use a NativeArray, where the ints are the pointers to each chunk, and access them in an unsafe way.

So essentially the problems I have, are using either in a job:
Accessing ISharedComponentData,
A collection of NativeArray with unknown size

Thank for you any thoughts or pointers!

13

If you consider using chunk as entity, with component of 6 neighbor chunks, then when you start from chunk A, you run your path finding, as similar as in 2D grid. Only this one is in 3D. The main difference is, you don’t use index in 2D/3D array, but index and version of neigbour entity.

Also, if you consider highly scalable in any direction, you could use array, but either you need give it right capacity at initialization, or need resize, when more chunks are discovered, than current capacity. Otherwise, using chunk as entity will be suitable.

In job, you just follow relevant direction. For example forward, then each every consecutive neighbor entity with neigbour component, you will read forward entity reference to next chunk. Repeat searching loop, until goal is reached. Or is failed.

Sorry, this post may be a bit hasty, as I typed it quickly. Let me know, if you need clarification.

1 Like
14

In fact, you just need a method where you pass the voxel coordinate (any one of them) and them if returns the block type you have at this position. It returns na index that let´s you look in a 1d array of block types. Just need some calculations inside, like Perlin Noise 3D, or thresholds, like Sand in the 3rd first blocks on top, rock below, etc…

Dont need to know the chunk the voxel is, just the position.

15

That’s all fine, as long you know size of voxel array.
This approach become more complicated, when considering undefined size of voxel array.

Imagine for example minecraft world.
If looking at blocks per chunk, you know their position perfectly. Fixed array size works in such case very well.
But looking at chunks in the world, they can be generated in any direction, for indefinite distance. Hence there isn’t any logical ordering in the array. Other than referencing neighbours. Quadtree and octree can assist in such cases, as it dosn’t care, where and what lies in such tree structure.

16

N

In my approaching of Voxel maps, I pass the world position to each voxel, summing the position to the coordinate, so my function to calculate the voxel block Works with absolute values. I use 16x16 chunks with a perlinNoise2d function to calculate Heights. As they use absolute world coordinates, every chunk is rendered at same position, but the mesh has the distance offset, and both functions can calculate any voxel in the world regardless the distance

public int GetVoxelHeight(int x, int y)
    {
        //Low tercian height is the minimum height of a chunk, while High Tercian Height is the variance between 0 and the value
        //of it added.
        return biomes[CurrentBiome].LowTercianHeight +
                    Mathf.FloorToInt(biomes[CurrentBiome].HighTercianHeight *
                    Noise.GetPerlin2D(this, new Vector2Int(x, y), biomes[CurrentBiome].Offset, biomes[CurrentBiome].TerrainScale));

    }


public byte GetVoxel(Vector3Int voxel)
    {
        if (voxel.y < 0 || voxel.y > GetVoxelHeight(voxel.x, voxel.z) - 1) return 0;
        else if (voxel.y == 0) return 1;
        else if (voxel.y > 0 && voxel.y < GetVoxelHeight(voxel.x, voxel.z) - 5) return 3;
        else if (voxel.y >= GetVoxelHeight(voxel.x, voxel.z) - 5 && voxel.y < GetVoxelHeight(voxel.x, voxel.z) - 1) return 4;
        else if (voxel.y == GetVoxelHeight(voxel.x, voxel.z) - 1) return 2;
        else return 0;
    }

public bool DrawFace(Vector3Int voxel, int face)
    {
        if (voxel.y < 0) return false;
        switch (face)
        {
            case 0: return blocks[GetVoxel(new Vector3Int(voxel.x, voxel.y, voxel.z - 1))].IsSolid;
            case 1: return blocks[GetVoxel(new Vector3Int(voxel.x, voxel.y, voxel.z + 1))].IsSolid;
            case 2: return blocks[GetVoxel(new Vector3Int(voxel.x, voxel.y + 1, voxel.z))].IsSolid;
            case 3: return blocks[GetVoxel(new Vector3Int(voxel.x, voxel.y - 1, voxel.z))].IsSolid;
            case 4: return blocks[GetVoxel(new Vector3Int(voxel.x - 1, voxel.y, voxel.z))].IsSolid;
            case 5: return blocks[GetVoxel(new Vector3Int(voxel.x + 1, voxel.y, voxel.z))].IsSolid;
            default: return true;
        }
    }
17

@FernandoDarci I understand your approach. And indeed, your approach to render voxels is functional.

My question however will be following, providing you haven’t answered it yet:
How do you store chunks of the world, and how you reference them? Lets say, you got chunks on both ends of the world? Or if you want render 24 chunks nearby?

You can have also a situation, where some chunks are detached from the rest and can be located in any arbitrary position of the map.

18

A pure ECS Voxel terrain is not the best approach, simple the amount of data i to large. As example if you has a terrain with 64 * 64 * 16 chunks with 161616 blocks, that makes 268,435,456 blocks. Each entity has two ints as value (Id, Version) that makes 2GB ram only Entity data, without any other data like Voxel type.

19

@runner78 it is irrelevant weather is ECS or not. Described problem applies to any type of approach or language.

The thing is, it matters how designer decides, how chunks are generated and allocated. You don’t want typically use a fixed size grid for chunks, but only populate chunks, which were discovered. Unless is strictly predefined world from the beginning, with all pre-generates chunks. Which is typically not what you want, or need for sandboxes open world based games.

However, it is not uncommon, for Minecraft worlds weight many GB of storage space. Special when traveling a lot.

Compression and frequent resetting chunks techniques, can help ro reduce and free memory of unused chunks.

20

One way to really cut down on your world filesize (assuming your generation algorithm is deterministic) is to flag chunks and voxels only once they’re modified. When you load in a chunk, you check for modified voxels within the chunk first, and let the rest default back to the world generation algorithm.

2 Likes