How to organize the data for Flowfield Pathfinding? What Datastructures to use?

Hello everyone,

I am trying to implement a Flow Field pathfinding alogrithm similar to the on described in this paper here.
This algorithm is also described and discussed by @eizenhorn [in his post stickied in this forum]( Unity DOTS case study in production page-2).

I am wondering what would be a good approach to store and access the necessary pieces of data.

I intend to have a Map that is divided into sections.

For the sake of an example let’s say we have 5*5 sections.

Each section is a 10*10 grid and for each grid cell I want to store some flow value.
The flow value indicates where a unit stanging on this grid cell should move.

Why do I need sections? I’d like to cache some of these to reuse them later. Also I think there might be situations, where I only need small areas of the flowfield and ideally can handle smaller amounts of data at a time. (Does that make sense?)

My initial idea was to have an entity for each FlowSection that contains an int2 representing this sections position on the map and a DynamicBuffer with InternalBufferCapacity(100) that contains the flow values.

Then I thought I will maintain another entity, a MapEntity that stores a DynamicBuffer containing the FlowSectionEntities.
However I learned that indirectly looking up Component data of Entites is not recommended to do.

So my questions are:

  1. Should I use Entities for MapSections?
    or

  2. Should I store this data as NativeArrays in the Systems I use them in?
    or

  3. Should every cell be an Entity?
    or

  4. Something else?

  5. Depending on 1-4: How to aggregate this data into some kind of Map representation?

  6. How to cache this data?

Maybe to illustrate what I will need in a system at some point (PSEUDOCODE):

Entities.ForEach((ref MovingEntityData movingEntityData, ref Translation translation) => {

var index = GetCellIndexFromWorldPosition(translation);

var MoveDir = flowFieldData[index];
...
}

The crucial part being the indexing of the flowFieldData.

All insights, thoughts and directions to look in very much appreciated.

Cheers

Vlad

Hi, so not everything needs to be an entity.

If your data is best to be stored in a raw format, e.g. as a memory block of some size of sorts, you can just plain store it in a native container. That could be NativeArray, or any other one.

Those can be freely be passed around, and if they’re readonly, its pretty safe to read from them in a jobs as well.

That is not true exactly. If problem is a random access one - can’t avoid having random accesses.
While its a bit “slower” from CPU perspective, in reality, impact of non-linear access is quite low.
Burst & Jobs in general makes up for it tenfolds.

So, if you want to access something on another entity, ComponentDataFromEntity or BufferFromEntity is exactly for that.

Probably not, its not really efficient to make lots of entities for temporary or “static” data. You could though.

This is better. Use single system that runs before anything else and makes temporary data required for you.
You can access native containers from systems just fine, as long as you take dependency from that system and combine it. This way container can be read safely from the jobs inside game logic systems. Alike a lookup system.

Rest depends on the algorithm you’re trying to implement. See what fits your needs.

There are native containers out there that can be used for spatial partitioning, alike:
https://github.com/marijnz/NativeOctree
and
https://github.com/marijnz/NativeQuadtree

I’ve used both of these, and they’re pretty good with insertion & lookup times.
So if its fast lookup by position, its worth using something of alike.

Alternatively, grid smashed into 1D array also works, although, its less efficient. Probably worth making some kind of a spatial partitioning tree. Although, if its a static grid, it may work anyway.

Haven’t done anything alike described in the paper, so maybe worth asking Eizenhorn what’s the data structure’s used.

1 Like

Well, despite I’ve described already our approach in linked thread, but maybe worth to go a bit deeper here :slight_smile:
In our game we have map, which can be any size (depends on loaded mission, endless mode ,challenge mode etc.), at level load we starting to generate all data required for map (as our levels procedural generated we can’t cache it AOT). In our game map built from map cells (each map cell is 5x5 Unity meters or 5x5 unit cells) for example map 100x100 in cell size will be 500x500 in Unity meters (unit cells 1x1). Then we have implemented idea of Areas and Regions (combination of approaches mentioned in paper linked in my thread and here and approach which is developer of RimWold using). Region is a map segment with a maximum size of 20x20 Unity meters (unit cells) in our game, from every cell in the Region you are guaranteed to be able to go to any other cell in the same Region. Area is list of connected Regions - from every Region in Area you are guaranteed you can reach any other Region in the same Area, as result from every unit cell (1x1) you are guaranteed you can reach any other unit cell in the same Area.
On image below, green cells - unit cells (1x1), gray cells - map cells (5x5), orange cells - maximum Region cells (but it doesn’t mean it’s always a Region of that size, it can be multiple Regions in that orange cell, I’ll describe that below)

Example below - green cells - is unit cells of one Region, rest of space in 20x20 Region cells are blocked as result we calculate bounds of Region, for simplifying calculations next and reducing unnecessary cells processing, yellow lines is borders of currently hovered Region. Blocks with white borders - is adjacent Regions connected with current one.

And as I mentioned before in one 20x20 cell can be multiple Regions

Example below is Areas - inside walls (closed area) is one Area, outside walls - another. Third one is example when we’ve opened gates, as result two Areas becomes one.


Every time in game something is built, destroyed, gates opened\closed, we invalidate Regions in which bounds it’s happened, invalidate and remove flow fields generated for that region, and recalculate new Regions on their place, update adjacent Regions and recalculate Area (in case new closed Area appeared or previously closed becomes connected with any other). As result instead of recalculating whole map every time we only recalculate very small piece of map. All this data is enough for everything in the game - we always can easily early out when we need to check if unit can reach point or not (just compare Area ID where unit stands and Area ID where it should go - if they’re the same - he can if not - can’t). By this data we can easily build hierarchical A* pathfinding for crowds - we just finding the path on Regions (20x20) cells and we can easily merge them if multiple units stands in the same Region and finding path to the another Region (again all requests path to the same Region) we just finding path once for all of them. The same for building flow fields - instead of building field for every unit for whole map - we just build it for small region and for all units in the same Region moving to the same Region edge - it will be the same, as result we generate it once and cache, and when someone will find path through this region to the same region edge - we don’t need to recalculate that - it’s already been calculated we just use already created flow field, and many many other things.

How it’s implemented. We have our custom native container GlobalMap which is associated with entity with class(!) IComponentData, as result it’s stored on singleton entity, which can be tracked in dependency chains (we have custom mechanism for dependency tracking in systems which read\write from\to native containers, in case of GlobalMap most of systems - read, and only one - write, as result dependency chain pretty straightforward). Inside this container we have many unsafe stuff. Arrays of different cell states (map cells) which indicates it’s water cell, or grass, or forest or cliff etc. Also it’s store multiple unsafe maps like UnitCell->RegionData, AreaId->Area, RegionLinkHash->RegionLinkData, PathKey->FlowFieldKey, and unsafe
FlowField native container which is storing map FlowFieldKey->RegionFlowField, where RegionFlowField is unsafe struct storing flow field vectors for every cell in this Region. And pathfinding itself doing some tricky unsafe stuff and allocate persistent memory right inside job when calculate and populate flow fields, we using our custom jobs for pathfinding, couple of additional unsafe structs for effective calculations, and heavily rely on IJobParallelForDefer.

15 Likes

Thx again @eizenhorn for superb explanations on pathfinding.

1 Like

Thank you very much eizenhorn! Very much appreciating the time and effort you put into these posts!

Looks very similar to pathfind used into Castle story, interesting.

Thanks for sharing that !

1 Like