Collider query help

Scenario is Collider bounds all fit in a grid. Bounds extents are all multiples of 0.25. Most colliders are box colliders a few are mesh colliders. Rotation is always at 90 degree angles.

I create an undirected graph of 'connected' colliders. Simplest way to think of connected is assume all colliders are boxes and ones that have faces that share the same space are connected.

Currently for every bounds I take a box collider of the same size/rotation and inflate it by 0.125 and do a CollisionWorld.ColliderCast.

The current approach has one functional issue which is it hits colliders that are only connected by a corner not face. I haven't implemented a fix for this part yet. Current idea is that since everything is at 90 degrees for each collider that I hit I could then take it's aabb and move it both directions on each axis and see if it intersects the source non inflated aabb. That's 6 bounds checks. Average number of connected colliders is probably in the 3-6 range.

The scale of this is into the low hundreds of thousands, although it's partitioned so never that many colliders in the scene. Building connections is incremental and persisted. But even with that there is a worst case where we have no connection data so it's worth optimizing more if we can.

FYI the scenario just in case there is some better high level approach I'm missing is an integrity system for a runtime building system. The connectivity graph contains data like is the collider grounded. I use BFS to do a pass to spread various strength values like an influence map. And when an item is removed a BFS to detect islands and ensure they are properly grounded.