HoloLens 2 Peer to Peer Network

Dear All,

I am currently working on setting up a peer-to-peer connection on a HoloLens 2 device and exploring the use of Purrnet for this purpose. However, I am encountering challenges with certain MRTK interactions not functioning as expected. I am not opposed to using another networking software, my main goal is to connect up to 5 HoloLens 2 headsets together using p2p networking, with spatial anchoring where all 5 headsets are in the same room/environment. I know this can and has been done before, however finding the information for MRTK3 and HoloLens 2 has been challenging to say the least.

Additionally, I need the 3D model to appear in a consistent position within the physical space, ensuring all users see it in the same location (local alignment). I recall that Microsoft previously provided tutorials and methods for establishing headset-to-headset connections utilizing spatial anchors.

Does anyone have experience or guidance on achieving this setup using MRTK 3 in Unity (version 2022.3)? Any help or resources would be greatly appreciated!

Best regards,
Sean

Hmm that was new to me and after researching, it actually is new. It had its first commit on GitHub 6 months ago and has been on the Asset Store for 3 days.

Honestly, my opinion is you do not want to consider using this networking framework! (not at this point in time)

Considering how much goes into making a good and stable and well optimized networking framework tells me this new solution isn’t going to be production ready (they say so themselves).

I question why this exists because on first sight I can’t spot anything that other frameworks aren’t already doing (their “unique features” list contains primarily superficial technical items). Sounds like a case of not invented here respectively “not my code”.

The main issue is going to be support. For most other frameworks you’ll find tutorials and instructions and community support. This one is far too new and likely not going to be popular since the market is oversaturated (NGO, NfE, Fish-Net, all of Photon, even good old Mirror, plus plenty plenty of other lesser known general purpose or lightweight or highly specialized solutions).

Because of this, any new networking framework has a high risk of being abandoned rather quickly. Over recent months aloine I’ve observed this happening to at least two new network frameworks. One frequenly got into a fight with other network devs, it’s been “burned” (may still continue to exist, didn’t check but didn’t hear anything of it either). Another one never saw a public release despite glossy marketing and several announcements, and probably will never see the light under its current name (possible trademark infringement).

Sorry for not answering the core question. :wink:
But by all means stick to proven and established tech particularly if you develop for hardware that only a tiny minority is developing for.

At the University I work at we developed patented technology to achieve this with minimal precision issues.

I don’t want to go into detail on how we did it because the approach is too complicated and honestly not necessarily.

If I was to do this again I would just use AR foundation and use the anchor system to align the world that way.

Also Peer to peer won’t work. I don’t know why you would want that.

Why not?

I suppose this could be a misunderstanding due to the conflicting meaning of “P2P”. On one side, it describes a network topography / architecture which, for games, plays no practical role anymore as client-server topologies rule for good reasons. In P2P every participant has authority over everything and whoever isn’t playing nice (or has a bug), would cause everyone else to desync and likely end the session. That is the original and “correct” meaning of P2P.

On the other hand P2P these days has become a synonym for any 1 vs 1 networked communication. Even if there’s a Relay server in between the two client-server (client-hosted) peers establishing the communication because rarely can two clients communicate directly with each other.

I appreciate the comment about Purrnet, I am not stuck with using it as it was just one of the solutions I found when researching. What I really need and will clarify in my post is just a p2p or as you said 1 vs 1 network solution for the HoloLens 2. I am wanting to avoid connecting to an external server for a few reasons.

My main goal is to connect up to 5 HoloLens2 headsets together using p2p/1v1 networking, with spatial anchoring where all 5 headsets are in the same room/environment. I know this can and has been done before, however finding the information for MRTK3 and HoloLens 2 has been challenging to say the least.

A direct connection to another device:

  • works fine (usually) if both/all devices are on the same network / subnet
  • requires port forwarding (and possibly firewall configuration changes) on the hosting client if other clients are to connect from another network (eg the internet)
  • NAT punchthrough / UDP hole punching is a technique that allows clients to connect directly to each other without host-side configuration steps, but it may fail for a number of reasons AND it requires a remote server accessible to all clients to negotiate the initial handshake process
  • the most reliable solution for establishing a connection ibetween clients is by using a Relay server/service - this is effectively a dumb dedicated server that just passes on traffic, but all clients can connect to it

So if you can guarantee the environment the app is used in and you can make certain that both devices are on the same wifi, a direct connection between them should work without issues.

For local network connectivity, you can use NGO v2.1.1+, distributed authority with Unity 6, and you do still need to install the multiplayer service SDK.

Set your network topology to distributed authority in the NetworkManager, but instead of connecting to the service you can start one of the clients as a host. This will let you have a local area peer to peer network topology.

If you decide you want to support remote connectivity later, then you will need to use the service to accomplish that.

2 Likes