HoloLens shared tap-to-place issues

Unity Engine
1

Hello, the problem I’m having is a little complicated as I tried different ways of solving it ending up with different issues, so I’ll try to break this down.
Using Unity 2017.4.1f1.

  1. Mixed Reality Toolkit (Example)
    I implemented this specific package containing the standard toolkit and a few examples, then proceeded to open the SharingTest scene. It had a missing script which I replaced with the TapToPlace script (I supposed that to be the missing script, since there was an Interpolator script as well), I then built it and connected two HoloLens. The outcome varied every time I rebuilt/restarted the application, giving me either:
  • No connection to the Sharing Service app
  • Only one HoloLens connecting
  • Both connecting with desynced anchor
  • Both connected but anchor kept going back to the original position no matter who tried to move it

  1. Mixed Reality Toolkit (NO example)
    After the failed experiment with the example scene, I tried making my own example adding the scripts I deemed necessary and ended up having a scene where both HoloLens could see what was going on but the scene was not synced correctly and all of the sudden the objects I used for testing purposes, with a TapToPlace script, kept going back to their last position letting no one move them. I only tried this twice.

  2. MR Sharing 240 tutorial
    At last, I tried following this specific tutorial and managed to set up the scene correctly, but I had two recurring problems:

  • Out of 10 tests, the app connected only once to the Sharing Service
  • Out of 10 tests, I managed to be able to place the Hologram only once (not sure if it was the same time the connection worked)

I am now at a complete defeat. I’ve been trying for days with no success and it seems no one is interested or just isn’t having these issues. What would you suggest?
My final purpose is to create an app with a Main User and a Spectator User that allows me to:

  • Open a scene with an object instantly placed by the main user that the spectator/s can see
  • Open a scene where the main user spawns and moves objects which the spectator/s can see in real-time or with some slight delay

Scenes should always be synced either manually or automatically. So far, I could accomplish neither of those.
Thank you for your time.

2

We recently did some prototyping on methods for synchronizing mixed reality experiences without streaming anchor data between devices. We tried several different strategies, but by far the simplest to implement was this:

  1. Establish several physical targets to act as registration points. These targets can be pre-existing physical objects (such as a doorknob, coffee cup, etc.) or manufactured targets (e.g., paper bullseyes or other fiducial markers).

  2. On each device, use Gaze and spatial collision meshes to capture the location of each target, and create a world anchor at each location. Use the same order for each device.

  3. Create a Game Object (Shared Root) which will act as a shared reference frame at the centroid of the targets; use one of the targets (e.g, the first target) to establish orientation. Since the anchor objects can move slightly, update the transform on Shared Root each update.

  4. Any content which is a child of Shared Root will then be perceived at the same physical space by all users, so use traditional network synchronization for manipulating shared content.

For something more advanced, you could use fiducial markers such as AprilTags which could be auto-recognized by processing video capture, obviating the need for manual target registration.