I was reading up on Acer’s new mixed reality headset and kept running into articles like the following.
While for certain usecases it will be a step up (basically when all input only makes sense if your hands are visible), there are numerous usecases where this will be worse than current outside-in tracking. How many times are you playing a shooter and you’re aiming but your hands aren’t visible to the head. Sports games like tennis, bowling, ping pong, etc. need hand tracking when they are not visible to the head.
Don’t get me wrong I’m not hating on the technology, I’ve played with leap motion on a HMD, I certainly believe certain usecases are great, like Minecraft AR, or certain job training AR. But this is the same good for some, bad for others as the current Desktop VR products have. Some games can tolerate small play spaces, some can’t. In some teleporting is ok and others it’s not ok. I see the same issue with inside out tracking and I’m grateful that there is little overlap so that lets say outside in fit 20% of games well and inside out also fits 20% of games well that we’ll get 40ish percent of games covered. I just don’t understand why a lot of people seem to think it fixes everything. Even when they are inevitably combined, It’s not like there won’t be some games that still want a seated VR experience with a controller because VR/AR can’t accommodate the game type (probably due to needing a super large open area like a football field).
The benefit to inside-out, as far as I understand, is that the tracking frame of reference moves with the player instead of being fixed to a physical location. The issue with hand visibility is an implementation detail that I can only assume will be improved over time. Fundamentally, inside-out tracking is more flexible than outside-in tracking to the point where an advanced enough inside-out system should be able to also do anything an outside-in system could do. With that in mind I can understand people being excited about the concept even if current implementations might be flawed.
Years down the road perhaps, but I’m not sold on either methods further than lets say 3 years (more likely they’ll be combined). There is any number of other possible approaches that could pop up in the meantime that might be better if we project that far out.
Within that timeframe though, every inside-out implementation is going to be using stereo depth perception of some sort. The first implementations will be on the headset itself. Within the implementation it will only be able to track hands, feet, whatever else you want, when it can see it. The previous usecases have critical swaths of time where those items that desire to be tracked aren’t seen hence it cannot do what outside in can. Even if I extrapolate out maybe multiple of these say another one on the chest it would expand it’s field of tracking but like in tennis you swing behind you so that wouldn’t work. Plus if we go as crazy to a whole suit it should be able to do the tracking without those depth sensors minus the one on your head which is simulating your vision.
I’m not knocking the technology, I’m glad that it will expand the number of possibilites and it fits a new percentage of games, I just don’t get why many places are touting it as a holy grail.
I totally get it for a good number of AR usecases. Not being confined to a cleared “play space” opens up a lot, plus in AR you still see the real reality so things like couches etc. are fine and are even incorporated into the app. In VR I’m a bit less sold on this aspect as you still don’t want to run into things so wherever your playing needs to be flat and cleared and since you lose track of reality having that virtual boundary appear when you wander too far is helpful. Unless your playing in something crazy like an empty football/soccer field.
@greggtwep16 I skimmed through the article, and as far as I can tell, the idea is to get rid of setting up some room beacons and other stuff like that.
Meaning that with inside-out tracking a user would be able to use headset anywhere without prior configuration.To put it bluntly, people could walk outside with mixed reality headset still worn.
I worked with a system that provides tracking information based on image-based recognition (zed camera) results… are not bad, actually.
Speaking of hand tracking human’s combined field of view is huge. Something aroudn 200 degrees horizontal and 100 degrees vertical, meaning you usually see your hands via peripheral vision. When people do something with their hands, they often look at them, so in most situations headset-based hand tracking would do a good job. I’d still prefer a data glove, though.
Isn’t the same true of outside-in systems? Either you use some non-visual system which requires a glove or held peripheral, or you’re using a visual system where the body can occlude one or both hands at times.
Yeah the tech definitely has it’s pros as well, fixing the 10 minute setup will be nice. As far as a humans field of view though reality essentially does the positional tracking :). One wide angle camera especially as it got towards the edges of the view wouldn’t be enough to positionally track. On anything I’ve seen so far it has to be seen by both cameras to positionally track which is a pretty narrow FOV.
When the cameras are at the corners there are very few times where it’s being occluded from both. Still possible, but adding more cameras will probably mitigate that further.
Here’s how I understand what they’re doing, fundamentally.
Outside-in:
Fixed sensors observe your (HMD) movement to ascertain where you are in space and your orientation/pose.
Translate it into useful data.
You apply it in your application.
Inside-out:
The HMD observes the environment to ascertain where you are in space and your orientation/pose. Potentially include environmental data.
Translate it into useful data.
You apply it in your application.
The idea for the hardware is either the HMD reads things around it or some fixed sensors try to track the HMD. I think they could potentially be pretty much the same except that the inside-out concept allows greater freedom and has greater scalability due to the fact that you are not bound to fixed sensors in an area. While those sensors could always observe and do exactly what the ‘inside-out’ method is doing, they would be doing it in a fixed space rather than on the HMD.
This is basically where things need to go to take things further for AR/VR. A major issue is self awareness in VR and being able to track hands. The inside-out concepts potentially solve those by making the HMD aware of everything around the wearer, which is what we care about. People were gluing Leap Motion trackers to their HMDs and using cameras to project real world data into VR years ago anyway, so we’re basically just moving forward on those concepts. Nothing particularly novel happening here except that it is actually getting done now. Simple iterative progress taking place.
AR HMD’s have a really long way to go before they’re practical. For the time being we’ll probably see VR make the biggest leaps like taking inside-out tracking to the next level and improving the technology that AR/VR share such as ‘environment understanding’ until AR displays start to become something more than exaggerated marketing.