http://www.google.com/atap/projecttango/
This is some amazing technology. Us programmers can easily create models from our phones now. Best part, it can directly port to Unity! !! I’m so excited!!
This is fantastic, bookmarked it. I am actually in the middle of learning Blender and for a programmer this is no easy task… this tech would be a god send!
Can’t wait to see the structure of real locations placed into games. Could be really interesting to allow players to create their own maps using this technology, too.
Looks very interesting, had been wondering what Johnny Lee had been up to since leaving MS having worked on the Kinect.
Looks interesting, just signed up.
So cool. The possibilities of this with things like VR and Augmented reality are just insane, especially that they can track your movement.
Also depth coordinates can be given out in c/c++ and java so that’s even better
it definitely has a lot of potential in many areas, especially games. i just wonder if its something that will catch on with the public. im imagining games where you walk around and the real world becomes a game world that you can find stuff and interact with etc.
Definitely a good chance, especially since Apple purchased the company who was responsible for the original Kinect technology last year, so I would expect to see similar tech to Google’s being integrated into iPhones/iPads soon. In fact I was slightly concerned that Apple might have simply sat on the technology and not brought it to the mobile space very quickly if at all, but with Google’s moves I think they have no choice but to follow suit. With MS building ontop of the original kinect technology i’m pretty sure they could join in too if they had too, making depth sensing (which is one of the key aspects of this tech) phones ubiquitous in the market place.
So was pretty excited by this, I’d been waiting for depth sensor capable mobile devices for a couple of years for various projects I wanted to implement. Unfortunately having looked at the terms it would seem this initial prototype is out of reach of independent/freelance developers due to the restrictions.
You need to have an incorporated company. Ok that’s not too hard to achieve, though setting one up (that is anyhting more than a shell for this opportunity) might not be possible prior to the deadline.
No P.O. Boxes or Home Offices - this is the killer, unless you have office space there is no way around this and office space is expensive. Maybe you could get away with some community office space or even a serviced office but that’s likely to still involve considerable more expense than not having them.
There are also restrictions on Countries, though those excluded are not listed, one would hope if yours is included int he dropdown list then its ok.
Obviously knowing someone with an existing and related company might mean you could make use of it, but i’m not sure what Google’s opinion would be.
Overall its a real shame, clearly with only 200 prototype units getting in on this was going to be difficult, but from my point of view this is now exclusively limited to large companies and completely removes the ability for anyone else to even try and get involved at this stage. Quite tragic as I feel there are plenty of independant developers for whom this technology would be highly interesting and useful and that those people may well be the ones who are perfectly placed to take advantage of it.
Have you checked out Metaio? It has slam detection so you can map a room using the webcam…vuforia’s working on the same thing as well.
I’m not familiar with ‘Slam’, but a quick google seems to imply its for topology and tracking which is sadly not what I need which is semi-accurate depth sensoring. I have used some tech that enabled basic topology mapping so you could detect horizontal verses vertically geometry in the world, but that is still not enough and in my experience was both process intensive (too slow) and not at all robust.
Metaio is pretty good, but its costs are (or were) prohibitive for the type of clients and projects I have in mind and it didn’t support in editor playback. Vuforia is pretty fantastic and for mobile AR has been my go to tech, but its lack of desktop/kiosk support restrictions its usability for 80% of my client work (not that I imagine Project Tango would really get around either).
As for actual depth sensing i’ve been using the Kinect for the last few years which is great for desktop/kiosks, but is pretty impractical for mobile applications, especially end -user experiences. Maybe i’ll fall back to using the Kinect to prototype my ideas further for mobile now I know Google is invested in the market and that Apple will likely (if they aren’t already) be forced to keep up using their purchase of PrimeSense.
Further to my previous comments I feel I should point out I fully understand Google being restricted from delivering this technology to certain Countries (though honestly a few Countries on the list are a bit odd - Italy?). What I don’t understand is why the FCC would have a restriction for distributing to Incorporated companies or why in this day and age one needs a physical office that is not a home.
I can’t find the video I’m thinking of, but yeah you scan the room with the phone and it maps it in 3d with colliders so you can shoot a ball from the phone at the wall/floor/countertops and it will bounce off accurately…
There’s pointcloud plugin for unity that does some pointcloud stuff. Also brekel pointcloud for kinect.
I use vuforia and kinect/asus xtion pro as well and don’t know why vuforia would go through the trouble to get it working on a desktop in the editor, but no ability to build for anything but android/ios.
Other slam examples I’ve seen let you drive a car in a room and it’ll collide with objects if you run into them.
From Vuforia’s site: http://www.qualcomm.com/research/projects/computer-vision/slam
Seems very limited since it requires special, probably expensive hardware in the device. So most people won’t likely have it, I think.
It does raise very interesting implications for the future of Google Glass though. If they could integrate this into that, you could do some very interesting things.
I’ll have to try and find that.
I’m kinda confused how depth sensing is possible with a single rgb camera, obviously being able to move the camera (though not simply rotation) should enable stereoscopic depth, but i’m still unsure how that can generate accurate depth per pixel of a scene. Relative depth I could understand, using known references for determining measurements I can understand, but beyond that i’m not sure.
Its sort of confusing because if that’s possible why hasn’t it been done before on mobile and why is Google and Apple falling back to dedicated technology to achieve the same thing?
Obviously one advantage of dedicated hardware is improve accuracy, robust and performance and maybe those things are enough, certainly its what i’ve been looking for, just so frustrating to be excluded at this point. Still debating whether to try teaming up with some friends who own companies and fullfil all the requirements.
Yep it is bizarre but after several years of almost everyone asking for it on the vuforia forum wishlist I think its clear its never going to happen. I believe the reason is solely due to licensing restrictions on some of the third party tech they use.
Thanks i’ll check this out, been stuck in Kinect projects for the last 6 months so not stayed to up to date on vuforia.
From the T&C of the competition they talk about the value being equivalent to $500 which I don’t believe is much in excess , if at all over other high priced mobile phones?
The actual technology is pretty cheap really, though it depends on exactly which tech they are using, a simple IR projector and RGB and IR camera should’t amount to much, maybe more expensive if using dedicated chips for the scene reconstruction, but I somewhat doubt that, what would be the point in not using the onboard cpu/gpu.
Most of the solutions are based on OpenCV…which has all kinds of crazy algorithms for finding lines/patterns from any image… a good example is a Unity plugin for “Extreme motion” that uses opencv to body track with a regular RGB webcam… http://www.xtr3d.com/ free to try and detects skeleton and gestures pretty well. It’s pretty mind blowing how it can find your limbs and such with a crappy webcam…AND IT WORKS ON MOBILES!
Example in Unity:
https://www.youtube.com/watch?v=_2t1wM-Bnlg
At my job we do stuff for video walls, digital signage, etc so I play around with everything I can find. Been on an oculus rift binge for a while, got my wii controllers working with it and haven’t been into body tracking as much lately.
edit:: Noisecrime, you ever use zigfu? I used the depth sensor as a mask to live key out the background from the player, but it has a crappy edge around the player, I need to find a shader that blends the edges better.
This was my prototype:
Yeah I was meant to get the Extreme Reality sdk back in the summer last year, but client projects and time conspired against me, which always seems to be the case these days, too much to do and never enough time.
It does seem very impressive, though I suspect its not quite ‘magic’ 
Without having done any research on it, I would imagine it uses some form of machine learning, just as skeleton tracking does on the Kinect, in that you record a thousand or so clips of people, segment them into various limbs, then use some form of fuzzy matching to determine the poses (naturally that is a gross simplification of the process).
I think on the kinect the depth sensor provided a very useful additional data point from which to segment the human body, but once trained I suspect depth is not as important and other methods of extracting users (velocity, segmenting etc) are almost as good, thus allowing the same technique to work in 2D. At least thats how I might approach the problem.
I found a enlightening white paper on the Kinect system back when it was released that describe their process - see here.
Cool, I do something similar but as a freelancer, mainly AR and Kinect stuff for the last few years. One of the projects I worked on last year that is a small part of a bigger interactive is a finalist (for the whole interactive, not my piece) at SXSW this year which is nice and just finished two Kinect based ‘sport’ games for clients in the U.S. Its simple stuff really and I wish I could move towards producing more artistic/abstract interactive pieces based on using the Kinect, but you have to go where the money is
Would love to get into Occulus, but at this point I feel like I might as well wait for the hardware update to better screen resolution. I just know if I bought a developer system now, a few weeks later they’d announce the improved version
My problem with the Rift though is that its not a great fit for client projects, its a 1 to 1 system, meaning people have to wait to take turns with it, where as normally you want interactive that many people can interactive with simulatniously. Its awkward to use, often requiring someone on hand to help people put it on and is somewhat unhygienic at least for trade shows/museums where you have many people using it day in day out. As a personnel technology though it looks amazing.
Not really, I stopped using their stuff at the point they made it commercial, but we were all working towards getting the kinect and openNI stuff working in Unity at one point a few years back in threads on this forum. They are cool guys though and I guess if i was starting from scratch now Zigfu would be a great starting point.
Ha, yeah I tried the same thing, wanting to have virtual objects orbit around the user, but as you say the label/depth map isn’t a close enough fit really and the depth/image cameras on the Kinect are lacking the resolution for it. I also felt that the projection of the depth camera onto the image camera, as well as slight differences in syncing between frames make the issue worse too.
Just wondering in that video you posted did you use the openNI feature to match/re-project the image camera to the depth camera (or it might be the other way around). I can’t remember the function name off-hand, but basically as the depth and RGB cameras are not physically in the same position on a kinect they result in slightly different images (possibly different lenses too), so you have to use a function to map from one camera space into the other.
Unfortunately i’m not sure it can be improved, you can’t really blur the depth map as that results in nonsensical data and there is little that can be done about sync-ing issues. Its a real shame as there is so much opportunity for interesting uses if it did work, especially if you are able to tie kinect depth to scene depth within a shader as you can literally put people into different environments.
Actually thinking about it in general this is the problem with the kinect. It works, but its not quite as robust as it needs to be and no matter how much effort you put into it, someone will always break it. For example i’m amazed at just how many different ways people will ‘throw a ball’, making gesture recognition for such an action really difficult, even worse if you factor in the difference between how kids vs adults tend to throw. It may only fail on 5% but when it does fail it feels so frustrating.
I just found it listing on Google play store for purchase but there is no buy button yet. Any one here got it? Is there a unity solution available for this?
For those interested here is the link to the Google Play Store listing - https://play.google.com/store/devices/details/Project_Tango_Tablet_Development_Kit_White?id=project_tango_tablet_white&hl=en