Hi @unity_andrewc ,
let’s say that I’m reading a frame from Hololens locatable camera, with size of 896x504. On a captured frame there is a point or many points of interest, be it a person’s face, edge of an object or anything else I’m trying to detect, let’s say it will be a tip of a person’s nose, that have some x,y coordinates on my frame. What I want to do is to take those x,y coordinates, process them and display a hologram in a world space, but in a way that this hologram position corresponds with real object - a person’s tip of a nose - in a real world environment.
Since on hololens the Screen size and frame size are the same, I’ve been trying to do this:
sphere.transform.position = Camera.main.ScreenToWorldPoint(new Vector3(known_x_from_frame,known_y_from_frame,fixed_z));
When I pass a frame onto canvas the result is OK and the sphere displays on my object of interest. But on Hololens the result is not precise.
After that I tried methods described at already quoted Microsoft website:
Vector2 ImagePosZeroToOne = new Vector2((1.0F * known_x_from_frame / (1.0F * frameWidth), 1.0F - ((1.0F * known_y_from_frame / (1.0F * frameHeight)));
Vector2 ImagePosProjected = ((ImagePosZeroToOne * 2.0F) - new Vector2(1.0F, 1.0F)); // -1 to 1 space
Vector3 CameraSpacePos = UnProjectVector(projectionMatrix, new Vector3(ImagePosProjected.x, ImagePosProjected.y, 15));
Vector3 WorldSpaceRayPoint2 = cameraToWorldMatrix * CameraSpacePos; // ray point in world space
I’ve been experimenting with every calculated measure, but with no precise result. I know that the depth parameter is difficult to achieve, so for now I would be more than happy only with having x,y coordinates of hologram precise enough, with some fixed z.
I hope the provided description is better, but if you or anyone else needs more details I will be happy to provide. At the time of writing this message I can’t post any screen shots, but if they are still needed, I will post them tomorrow.
