AR Scale in ARCore Preview 2 w/o the experimental AR interface [Tutorial]


Blender File by : fokuspace
Introduction
Based on his Unite Austin talk, Rick Johnson wrote a blog post describing the concept of scale in AR.

Link: https://blogs.unity3d.com/2017/11/16/dealing-with-scale-in-ar/

However, you might ask, how do I solve the scaling issue without using the AR interface. Fear not, I tinkered around with the new SDK and learned how to achieve a similar result.

Tutorial

Importing the SDK
We’re going to start with a new project and a fresh import of the ARCore preview 2 SDK. To learn how to setup the ARCore sdk in your project follow the official documentation here
link: https://developers.google.com/ar/develop/unity/getting-started
GitHub Releases: https://github.com/google-ar/arcore-android-sdk/releases

**Important*: Preview 2 only runs on devices that have the ARCore Preview 2 apk installed, and requires Unity 2017.3 or higher

Handling scaling
By default the “AR Core Device” prefab is setup to support scaling. The component responsible for this is the “Tracked Pose Driver.cs” The component is attached to the child, “First Person Camera”, notice how “Use Relative Transform” is checked.

While the description in the documentation might not be obvious, the property handles the scale transformations to the camera, relative to the parent.

The documentation for the tracked pose driver: https://docs.unity3d.com/ScriptReference/SpatialTracking.TrackedPoseDriver.html

Scaling the AR Components
Because the scale is handled relatively, the structure of the hierarchy is important. To efficiently scale the AR camera, you must scale all the AR components in the Hello AR Scene. Scaling all the components insures that all the information captured by the AR SDK is being presented accurately, around the same origin.

Example of hierarchy that is ready to scale.

*Scaling only the camera component will result in the point cloud appearing in a different scale than the camera rig.

**Not all of these components are required*, but I like to group my SDK components under a single parent.

Apply the Scale
Now that our hierarchy is setup correctly*, make sure that all the child objects are located at the origin* (not including the light or example controller) 0,0,0. Relative to the parent object. Then you can scale the parent object. I scaled mine to be 20x the regular scale.

*Note scaling the parent object will not scale the “Session.Raycast” used in the Example Controller

Positioning the Camera Rig Relative the environment.
Rick Johnson also described positioning the camera rig so that it appears that the terrain is placed on top of an anchor point, when in fact the camera rig is being offset. To achieve this we have to do two things,

  1. Scale the Sessions.Raycast by the parent transform scale so that the hit point is accurate to our camera scale.

  2. Offset the parent transform by the inverse of the rayhit position.

I’ve modified the Example Controller code, I won’t go into much detail as I’ve tried to name the variables properly so that the code is easy to follow. Feel free to leave comments.

   // Raycast against the location the player touched to search for planes.
        TrackableHit hit;
        TrackableHitFlags raycastFilter = TrackableHitFlags.PlaneWithinBounds | TrackableHitFlags.PlaneWithinPolygon;

        if (Session.Raycast(touch.position.x, touch.position.y, raycastFilter, out hit))
        {
            //Store the parent transform
            Transform rootTransform = m_ARCamera.transform.parent;
            //Store the hitPosition for scaling.
            Vector3 hitPosition = hit.Pose.position;
            if (rootTransform != null)
            {
                //Scale the hitPosition by the scale of the root transform
                hitPosition.Scale(rootTransform.transform.localScale);
                //Position the root transform by negative the hitPosition
                rootTransform.localPosition = hitPosition * -1;
            }
              //I added a conditional property so that I can load a scene
            if (objectPlace == false)
            {
                //I load a scene , for this example the origin of the scene must be at 0,0,0
                SceneManager.LoadSceneAsync(scene, LoadSceneMode.Additive);
                objectPlace = true;
            }

        }
4 Likes

This is great work, @kbabilinski , but I'd like to correct a few misconceptions in your post.

The dealing with scale blog was based on @tdmowrer and my talk. If you look in that link, our talk also dealt with cross-platform AR across ARKit and ARCore, and provided some experimental code: https://github.com/Unity-Technologies/experimental-ARInterface

That code has recently been updated to work with ARCore Preview2 , and the scaling works just fine.

This is embarrassing :hushed::sweat_smile:I knew that the AR Interface existed and I thought I cloned it onto my computer. When I was trying to incorporate the ar interface in my project I swore I was looking at the correct thing. But, I just looked at my copy and I found out that I was using the Mapbox project.

I’ll update the post:smile:

[quote=“jimmya”, post:2, topic: 687954]
This is great work, @kbabilinski , but I’d like to correct a few misconceptions in your post.

The dealing with scale blog was based on @tdmowrer and my talk. If you look in that link, our talk also dealt with cross-platform AR across ARKit and ARCore, and provided some experimental code: https://github.com/Unity-Technologies/experimental-ARInterface

That code has recently been updated to work with ARCore Preview2 , and the scaling works just fine.
[/quote]

Updated the title to be more clear, incase anyone wants to scale the scene without using the interface.

[quote=“jimmya”, post:2, topic: 687954]
This is great work, @kbabilinski , but I’d like to correct a few misconceptions in your post.

The dealing with scale blog was based on @tdmowrer and my talk. If you look in that link, our talk also dealt with cross-platform AR across ARKit and ARCore, and provided some experimental code: https://github.com/Unity-Technologies/experimental-ARInterface

That code has recently been updated to work with ARCore Preview2 , and the scaling works just fine.
[/quote]

Thanks for this, @kbabilinski ! Exactly what I am looking for while I try to figure out how to use the ARInterface. I am not sure I am understanding the changes to the code entirely, here is how I changed the Session.Raycast code that places my object.

if (Session.Raycast(touch.position.x, touch.position.y, raycastFilter, out hit))
{
    //Store the parent transform
    Transform rootTransform = FirstPersonCamera.transform.parent;
    // Store the hitPosition for scaling.
    Vector3 hitPosition = hit.Pose.position;
    if (rootTransform != null)
    {
    //Scale the hitPosition by the scale of the root transform
    hitPosition.Scale(rootTransform.transform.localScale);
    //Position the root transform by negative the hitPosition
    rootTransform.localPosition = hitPosition * -1;
    }

    var placedObject = Instantiate(objectToPlace, hitPosition, hit.Pose.rotation);
                    // Create an anchor to allow ARCore to track the hitpoint as understanding of the physical world evolves.
    var anchor = hit.Trackable.CreateAnchor(hit.Pose);

    // Make object a child of the anchor.
    placedObject.transform.parent = anchor.transform;
}

When I build & run, the object gets placed but not exactly where I touched. Additionally, the script on my placed object that handles touches on the object using Physics.Raycast isn't working properly anymore...when I touch the object nothing is moving as it is supposed to. I suppose maybe those Physics.Raycast/touches have to be scaled as well? I'm a bit confused here.

@nerkderk I'm glad my post helped :) I have transitioned to the Unity AR Interface after I learned that I was using the wrong repo, so my understanding of the SDK is still limited. But I can try to help.

I'm not sure what's the exact issue you're experiencing but I can share some thoughts. One idea is that you're placing your object in the incorrect position. We use the hitPosition as an offset for the parent transform. That's why we do

rootTransform.localPosition = hitposition *-1;

Here is a bad diagram of what's happening :


*We're moving the camera back so it appears that the object was scaled.

So I would suggest changing

var placedObject = Instantiate(objectToPlace, hitPosition, hit.Pose.rotation);

to the default hit position .

[code=CSharp]var placedObject = Instantiate(hit.Pose.position, hit.Pose.rotation);[/code]

Let me know if that works. Stay sharp ;) [Bad code joke]

@kbabilinski Ah yea, that seems to have helped greatly! Thank you!

I guess I should probably switch over to using the ARInterface, but I am having a hard time getting started with it. For example, I haven't even been able to find the proper code for instantiating an object at a touch point, to see how it differs from ARCore. It seems like they don't have an example of such a project...or documentation on how to use it. Although I am most likely just missing something. Do you have any tips to get me started / point me in the right direction of how to get acquainted with the ARInterface?

@jimmya

Does the experimental library's scale option works correctly after it was updated to ARcore 1?

I opened an issue about scale on github but it hasn't received a response so I don't know if its broken or if I'm doing something wrong..

Hi All,

When i use the same kind of hierarchy which @kbabilinski have suggested for scaling, i had a problem, that my object is anchored below the Plane Generator/DetectedPlane which in turn makes my object to move away from the anchor position and it follows the camera when i try to move the camera.
I am new to unity can anyone please help me how to place a object above the plane.

Thanks in advance.