Scaling with ARFoundation

[tdmowrer]

We wanted to make scale a first class citizen in ARFoundation, so I’d like to clarify how it works. A basic AR scene has an ARSession and an ARSessionOrigin. The job of the ARSessionOrigin is to transform “session space” (the coordinate system used by the device) into “world space” (a position, rotation, and scale in Unity). The AR camera should be a child of the ARSessionOrigin, so if you want your AR camera to start at a non trivial position in the Unity scene, simply move the session origin to that position. That seems straightforward enough.

However, you might find scale harder to get your head around. What are we scaling anyway? What does that mean? If I have a single Unity cube in my scene, and I set the scale to something other than (1, 1, 1), what should I expect to happen?

WTF? Why is my cube moving all over the place? Why isn’t it scaling?

The cube is changing apparent size, but also changing apparent position. Imagine if I had two cubes, what would you expect to happen? If both cubes got bigger without moving, then they would overlap. Clearly, they also have to appear to move.

I say “appear” because we aren’t actually scaling the cube; we’re scaling session origin. This means scaling the camera’s position and also all the trackables, such as planes. The blue plane in the GIF above doesn’t appear to move because it and the camera are being scaled together. The end result is that it looks like the entire scene has scaled, when in fact, the session origin (camera + planes) have been scaled by the opposite amount.

Looking instead at the scene view, we see what’s actually happening: the cube doesn’t move at all.

We’re trying to scale the apparent size of the entire scene, so everything will appear to get bigger or smaller and move away from or toward some origin. In fact, the session origin.

So what should you do? Put the session origin at the “middle” of whatever scene you’re trying to scale, and add another GameObject in the hierarchy (between ARSessionOrigin and Camera) to add an additional positional offset to the camera. Wouldn’t it be nice if there were a convenience method to achieve this? In fact, there is!

ARSessionOrigin.MakeContentAppearAt does exactly this. It doesn’t move the content; instead it moves the ARSessionOrigin such that the content you specify appears to be the the position you’ve specified. Example: instead of moving the entire scene 3 meters to the left, move the ARSessionOrigin 3 meters to the right.

Effectively, this puts the ARSessionOrigin at the “center” of the scene, and then also moves the camera by the opposite amount. The effect is that the origin for which all scaling is done is now well defined and controllable.

Here, I’ve created a GameObject in the center of this complex scene, and asked the ARSessionOrigin to MakeContentAppearAt some point I selected on the plane. On the right, we have the GameView, what you would see on your device. It looks like the scene is moving while the plane remains still. However, the scene view on the left reveals that we’re actually moving the session origin around.

Using this method, scaling the session origin finally does what we want:

The content is “scaled” around a point I’ve specified.

And it works for rotation too:

Note how the camera effectively orbits the scene, making it look like the content is rotating.

So why go through all this headache in the first place? Why not just scale the scene and leave the session origin alone? For simple content, like a cube, it’s probably much easier just to scale the content. But for complex scenes, like this village, it might be very expensive, or even impossible. Particle effects cannot be trivially scaled. Scaling physics or nav meshes down to be very tiny is a bad idea – things will start falling through the world, and nav meshes will be too small to work properly. Terrain is static at runtime, and cannot be moved, rotated, or scaled. For these reasons, it’s much more effective to scale the camera + trackables, not the scene content.

Hope that helps clear up some confusion. I’ll look for your questions and feedback below!

-Tim

[Tarrag]

That’s rad @tdmowrer , thanks a bunch , it clarified the concept.

I didn’t get a chance to look further into this but it appears the light/shadows rotate with the ARSessionOriginGO. I have a directional light in the scene.

MakeContentAppearAt you described worked for me for horizontal planes if I instantiated on a horizontal plane but in this scenario I run into problems when moving the object to/on vertical planes.

I used MakeContentAppearAt method when I instantiating or moving the object. At instantiation I used

if (m_SessionOrigin.Raycast(touch.position, s_Hits, TrackableType.PlaneWithinPolygon))
hitPose = s_Hits[0].pose;
spawnedObject = Instantiate(m_PlacedPrefab, hitPose.position, hitPose.rotation);
MakeContentAppearAt(spawnedObject.transform, spawnedObject.transform.position, spawnedObject.transform.rotation);

When moving an object to a vertical plane after instantiated at a horizontal plane if I used MakeContentAppearAt(spawnedObject.transform, hitPose.position, spawnedObject.transform.rotation); the object sits vertically on a vertical plane. So instead I tried using MakeContentAppearAt(spawnedObject.transform, hitPose.position, hitPose.rotation); However hitPose.rotation changes within a plane so the object jitters when moved and in a vertical plane it does not sit or rotate correctly.

To avoid the jerky movement I am experimienting by changing the actual rotation of the object to hitPose.rotation when changing plane alignments, use MakeContentAppearAt(spawnedObject.transform, hitPose.position, spawnedObject.transform.rotation); and in the case of vertical planes change the rotation of the ARSessionGO to Quaternion.AngleAxis(rotationSlider.value, Vector3.right); though haven’t seen good results.

Cheers! Sergio

[rxmarccall_1]

@tdmowrer
I was just trying to understand this better today! Good timing.

I wanted to ask if it’d be possible for you to upload / share the project above so we can easily play around with the example project you used above?

I’ve been trying to understand how I can manually make a trackable plane in the editor like this to test with.

[tdmowrer]

@Tarrag

I didn’t get a chance to look further into this but it appears the light/shadows rotate with the ARSessionOriginGO. I have a directional light in the scene.

The lights/shadows don’t rotate with the session origin, but they might appear that way, since the camera is orbiting the scene. Depending on the effect you want to achieve, you might want to parent the light to the session origin (so it moves and rotates too).

MakeContentAppearAt you described worked for me for horizontal planes if I instantiated on a horizontal plane but in this scenario I run into problems when moving the object to/on vertical planes.

I used MakeContentAppearAt method when I instantiating or moving the object. At instantiation I used

if (m_SessionOrigin.Raycast(touch.position, s_Hits, TrackableType.PlaneWithinPolygon))
hitPose = s_Hits[0].pose;
spawnedObject = Instantiate(m_PlacedPrefab, hitPose.position, hitPose.rotation);
MakeContentAppearAt(spawnedObject.transform, spawnedObject.transform.position, spawnedObject.transform.rotation);

You are both spawning an object at the hit point and then also using MakeContentAppearAt to make it appear at the hit point. I suppose that’s fine, but it doesn’t really matter where you spawn the object in that case, since you’re also moving the session origin.

Sounds like you don’t want the vertical orientation. In that case, you can use the variant that doesn’t take a rotation. Maybe something like this:

if (m_SessionOrigin.Raycast(touch.position, s_Hits, TrackableType.PlaneWithinPolygon))
{
  hitPose = s_Hits[0].pose;
  spawnedObject = Instantiate(m_PlacedPrefab);
  m_SessionOrigin.MakeContentAppearAt(spawnedObject.transform, hitPose.position);
}

If you want to just adjust the rotation later, use the variant that only takes a rotation.

[tdmowrer]

@rxmarccall_1

@tdmowrer I wanted to ask if it’d be possible for you to upload / share the project above so we can easily play around with the example project you used above?

I had a feeling that would be one of the first questions

I used a prototype of the simulation package I’ve mentioned in some other threads. That’s not done yet, but it should also work as a standalone sample; I’ll look at adding it to the ARFoundation Samples repo.

[Balours]

Thanks for the explanation @tdmowrer , it’s working fine!
Now my issue, I want to attach an object to the camera that will interact with the scaled objects, for example a stick that will illuminate the objects when touched, but of course if the camera is moved away I lost the interaction. How should I manage the scale of the attached object?
Thank you!

[tdmowrer]

@Balours

Thanks for the explanation @tdmowrer , it’s working fine!
Now my issue, I want to attach an object to the camera that will interact with the scaled objects, for example a stick that will illuminate the objects when touched, but of course if the camera is moved away I lost the interaction. How should I manage the scale of the attached object?
Thank you!

If I understand correctly, you want the “stick” to appear to remain unscaled, that is, not change size with the content. In that case, you can parent it to the ARSessionOrigin. If you also want it to move with the camera (e.g., a fixed stick dangling in front of your head), then you could parent it to the AR Camera.

Here’s what that looks like:

And here’s my scene hierarchy:

Remember, in this setup, the blue cube is not moving or scaling; it’s the plane, camera, and “stick” that get scaled together.

[Balours]

Yes that’s close to what I want, thanks a lot for the example. However, I would need the stick to stay proportional to the content, is it the case in your example?

[Tarrag]

Apologies for the late response @tdmowrer , that time of the year
That works, thank you! Cheers, Sergio

[tdmowrer]

@Balours

Yes that’s close to what I want, thanks a lot for the example. However, I would need the stick to stay proportional to the content, is it the case in your example?

To stay proportional to the content, the “stick” should be outside of the ARSessionOrigin hierarchy. You could then have a script that places it in front of the camera every frame.

[Balours]

That’s perfect, and it makes sense. Thank you for your help!

[HeavyProduction]

Any update on the scale project sample?

[maurogarcia]

It would be very useful an update of the Sample Project, 'cause still not contain anything about “Scaling”.
Thanks!

[tonOnWu]

Hi. We’re moving an old version of one of our AR apps, but we really need more examples. Particularly we need a “Scaling” example and how to remove planes and plane’s detection once the scene has been set.

[tonOnWu]

Hi everybody. Well, while we wait for an example that explains how to do this, I’ve been trying to solve it by myself. I’ll present what I’ve done so far. Any helping ideas will be well received.

First I created a Component call Scaler which was assigned to AR Session Origin. This is the code:

[RequireComponent(typeof(ARSessionOrigin))]
public class Scaler : MonoBehaviour
{

    ARSessionOrigin m_SessionOrigin;
    [SerializeField] GameObject referenceToScale;

    void Start()
    {
        m_SessionOrigin = GetComponent<ARSessionOrigin>();
    }

    // Method called by a Slider
    public void OnValueChange(float value)
    {
        Debug.Log("New Value: " + value);
        Transform t = gameObject.transform;

        m_SessionOrigin.MakeContentAppearAt(referenceToScale.transform, Quaternion.identity);

        referenceToScale.transform.localScale = new Vector3(value, value, value);
    }
}

The method OnValueChange is called by a Slider.

Now, following what I understand on the first post of this thread, I did set a MiddleReference game object in the hierarchy and has located between the AR Session Origin and AR Camera. This MiddleReference is the “referenceToScale” of the previous code. I’m pretty sure that my error is on the code, but I don’t know what it is.

Attached is the image of the Hierarchy Object’s disposition.

Thanks.

[doelwit]

    public void updateSize()
    {
        float val = scaleSlider.value;
      
        m_SessionOrigin.transform.localScale = new Vector3(val, val, val);
     
    }

    public void updateRotation()
    {
        float val = rotateSlider.value;
      
        m_SessionOrigin.transform.rotation = Quaternion.Euler(0, val, 0);

    }

these seem to work for me, they are both onvaluechanged listener delegates which I’ve added to the PlaceOnPlane script

[tonOnWu]

Ok, after a bit more than 24 hours later, I found the solution I want it. Now I’ll share it for other people that could have been having the same problem.

First I created an Empty Object called MiddleReference in the hierarchy and located in a middle position between AR Session Origin and its child AR Camera.

Then, in the AR Session Origin I placed the components: PlaceOnPlane:

using System.Collections.Generic;
using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.Experimental.XR;
using UnityEngine.XR.ARFoundation;

[RequireComponent(typeof(ARSessionOrigin), typeof(Scaler))]
public class PlaceOnPlane : MonoBehaviour
{
    ARPlaneManager planeManager;      
    private bool scenesPositionSet = false;     // Indicate that scale scenario and position has been set. If true, the game is ready to start
    private Scaler m_scaler;                      // We'll use this to change the reference for scaling

    [SerializeField]
    [Tooltip("Instantiates this prefab on a plane at the touch location.")]
    GameObject m_PlacedPrefab;          // Element to be set on scene. In case of a game, it will be the entire scenario
    public GameObject placedPrefab      // The prefab to instantiate on touch.
    {
        get { return m_PlacedPrefab; }
        set { m_PlacedPrefab = value; }
    }
  
    // The object instantiated as a result of a successful raycast intersection with a plane.
    public GameObject spawnedObject { get; private set; }

    ARSessionOrigin m_SessionOrigin;
  
    static List<ARRaycastHit> s_Hits = new List<ARRaycastHit>();
  
    void Awake()
    {
        m_SessionOrigin = GetComponent<ARSessionOrigin>();
        m_scaler = GetComponent<Scaler>();
    }

    void Update()
    {
        if (Input.touchCount > 0 && !IsPointerOverUIObject() && !scenesPositionSet)
        {
            Touch touch = Input.GetTouch(0);

            if (m_SessionOrigin.Raycast(touch.position, s_Hits, TrackableType.PlaneWithinPolygon))
            {
                Pose hitPose = s_Hits[0].pose;

                if (spawnedObject == null)
                {
                    spawnedObject = Instantiate(m_PlacedPrefab, hitPose.position, hitPose.rotation);
                }
                else
                {
                    spawnedObject.transform.position = hitPose.position;
                }

                // Locating reference on the same position
                if (m_scaler != null)
                {
                    m_scaler.referenceToScale.transform.position = hitPose.position;
                }
                else
                {
                    Debug.Log("Error: Scaler has not being initialized");
                }
            }
        }
    }

    // Remove planes, points and set everything
    public void SetScenario()
    {

        ARPlaneManager aRPlaneManager = GetComponent<ARPlaneManager>();
        aRPlaneManager.enabled = false;

        // Scene position has been set. So it won't be able to move it
        scenesPositionSet = true;

        //DisablePlanes();       // Uncomment this if you want to visual remove planes.

    }

    void OnPositionContent()
    {
        if (Input.touchCount > 0)
        {
            Touch touch = Input.GetTouch(0);

            if (m_SessionOrigin.Raycast(touch.position, s_Hits, TrackableType.PlaneWithinPolygon))
            {
                Pose hitPose = s_Hits[0].pose;

                if (spawnedObject == null)
                {
                    spawnedObject = Instantiate(m_PlacedPrefab);
                    DisablePlanes();
                }
                else
                {
                    spawnedObject.transform.position = hitPose.position;
                }
            }
        }
    }

    /* Procedure to remove planes from view to the user */
    void DisablePlanes()
    {
        planeManager = GetComponent<ARPlaneManager>();
        List<ARPlane> allPlanes = new List<ARPlane>();

        planeManager.GetAllPlanes(allPlanes);
        planeManager.enabled = false;
        foreach (ARPlane plane in allPlanes)
        {
            plane.gameObject.SetActive(false);
        }
    }

    /* Procedure to check if has been Tap over an UI object in a Canvas.
     * Returns True if an object in the Canvas has been Tapped. */
    private bool IsPointerOverUIObject()
    {        
        PointerEventData eventDataCurrentPosition = new PointerEventData(EventSystem.current);
        eventDataCurrentPosition.position = new Vector2(Input.mousePosition.x, Input.mousePosition.y);

        List<RaycastResult> results = new List<RaycastResult>();
        EventSystem.current.RaycastAll(eventDataCurrentPosition, results);
        return results.Count > 0;
    }

}

and the Scaler Component:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

using UnityEngine.Experimental.XR;
using UnityEngine.XR.ARFoundation;

[RequireComponent(typeof(ARSessionOrigin))]
public class Scaler : MonoBehaviour {

    ARSessionOrigin m_SessionOrigin;
    public GameObject referenceToScale;

    /* Next values must be the same min and max values of
     * the Slider to change the scale */
    private float m_maxScaleValue = 10.0f;
    private float m_minScaleValue = 0.0f;
    private float m_defaultScaleValue = 5.0f;

    void Awake()
    {
        m_SessionOrigin = GetComponent<ARSessionOrigin>();
    }

    // Method called by a Slider
    public void OnValueChange(float value)
    {
        Transform t = gameObject.transform;

        m_SessionOrigin.MakeContentAppearAt(
            referenceToScale.transform,
            referenceToScale.transform.position,
            referenceToScale.transform.rotation);

        float scaleValue = Mathf.Clamp(value, m_minScaleValue, m_maxScaleValue);
        t.localScale = new Vector3(scaleValue, scaleValue, scaleValue);
    }

    private void Start()
    {
        OnValueChange(m_defaultScaleValue);
    }
}

The Scaler component is called by a slider. The ReferenceToScale in the middle object mentioned at the beginning of this thread. But, with this code you can also move the object over an identified plane and then scale the object.

I attached an image of the setting. I hope more people can use this.

[Edur-Games]

For the object scale, using the ARSessionOrigin transform scale already works.

Scale (50, 50, 50) you’re scaling 50 times the smallest object.
This works well for me. About stabilizing objects once placed needs more debugging

[enhawk]

heres a script you can put on AR Session Origin to test this out without needing the example scene:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class CameraScaler : MonoBehaviour {

    [Range(1f,3f)]
    public float scale;
   
    // Update is called once per frame
    void Update () {
        transform.localScale = new Vector3(scale,scale,scale);
    }
}

[real2u]

@tdmowrer

To stay proportional to the content, the “stick” should be outside of the ARSessionOrigin hierarchy. You could then have a script that places it in front of the camera every frame.

Thanks for everything @tdmowrer , I’ve managed to get everything working flawlessly. Both scaling and rotating.

I was wondering if it’s possible to use a Static Gameobject due to it’s perfomance advantages. I’ve found this post https://blogs.unity3d.com/2017/11/16/dealing-with-scale-in-ar/ in which being Static is mentioned, is it possible to replicate with ARFoundation?

In my scene instead of using a prefab, I’m using a baked game object and I can move it as a I wish without a problem. But I would love to make it as performatic as possible.

Best regards,