Re-sizing/scaling menus as holo user walks around

I am trying to write two scripts, one attached to the HoloLensCamera, where it updates the size/scale of a canvas as the user (i.e. the hololens camera) moves back and forth, so that it always stays as big, and the reverse; and the other attached to any game object, as a generalization of the first script, to work on any scene object.

I thought (late)updating the height of the UI canvas to the holo cam’s frustum height would make sense, at least initially, so that the menu always takes up exactly the vertical height of the camera’s view. In the second (more general) script, I basically re-scale the game object.

The following shows my work so far, but I need help. First, I would like to know if this is the right approach. Second, I would like to know how to re-size any child objects (e.g. canvas elements) so that the growing and shrinking is consistent across the entire canvas and panels and buttons, etc… In general, I would appreciate help in getting this right.

/// <summary>
/// Updates the size/scale of a canvas as the user
/// (i.e. the hololens camera) moves back and forth.
/// This script should be attached to the HoloLensCamera.
/// It will re-size the canvas to fit the frustum height,
/// or whatever specified portion of it.
/// </summary>
public class CanvasScaler : MonoBehaviour
{
    /// <summary>
    /// In the Inspector panel, drag and drop the UI
    /// element whose size you would like to adjust.
    /// Initially, this is set to the UI Canvas.
    /// </summary>
    public RectTransform _rectTransform;

    /// <summary>
    /// The width-to-height ratio of the UI element.
    /// Initially the UI Canvas.
    /// </summary>
    private float _ratio;

    /// <summary>
    /// Pixels per Unit for converting units to pixels.
    /// </summary>
    private float _pixPerUnit = 100f;

    /// <summary>
    /// Th gaze origin which is the user's head position
    /// as well as the HoloLensCamera' location in the world.
    /// </summary>
    private Vector3 _gazeOrigin;

    /// <summary>
    /// The UI element's origin. Initially the UI Canvas.
    /// </summary>
    private Vector3 _uiOrigin;

    /// <summary>
    /// The HoloLensCamera's (vertical) frustum height.
    /// The new re-sized Canvas height should be set to this,
    /// or some variation of it. Here, it is 1.5f instead of 2f
    /// that is used in the calculation formula taken from Unity Docs.
    /// </summary>
    private float _frustumHeight;

    /// <summary>
    /// The distance from the HoloLensCamera
    /// to the UI element. Initially the UI Canvas.
    /// </summary>
    private float _distance;

    /// <summary>
    /// Whether or not you want the size/scale re-adjusting.
    /// </summary>
    public bool _reSizeCanvas;


    private void Start()
    {
        Debug.Log("Canvas width: " + _rectTransform.rect.width + "  Canvas height: " + _rectTransform.rect.height);

        // In the Inspector Panel, check this option if
        // you would like the UI size-adjusting to work.
        if (_reSizeCanvas)
        {
            Debug.Log("UI size-adjusting in progress!");
            //Vector2 initialSize = _rectTransform.sizeDelta;
            _ratio = _rectTransform.rect.width / _rectTransform.rect.height;
        }
    }


    private void LateUpdate()
    {
        _gazeOrigin = transform.position;
        _uiOrigin = _rectTransform.position;
        _distance = Vector3.Distance(_gazeOrigin, _uiOrigin);

        // Calculate the new holoLens camera's (vertical) frustum height,
        // so that you can set the new re-sized Canvas height to this value.
        _frustumHeight = 1.5f * _distance * Mathf.Tan(this.GetComponent<Camera>().fieldOfView * 0.5f * Mathf.Deg2Rad);

        // Lerp from initial size to new size based on the distance change.
        // Find the new height from the camera frustum and the new width from the
        // initial canvas ratio, so that the width-to-height ratio always stays the same.
        _rectTransform.sizeDelta = new Vector2(_ratio * _frustumHeight * _pixPerUnit, _frustumHeight * _pixPerUnit); 
    }
}
/// <summary>
/// Updates the size/scale of an object as the user
/// (i.e. the hololens camera) moves back and forth.
/// This script should be attached to the game object.
/// </summary>
public class ObjectScaler : MonoBehaviour
{
    /// <summary>
    /// In the Inspector panel, drag and drop the
    /// intended object that needs to be re-sized.
    /// </summary>
    [SerializeField]
    public GameObject _renderCamera;

    private Vector3 _startingScale;
    private float _startingDistance;
    private float _currentDistance;


    void Start()
    {
        var objectTransformPos = this.transform.position;
        var cameraTransformPos = _renderCamera.transform.position;

        _startingDistance = Vector3.Distance(cameraTransformPos, objectTransformPos);
        _startingScale = this.transform.localScale;

        // min scale ?
    }


    void LateUpdate()
    {
        _currentDistance = Vector3.Distance(_renderCamera.transform.position, this.transform.position);
        this.transform.localScale = _startingScale * (_currentDistance / _startingDistance);
    }
}

As to whether or not it’s the right approach, that depends on the runtime behavior you’re looking for. If you always want the canvas to follow the user’s head around as if it were a screen-space camera, then that sounds like more-or-less the right approach to me (haven’t dug into the code you posted for the specifics, though).

However, as I said, that sounds like you’re basically making it a screen-space camera, which - in my experience - directly making it a screen-space camera in the first place is a pretty horribly, headache-inducing experience, which is why the thread I made here about Unity UI on the HoloLens doesn’t mention doing anything besides a world-space canvas.

I would consider looking at the behavior of the splash screen when you start up an app. I think that “tagalong” behavior would be pretty cool on UI, and should be somewhat straightforward. I’ve actually been wanting to expose that behavior in a component so it’d be easy to just attach it to a canvas for UI, but haven’t found the time for that just yet.

1 Like

Thanks for your reply. I have read through your informative thread. If I understand you correctly, you are saying that the canvas being in World Space is better. Right?

Essentially, I am not after a TagAlong or BillBoard kinda behavior. I just want the canvas to grow proportionately (along with its children; e.g. buttons, etc…) as the user walks off, so that it always seems like it is the same size.

Correct, any attempt that I’ve made at using Screen Space has only ended in tears and headaches. World Space is better.

To clarify, you want to keep the menu in the same spot, but grow the canvas then? It seems like you could get the same behavior in a more straight-forward way by repositioning the canvas like you would a billboard, but without changing its rotation (since you say you don’t want billboard-like behavior).

Let’s say, for example, that you find a good size for the canvas happens to line up with it being 5 meters away at some scale you experimentally find. You could write a component that remembers the canvas’s initial position, and knowing where the player is, you could just adjust it’s position based on this.

  • Get the vector from the player to the canvas’s original position
  • Normalize that vector, then scale it (in this example) by 5
  • Set the canvas’s position to the player’s position, offset by that normalized-then-scaled vector

If that isn’t the behavior you’re looking for, it sounds like your approach is on the right track, and I would need to take a deeper look at your code to correct whatever’s wrong.

1 Like

Precisely. I want to keep the menu in the same spot, but grow the canvas as the user (i.e. HoloLensCamera) moves away. I will edit my post to reflect my latest attempt at the code. Basically, now I am only trying to re-scale the fixed canvas (generally, any game object) but of course I do not want it to shrink smaller than a certain size. And my next challenge will be to re-size all the children as well, because I have noticed that even in WorldSpace render mode, the
canvas elements do not re-size/re-scale automatically. For example, if you just add a button, you will find that, upon running, the canvas will re-size but the button will stay where it is, even though it is technically a child of canvas. To fix this, I chose an anchor (for instance bottom right for a button) which leads to the button to be included in the re-sized canvas in the correct place, but still does not get re-sized itself…