How to best extend builtin component functionality in the editor

Unity Engine
1

This is more of a general question with a specific example: What are good ways to extend Unity builtin components such as Collider or LineRenderer with custom functionality in regards to custom editor code?

Of course at runtime, I mostly only have a choice of composing my own MonoBehaviour with a reference to said Unity components and then add my own fields and methods there. So now I can write a custom editor for my custom type and provide some additional editor only functionality.

Example:
I have a LineRenderer, which should always “loop” like a circle shape; this means the first and last point of the positions array need to be the same. Also I would like scene view Handles to drag each LineRenderer point in the scene.

I already have my desired features coded and working, but there are a few inconveniences:

  • Since I don’t have access to the LineRenderers editor, I need to create my own SerializedObject from a GetComponent reference and cache all SerializedProperties which I’m interested in myself.
  • I don’t get any changed events like GUI.changed, BeginChangeCheck, OnValidate or Reset.
  • Overall I have to write a lot of boilerplate code just to get information, which the editor already knows about, but doesn’t expose.

Is there any better approach to this sort of problem? I can post my code as an example, but I’m also interested in general advice.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
#if UNITY_EDITOR
using UnityEditor;
#endif

[RequireComponent(typeof(LineRenderer))]
public class LoopLineRenderer : MonoBehaviour
{
    void OnEnable()
    {
        // This class is empty because we only need it as a hook for our custom inspector.
        // We define OnEnable, because we can use the toggle to temporily disable the loop behaviour.
    }
}

#if UNITY_EDITOR
[CustomEditor(typeof(LoopLineRenderer))]
public class LoopLineRendererEditor : Editor
{
    Vector3 previousFirst;
    Vector3 previousLast;
    LoopLineRenderer script;
    SerializedObject lineRendererSO;
    SerializedProperty positionsProp;
    System.Func<Vector3, Vector3> handleTransformation;
    System.Func<Vector3, Vector3> handleInverseTransformation;

    void OnEnable()
    {
        script = (LoopLineRenderer)target;

        LineRenderer lineRenderer = script.GetComponent<LineRenderer>();
        if(lineRenderer.useWorldSpace)
            handleTransformation = v => v;
        else
            handleTransformation = v => lineRenderer.transform.TransformPoint(v);
       
        if(lineRenderer.useWorldSpace)
            handleInverseTransformation = v => v;
        else
            handleInverseTransformation = v => lineRenderer.transform.InverseTransformPoint(v);

        lineRendererSO = new SerializedObject(lineRenderer);
        positionsProp = lineRendererSO.FindProperty("m_Positions");
    }

    public override void OnInspectorGUI ()
    {
        base.OnInspectorGUI();

        EditorGUILayout.HelpBox("When enabled and folded out, this component ensures, that the first and last position vector of an attached LineRenderer match.", MessageType.Info);

        if(positionsProp.arraySize < 3)
            script.enabled = false;

        if(script.enabled && positionsProp.arraySize > 2)
        {
            lineRendererSO.Update();
            var first = positionsProp.GetArrayElementAtIndex(0);
            var last = positionsProp.GetArrayElementAtIndex(positionsProp.arraySize - 1);

            if(first.vector3Value != previousFirst)
                last.vector3Value = first.vector3Value;
           
            else if(last.vector3Value != previousLast)
                first.vector3Value = last.vector3Value;

            lineRendererSO.ApplyModifiedProperties();

            previousFirst = first.vector3Value;
            previousLast = last.vector3Value;
        }
    }

    void OnSceneGUI()
    {
        if(!script.enabled)
            return;
       
        var first = positionsProp.GetArrayElementAtIndex(0);
        var last = positionsProp.GetArrayElementAtIndex(positionsProp.arraySize - 1);
        drawHandleForProps(script.transform, first, last);

        for (int i = 1; i < positionsProp.arraySize - 1; i++)
            drawHandleForProp(script.transform, positionsProp.GetArrayElementAtIndex(i));

        lineRendererSO.ApplyModifiedProperties();
    }

    void drawHandleForProp(Transform transform, SerializedProperty positionProp)
    {
        EditorGUI.BeginChangeCheck();
        Vector3 newValue = Handles.PositionHandle(handleTransformation(positionProp.vector3Value), transform.rotation);
        if(EditorGUI.EndChangeCheck())
        {
            positionProp.vector3Value = handleInverseTransformation(newValue);
        }
    }

    void drawHandleForProps(Transform transform, SerializedProperty positionPropA, SerializedProperty positionPropB)
    {
        EditorGUI.BeginChangeCheck();
        Vector3 newValue = Handles.PositionHandle(handleTransformation(positionPropA.vector3Value), transform.rotation);
        if(EditorGUI.EndChangeCheck())
        {
            positionPropA.vector3Value = handleInverseTransformation(newValue);
            positionPropB.vector3Value = handleInverseTransformation(newValue);
        }
    }
}
#endif
2

Somehow all editor extension code eventually ends up looking something like this. People more acquainted with the API may do some fancy tricks by retrieving information using reflection, and people with an enterprise deal will just get the source code and change what needs changing. There is not much you can do in terms of clever OOP. Some classes here and there to wrap some functionality, but still ugly on the lowest level. For example, to check whether an orthographic SceneView is currently viewing from one of the main axes, I need to check the camera transform’s forward vector. However, there are rounding errors, so I need to do some kind of AlmostEqual() with some small threshold value. It’s ugly as hell, but it gets the job done. I would not worry about the code too much. It always gets messy as soon as you want to do just slightly more advanced stuff. Just make sure it actually works, is what I say anyway.

2 Likes
3

I think it’s better to do :

using UnityEngine;
using UnityEditor;

[CustomEditor(typeof(LineRenderer))]
public class LineRendererEditor : Editor
{
    private SerializedProperty m_positionProperty;
    private bool m_loop = false;
    private GUIContent m_loopContent = new GUIContent("Looping");

    protected LineRenderer target
    {
        get { return base.target as LineRenderer; }
    }
 
    public void OnEnable()
    {
        m_positionProperty = serializedObject.FindProperty("m_Positions");
        if (m_positionProperty.arraySize > 1)
        {
            m_loop = m_positionProperty.GetArrayElementAtIndex(0).vector3Value == m_positionProperty.GetArrayElementAtIndex(m_positionProperty.arraySize - 1).vector3Value;
        }
    }

    public override void OnInspectorGUI()
    {
        serializedObject.Update();
     
        EditorGUI.BeginChangeCheck();

        DrawDefaultInspector();

        m_loop = EditorGUILayout.Toggle(m_loopContent, m_loop);
        if (EditorGUI.EndChangeCheck())
        {
            CheckForceLoop();
        }

        serializedObject.ApplyModifiedProperties();
    }

    void OnSceneGUI()
    {
        serializedObject.Update();

        EditorGUI.BeginChangeCheck();
        for (int i = 0; i < (m_loop?m_positionProperty.arraySize-1:m_positionProperty.arraySize); i++)
            DrawHandleForProp(target.transform, m_positionProperty.GetArrayElementAtIndex(i));
        if (EditorGUI.EndChangeCheck())
        {
            CheckForceLoop();
        }
        serializedObject.ApplyModifiedProperties();
    }

    void DrawHandleForProp(Transform transform, SerializedProperty positionProp)
    {
        EditorGUI.BeginChangeCheck();
        Vector3 position = positionProp.vector3Value;
        if (!target.useWorldSpace)
            position = target.transform.TransformPoint(position);

        position = Handles.PositionHandle(position, transform.rotation);
        if (EditorGUI.EndChangeCheck())
        {
            if (!target.useWorldSpace)
                position = target.transform.InverseTransformPoint(position);
            positionProp.vector3Value = position;
        }
    }

    void CheckForceLoop()
    {
        if (m_loop && m_positionProperty.arraySize > 1)
        {
            m_positionProperty.GetArrayElementAtIndex(m_positionProperty.arraySize - 1).vector3Value = m_positionProperty.GetArrayElementAtIndex(0).vector3Value;
        }
    }
}

You can make a custom editor for a built-in type, so no more useless component LoopLineRenderer and no need to create a new SerializedObject.

Enjoy!

4

Thanks for this suggestion! Some ideas I like, however, one issue with this approach is, that I might not want all my LineRenderers to have this functionality. At least with some other more specialized things, it could be an issue. With this, it would probably be ok.

5

Great piece of code. But there is no such thing as “base editor”, instead just “Editor”

6

Whoops, sorry. Forgot to replace my custom class, I edited my post, thanks.

7

Just wanted to share, that it works pretty well to extend Unity components in the following pattern:

  • Override Editor for builtin component
  • Check for specific conditions like “Use custom inspector only when attached renderer also has a certain material”
  • Search UnityEditor assembly for specific builtin inspector e.g. SpriteRendererEditor and instantiate
  • Use the instantiated editor within custom editor to draw default implementation and optionally adding custom functionality depending on condition

This of course only applies for editors, which are not exposed and have a custom implementation made by Unity.

2 Likes