Feature Request: Control of mobile TouchScreenKeyboard visibility on focus change

Unity Engine
,
21

Which UI system are you using ? IMGUI or UGUI?

22

@Tomas1856 uGUI

23

@Tomas1856 Any update on this?

24

We talked internally, and it seems thereā€™s a mixup with Unity - Scripting API: TouchScreenKeyboard.Android.closeKeyboardOnOutsideTap name.

What it actually does - it controls to consume touch events or pass touch events through while TouchScreenKeyboard is shown.

The property will be renamed to consumesTouchEvents to reflect better its functionality.

In 2022.1, its default value is false, thus TouchScreenKeyboard on Android will behave the same as iOS
Prior to 2022.1, its default value is true, thus not to break users who rely on legacy TouchScreenKeyboard Android behavior.

In 2023.x, this property will be deprecated, and finally in 2024.x it will be removed, thus ending TouchScreenKeyboard Android behavior transition to match iOS.

Finally, for the issue above, to hide or to keep TouchScreenKeyboard when touch events are passed through, is controlled by UGUI system itself, I assume current logic dictates if control looses focus, UGUI tells TouchScreenKeyboard to hide. I am not aware of any properties allow to override this behavior.

25

@Tomas1856 I feel kinda dumb but how do you set closeKeyboardOnOutsideTap to false? It doesnā€™t seem to excist when I type it (2021.1.12f1)

26

2021.1 is an unsupported version, try using 2021.2

27

Hey there, I appreciate the work being done to support this feature. Thank you! This is a great step towards it, I can now interact with unity while the keyboard is open.

Itā€™s still not perfect though as Nyankoooo and TheVirtualMunk have said. My interaction is limited to a single tap, which forces the keyboard to close. This means, for example, no scrolling through a list of results that are populated as you type. A common desired end result for a feature like this I would say. If thereā€™s a way to keep the keyboard open as input is passed through, this would be 100% solved!

28

Facing the same problem so annoying. Pleas resolve this.

29

I have the same issue. "closeKeyboardOnOutsideTap " does not completely solve the problem.
I would like a way to keep the mobile TouchScreenKeyboard open while focus is on the inputfield.

30

I am also facing the same problem and very confused.
In my app, the operability of this UI is one of the important functions.
I think Unity is a great service for creating games.
By all means, I would like to ask you to improve the UI.
Hey Unity, do you have any plans to fix this?

31

Unity mobile keyboard controller bridge is horrible! For more then 6 years you are focused on mobile app development and still canā€™t make is work well with full functionality. We have to use custom native plugins to make it work well. Furthermore, on Android device TouchKeyboard NEVER shows the keyboard height. Wtf?

32

If I set TouchScreenKeyboard.Android.consumesOutsideTouches = true; in 2021.3 and tap into the multiline input field, the mobile keyboard shows up on the screen. When I tap again onto the mobile screen, somewhere in the app, the mobile keyboard does not disappear; it stays open. This is great, butā€¦ the app does not react on any taps anymore.

This is a bit unfortunate, as I cannot even move the cursor in the input field to a different position with a touch while the keyboard is open. I need to close the keyboard, and tap again, hoping that my tap is accurate enough to place the cursor accurately.

Great so far, but still room for improvement, certainly.

33

Apologies; typo in my reply above:
The app does react on taps, but the input field doesnā€™t.
Sorry; I canā€™t edit the post; get a spam warning.

34

I just had to edit the text mesh pro package. Itā€™s quite an annoying solution, Iā€™m surprised that we canā€™t have full control of the keyboard easily without having to make our ownā€¦ Interesting eh? o.O

35

Hi there,
I think I found the culprit that prevents the selection if Hide Input Mobile = true with a TMP_InputField and the option TouchScreenKeyboard.Android.consumesOutsideTouches = true.
In my opinion there is a ! missing in front of m_HideMobileInput on line 1623 from TMP_InputField.
To solve the problem temporarily, I created a class that overrides TMP_InputField and therefore calls Reflection.
Itā€™s not a long term solution but it do the trick for the moment.

using UnityEditor;
using TMPro.EditorUtilities;
using TMPro;

[CustomEditor(typeof(TMP_TouchInputField))]
public class TMP_TouchInputFieldEditor : TMP_InputFieldEditor
{
    #region Context menu
    [MenuItem("CONTEXT/TMP_InputField/Convert To TMP_TouchInputField", true)]
    private static bool _ConvertToTMP_TouchInputField(MenuCommand command)
    {
       return EditorUtils.CanConvertTo<TMP_TouchInputField>(command.context);
    }

    [MenuItem("CONTEXT/TMP_InputField/Convert To TMP_TouchInputField", false)]
    private static void ConvertToTMP_TouchInputField(MenuCommand command)
    {
       EditorUtils.ConvertTo<TMP_TouchInputField>(command.context);
    }

    [MenuItem("CONTEXT/TMP_TouchInputField/Convert To TMP_InputField", true)]
    private static bool _ConvertToTMP_InputField(MenuCommand command)
    {
        return EditorUtils.CanConvertTo<TMP_InputField>(command.context);
    }

    [MenuItem("CONTEXT/TMP_TouchInputField/Convert To TMP_InputField", false)]
    private static void ConvertToTMP_InputField(MenuCommand command)
    {
        EditorUtils.ConvertTo<TMP_InputField>(command.context);
    }
   #endregion
}

public static class EditorUtils
    {
        /// <summary>
        /// Verify whether it can be converted to the specified component.
        /// </summary>
        public static bool CanConvertTo<T>(Object context) where T : MonoBehaviour
        {
            return context && context.GetType() != typeof(T);
        }

        /// <summary>
        /// Convert to the specified component.
        /// </summary>
        public static void ConvertTo<T>(Object context) where T : MonoBehaviour
        {
            if (PrefabUtility.IsPartOfVariantPrefab(context))
            {
                Debug.LogError("Cannot convert a component from a variant prefab!");
                return;
            }

            var target = context as MonoBehaviour;
            var so = new SerializedObject(target);
            so.Update();

            var oldEnable = target.enabled;
            target.enabled = false;

            // Find MonoScript of the specified component.
            foreach (var script in Resources.FindObjectsOfTypeAll<MonoScript>())
            {
                if (script.GetClass() != typeof(T))
                    continue;

                // Set 'm_Script' to convert.
                so.FindProperty("m_Script").objectReferenceValue = script;
                so.ApplyModifiedProperties();
                break;
            }

            (so.targetObject as MonoBehaviour).enabled = oldEnable;
        }
    }

using UnityEngine;
using TMPro;
using UnityEngine.EventSystems;
using System.Reflection;


/// <summary>
/// TMP_InputField overload class to fix an Android specific bug.
/// [Bug: cannot select a part of the text if m_HideMobileInput = true]
/// This class will disappear when the bug will be fixed by Unity in a future update.
/// </summary>

public class TMP_TouchInputField : TMP_InputField
{
    #region Attributes
    private FieldInfo m_ShouldActivateNextUpdateInfo;
    private MethodInfo activateInputFieldInternalInfo;
    private FieldInfo m_SelectionStillActiveInfo;
    private FieldInfo m_PreviouslySelectedObjectInfo;
    private FieldInfo m_KeyDownStartTimeInfo;
    private FieldInfo m_DoubleClickDelayInfo;
    private MethodInfo updateMaskRegionsInfo;
    private MethodInfo inPlaceEditingInfo;
    private MethodInfo isKeyboardUsingEventsInfo;
    private MethodInfo assignPositioningIfNeededInfo;
    private FieldInfo m_ReleaseSelectionInfo;
    private FieldInfo m_WasCanceledInfo;
    private MethodInfo updateStringPositionFromKeyboardInfo;
    private MethodInfo sendOnValueChangedAndUpdateLabelInfo;
    private FieldInfo m_ProcessingEventInfo;
    #endregion

    #region Unity methods
    protected override void Awake()
    {
        base.Awake();

        m_ShouldActivateNextUpdateInfo = typeof(TMP_InputField).GetField("m_ShouldActivateNextUpdate", BindingFlags.NonPublic | BindingFlags.Instance);
        activateInputFieldInternalInfo = typeof(TMP_InputField).GetMethod("ActivateInputFieldInternal", BindingFlags.NonPublic | BindingFlags.Instance);
        m_SelectionStillActiveInfo = typeof(TMP_InputField).GetField("m_SelectionStillActive", BindingFlags.NonPublic | BindingFlags.Instance);
        m_PreviouslySelectedObjectInfo = typeof(TMP_InputField).GetField("m_PreviouslySelectedObject", BindingFlags.NonPublic | BindingFlags.Instance);
        m_KeyDownStartTimeInfo = typeof(TMP_InputField).GetField("m_KeyDownStartTime", BindingFlags.NonPublic | BindingFlags.Instance);
        m_DoubleClickDelayInfo = typeof(TMP_InputField).GetField("m_DoubleClickDelay", BindingFlags.NonPublic | BindingFlags.Instance);
        updateMaskRegionsInfo = typeof(TMP_InputField).GetMethod("UpdateMaskRegions", BindingFlags.NonPublic | BindingFlags.Instance);
        inPlaceEditingInfo = typeof(TMP_InputField).GetMethod("InPlaceEditing", BindingFlags.NonPublic | BindingFlags.Instance);
        isKeyboardUsingEventsInfo = typeof(TMP_InputField).GetMethod("isKeyboardUsingEvents", BindingFlags.NonPublic | BindingFlags.Instance);
        assignPositioningIfNeededInfo = typeof(TMP_InputField).GetMethod("AssignPositioningIfNeeded", BindingFlags.NonPublic | BindingFlags.Instance);
        m_ReleaseSelectionInfo = typeof(TMP_InputField).GetField("m_ReleaseSelection", BindingFlags.NonPublic | BindingFlags.Instance);
        m_WasCanceledInfo = typeof(TMP_InputField).GetField("m_WasCanceled", BindingFlags.NonPublic | BindingFlags.Instance);
        updateStringPositionFromKeyboardInfo = typeof(TMP_InputField).GetMethod("UpdateStringPositionFromKeyboard", BindingFlags.NonPublic | BindingFlags.Instance);
        sendOnValueChangedAndUpdateLabelInfo = typeof(TMP_InputField).GetMethod("SendOnValueChangedAndUpdateLabel", BindingFlags.NonPublic | BindingFlags.Instance);
        m_ProcessingEventInfo = typeof(TMP_InputField).GetField("m_ProcessingEvent", BindingFlags.NonPublic | BindingFlags.Instance);
    }

    protected override void LateUpdate()
    {
        if (!Application.isPlaying || Application.platform != RuntimePlatform.Android)
        {
            base.LateUpdate();
            return;
        }

        // Only activate if we are not already activated.
        if ((bool)m_ShouldActivateNextUpdateInfo.GetValue(this)) // => if (m_ShouldActivateNextUpdate)
        {
            if (!isFocused)
            {
                activateInputFieldInternalInfo.Invoke(this, null); // => ActivateInputFieldInternal();
                m_ShouldActivateNextUpdateInfo.SetValue(this, false); // => m_ShouldActivateNextUpdate = false;
                return;
            }

            // Reset as we are already activated.
            m_ShouldActivateNextUpdateInfo.SetValue(this, false); // => m_ShouldActivateNextUpdate = false;
        }

        // Handle double click to reset / deselect Input Field when ResetOnActivation is false.
        if (!isFocused && (bool)m_SelectionStillActiveInfo.GetValue(this)) // => if (!isFocused && m_SelectionStillActive)
        {
            GameObject selectedObject = EventSystem.current != null ? EventSystem.current.currentSelectedGameObject : null;

            if (selectedObject == null && m_ResetOnDeActivation)
            {
                ReleaseSelection();
                return;
            }

            if (selectedObject != null && selectedObject != this.gameObject)
            {
                if (selectedObject == (GameObject)m_PreviouslySelectedObjectInfo.GetValue(this)) // => if (selectedObject == m_PreviouslySelectedObject)
                    return;

                m_PreviouslySelectedObjectInfo.SetValue(this, selectedObject); // => m_PreviouslySelectedObject = selectedObject;

                // Special handling for Vertical Scrollbar
                if (m_VerticalScrollbar && selectedObject == m_VerticalScrollbar.gameObject)
                {
                    // Do not release selection
                    return;
                }

                // Release selection for all objects when ResetOnDeActivation is true
                if (m_ResetOnDeActivation)
                {
                    ReleaseSelection();
                    return;
                }

                // Release current selection of selected object is another Input Field
                if (selectedObject.GetComponent<TMP_InputField>() != null)
                    ReleaseSelection();

                return;
            }
#if ENABLE_INPUT_SYSTEM
                Event processingEvent = (Event)m_ProcessingEventInfo.GetValue(this);

                //if (m_ProcessingEvent != null && m_ProcessingEvent.rawType == EventType.MouseDown && m_ProcessingEvent.button == 0)
                if (processingEvent != null && processingEvent.rawType == EventType.MouseDown && processingEvent.button == 0)
                {
                    // Check for Double Click
                    bool isDoubleClick = false;
                    float timeStamp = Time.unscaledTime;

                    if ((float)m_KeyDownStartTimeInfo.GetValue(this) + (float)m_DoubleClickDelayInfo.GetValue(this) > timeStamp) // => if (m_KeyDownStartTime + m_DoubleClickDelay > timeStamp)
                        isDoubleClick = true;

                    m_KeyDownStartTimeInfo.SetValue(this, timeStamp); // => m_KeyDownStartTime = timeStamp;

                    if (isDoubleClick)
                    {
                        //m_StringPosition = m_StringSelectPosition = 0;
                        //m_CaretPosition = m_CaretSelectPosition = 0;
                        //m_TextComponent.rectTransform.localPosition = m_DefaultTransformPosition;

                        //if (caretRectTrans != null)
                        //    caretRectTrans.localPosition = Vector3.zero;

                        ReleaseSelection();

                        return;
                    }
                }
#else
            if (Input.GetKeyDown(KeyCode.Mouse0))
            {
                // Check for Double Click
                bool isDoubleClick = false;
                float timeStamp = Time.unscaledTime;

                if ((float)m_KeyDownStartTimeInfo.GetValue(this) + (float)m_DoubleClickDelayInfo.GetValue(this) > timeStamp) // => if (m_KeyDownStartTime + m_DoubleClickDelay > timeStamp)
                    isDoubleClick = true;

                m_KeyDownStartTimeInfo.SetValue(this, timeStamp); // => m_KeyDownStartTime = timeStamp;

                if (isDoubleClick)
                {
                    //m_StringPosition = m_StringSelectPosition = 0;
                    //m_CaretPosition = m_CaretSelectPosition = 0;
                    //m_TextComponent.rectTransform.localPosition = m_DefaultTransformPosition;

                    //if (caretRectTrans != null)
                    //    caretRectTrans.localPosition = Vector3.zero;

                    ReleaseSelection();

                    return;
                }
            }
#endif
        }

        updateMaskRegionsInfo.Invoke(this, null); // => UpdateMaskRegions();

        if ((bool)inPlaceEditingInfo.Invoke(this, null) && (bool)isKeyboardUsingEventsInfo.Invoke(this, null) || !isFocused) // => if (InPlaceEditing() && isKeyboardUsingEvents() || !isFocused)
        {
            return;
        }

        assignPositioningIfNeededInfo.Invoke(this, null); // => AssignPositioningIfNeeded();

        if (m_SoftKeyboard == null || m_SoftKeyboard.status != TouchScreenKeyboard.Status.Visible)
        {
            if (m_SoftKeyboard != null)
            {
                if (!readOnly)
                    text = m_SoftKeyboard.text;

                if (m_SoftKeyboard.status == TouchScreenKeyboard.Status.LostFocus)
                    SendTouchScreenKeyboardStatusChanged();

                if (m_SoftKeyboard.status == TouchScreenKeyboard.Status.Canceled)
                {
                    m_ReleaseSelectionInfo.SetValue(this, true); // => m_ReleaseSelection = true;
                    m_WasCanceledInfo.SetValue(this, true); // => m_WasCanceled = true;
                    SendTouchScreenKeyboardStatusChanged();
                }

                if (m_SoftKeyboard.status == TouchScreenKeyboard.Status.Done)
                {
                    m_ReleaseSelectionInfo.SetValue(this, true); // => m_ReleaseSelection = true;
                    OnSubmit(null);
                    SendTouchScreenKeyboardStatusChanged();
                }
            }

            OnDeselect(null);
            return;
        }

        string val = m_SoftKeyboard.text;

        if (m_Text != val)
        {
            if (readOnly) // => if (m_ReadOnly)
            {
                m_SoftKeyboard.text = m_Text;
            }
            else
            {
                m_Text = "";

                for (int i = 0; i < val.Length; ++i)
                {
                    char c = val[i];

                    if (c == '\r' || c == 3)
                        c = '\n';

                    if (onValidateInput != null)
                        c = onValidateInput(m_Text, m_Text.Length, c);
                    else if (characterValidation != CharacterValidation.None)
                        c = Validate(m_Text, m_Text.Length, c);

                    if (lineType == LineType.MultiLineSubmit && c == '\n')
                    {
                        m_SoftKeyboard.text = m_Text;

                        OnSubmit(null);
                        OnDeselect(null);
                        return;
                    }

                    if (c != 0)
                        m_Text += c;
                }

                if (characterLimit > 0 && m_Text.Length > characterLimit)
                    m_Text = m_Text.Substring(0, characterLimit);

                updateStringPositionFromKeyboardInfo.Invoke(this, null); // => UpdateStringPositionFromKeyboard();

                // Set keyboard text before updating label, as we might have changed it with validation
                // and update label will take the old value from keyboard if we don't change it here
                if (m_Text != val)
                    m_SoftKeyboard.text = m_Text;

                sendOnValueChangedAndUpdateLabelInfo.Invoke(this, null); // => SendOnValueChangedAndUpdateLabel();
            }
        }
        else if (!shouldHideMobileInput && Application.platform == RuntimePlatform.Android)
        {
            updateStringPositionFromKeyboardInfo.Invoke(this, null); // => UpdateStringPositionFromKeyboard();
        }

        //else if (m_HideMobileInput) // m_Keyboard.canSetSelection
        //{
        //    int length = stringPositionInternal < stringSelectPositionInternal ? stringSelectPositionInternal - stringPositionInternal : stringPositionInternal - stringSelectPositionInternal;
        //    m_SoftKeyboard.selection = new RangeInt(stringPositionInternal < stringSelectPositionInternal ? stringPositionInternal : stringSelectPositionInternal, length);
        //}
        //else if (!m_HideMobileInput) // m_Keyboard.canGetSelection)
        //{
        //    UpdateStringPositionFromKeyboard();
        //}

        if (m_SoftKeyboard != null && m_SoftKeyboard.status != TouchScreenKeyboard.Status.Visible)
        {
            if (m_SoftKeyboard.status == TouchScreenKeyboard.Status.Canceled)
                m_WasCanceledInfo.SetValue(this, true); // => m_WasCanceled = true;

            OnDeselect(null);
        }
    }
    #endregion
}
36

Bump. There is the same problem with a couple of projects.

37

Ah, this problem has troubled me for a long time, thank you very much for locating the problem.
However, this modification will cause the arrow keys, select-all button, etc. in my phoneā€™s IME to not work properly.
It seems that the problem is that functions such as OnDrag() update the selection area, but m_SoftKeyboard.selection is not refreshed synchronously.
So the better solution may be refreshing the selection area of the soft keyboard synchronously(call the code UpdateSelection() below) after OnDragEnd ,OnPointerDown,
MouseDragOutsideRect, and prevent UpdateStringPositionFromKeyboard when dragging(m_UpdateDrag==true)
Too many private functions, it seems to be a better idea to fork out to modify(as attached file)

 void UpdateSelection()
        {
              
            int length = stringPositionInternal < stringSelectPositionInternal ? stringSelectPositionInternal - stringPositionInternal : stringPositionInternal - stringSelectPositionInternal;
            m_SoftKeyboard.selection = new RangeInt(stringPositionInternal < stringSelectPositionInternal ? stringPositionInternal : stringSelectPositionInternal, length);
          
        }

9002758ā€“1240453ā€“TMP_InputField.cs (167 KB)

38
Reply

Since the whole TMP_InputField class is open-source, it would be better to just copy the code directly and fiddle however you want instead of relying on reflection.

39

if anyone is still looking for an answer, try this. This worked for me.

  1. Copy Whole TMP_InputField script, and make a new file.

  2. Delete following codes in DeactivateInputField(bool) method.

m_SoftKeyboard.active = false;
m_SoftKeyboard = null;

I would recommend making a new method like this, because you need to close the keyboard manually.

public void ReleaseKeyboard()
{
     if(m_SoftKeyboard == null) return;

     m_SoftKeyboard.active = false;
     m_SoftKeyboard = null;
}
  1. For Android set TouchScreenKeyboard.Android.consumesOutsideTouches to false.
    The official document says itā€™s default value is false, but somehow didnā€™t work until i manually set it to false.
    Unity - Scripting API: TouchScreenKeyboard.Android.consumesOutsideTouches
40

Any official news about this? Itā€™s been 4 years from first message.