UWP on-screen keyboard C++ plugin

[manakami]

Hi,

I am developing application of XboxOne and Win10 in UWP.

・Unity 2017.2.1p3
・script backend : IL2CPP
・Build Type : D3D

So we need On-Screen Keyboard on XboxOne.
However it does not seem to work with UWP add-on(?) as it is in the following thread:

UWP, XboxOne, InputFields, and onscreen keyboards (or lack thereof)

I've been porting a large project to the XboxOne on the UWP platform. Our game uses "D3D", rather than "XAML", for performance reasons. I've been working with Unity 5.6.3p4 mostly, though all of the above seems to be true for 2017.2 as well. I submitted a bug for an issue I've been struggling with: on this platform, selecting an InputField does not launch the On-Screen Keyboard (OSK) for text entry. Unity launches a touch keyboard automatically on mobile platforms (including, I think, UWP on W…

(UnityEngine.TouchScreenKeyboard)

So I am making plugin with C++/CX.
The goal is the same behavior as the TextSystem Plugin developed by XboxOne using XDK.
(On-Screen Keyboard + Entry Fiedl(TextBox?) + list of suggestions)

My strategy is to add a TextBox created with new to the current content (View? Page?).
And focus on TextBox, On-Screen Keyboard will automatically appear.

but have trouble.
In the case of non-XAML and C ++, I do not know how to dynamically add TextBox.
(In the case of XAML, information to be added to StackPanel etc. can be found)

Please tell me how to add TextBox dynamically or a better solution!

[Aurimas-Cernius]

Have you tried the Unity TouchScreenKeyboard?
It should work and it’s probably a bug if it doesn’t.

As for implementing it youself, CoreTextEdit is a better approach.

[manakami]

Yes, I tried it.
And it was as reported in the link.
(It does not work properly)

1. Cancelling the keyboard with the “B” button is completely not recognized by Unity. The keyboard disappears, and no other controller input is accepted; but the TouchScreenKeyboard instance, and the static variables, are all unchanged when this happens, so there’s no way to detect this state. (Tried the “visible”, “active”, “done”, “status”, and “wasCanceled” flags, to no success.)

UWP, XboxOne, InputFields, and onscreen keyboards (or lack thereof)

I’ve been porting a large project to the XboxOne on the UWP platform. Our game uses “D3D”, rather than “XAML”, for performance reasons. I’ve been working with Unity 5.6.3p4 mostly, though all of the above seems to be true for 2017.2 as well. I submitted a bug for an issue I’ve been struggling with: on this platform, selecting an InputField does not launch the On-Screen Keyboard (OSK) for text entry. Unity launches a touch keyboard automatically on mobile platforms (including, I think, UWP on Wi…

↑

Dec 18, 2017
Report

The Unity person at the linked site says the same thing as you, but the bug has not been fixed and we have no waiting time.
(Do you consider combining IL2CPP script backend and BuildType D3D?)

Oh…, maybe the version of .NET may also be involved.
Scripting Runtime Version : Experimental (.NET 4.6 Equivalemt)
Api Compatibility Level : .NET 4.6

But they are not important, they do not work in reality so they need to be replaced!
Is there any other way to avoid C ++ / CX programming?

[manakami]

– The second question –

We are creating on Unity’s Universal Windows platform.(
(Do not use XDK)
But the runtime environment is XboxOne.
Should this problem(issue?) be posted to a thread on XboxOne (other platform)?

[Tautvydas-Zilys]

This is the right forum when targeting UWP.

[manakami]

The second question was solved.
Thank you.

But the first question has not been solved.
The problem with UnityEngine.TouchScreenKeyboard is a bug in Unity.
Why does the Unity person not seriously deal with this problem?

Unity 2017.2.1p3
Universal Windows platform
Script backend : IL2CPP
Build Type : D3D
runtime : XboxOne
↑
Using UnityEngine.TouchScreenKeyboard is not a proper solution.
(Do not work)

Return to the first question.

using namespace Windows::UI::Xaml;
using namespace Windows::UI::Xaml::Controls;

// I want to add a TextBox to the current screen
auto textBox = new TextBox();

/*
I looked up to the following, but I do not know the future.
Or is there a different approach?

auto rootFrame = dynamic_castControls::Frame^(Window::Current->Content);

if (rootFrame != nullptr) {
auto pageType = rootFrame->SourcePageType;
// I do not know what to do from now on…
}
*/

Please help me.

[Tautvydas-Zilys]

@manakami

The problem with UnityEngine.TouchScreenKeyboard is a bug in Unity.
Why does the Unity person not seriously deal with this problem?

It’s on our backlog to look into. We will get to it soon.

[manakami]

Thank you!
I want you to release it soon.

And We do not have time, so please continue to tell me how to continue with C++CX.

[Aurimas-Cernius]

@manakami

And We do not have time, so please continue to tell me how to continue with C++CX.

If you are using D3D build type, I recommend you to use CoreTextEdit framework for opening touch keyboard, not XAML controls.
https://docs.microsoft.com/en-us/uwp/api/Windows.UI.Text.Core.CoreTextEditContext

[manakami]

If you are using D3D build type, I recommend you to use CoreTextEdit framework for opening touch keyboard, not XAML controls.

Thank you!
I was hoping for this expansion.

But I already use CoreTextEditContext.
Our goal is to make it the same as the InputPane appearing on the XboxOne XDK plugin.
There is a TextBox in which the text entered appears and the appearance of input prediction appears.

We believe CoreTextEditContext alone, it will not happen.
Please advise or tell me how to make it!

[manakami]

As a possibility…
Does CoreTextInputScope have specifications that will become functions we want?

We tried Text(57) and UserName(6), but only InputPane appeared.

CoreTextInputScope Enum (Windows.UI.Text.Core) - Windows apps

[manakami]

Not all, but our current code.
We think that it is necessary to add TextBox to InputPane(possible?) or Current View(Frame? Page?).

#include "pch.h"
#include "agile.h"
#include "UWPSoftInputPanel.h" // Our headers

using namespace Windows::ApplicationModel::Core;
using namespace Windows::UI::Text::Core;
using namespace Windows::UI::ViewManagement;
using namespace Windows::Foundation;
using namespace UWP; // Our namespace

// All methods are static so that they become Unity plugins.
// CreateOpen, OnTextUpdating, OnSelectionUpdating, OnShowing, OnHiding
void SoftInputPanel::CreateOpen()
{
    CoreApplication::MainView->Dispatcher->RunAsync(Windows::UI::Core::CoreDispatcherPriority::Normal, ref new Windows::UI::Core::smile:ispatchedHandler([=]()
    {
        // CoreTextEditContext^ _editContext
        _editContext = CoreTextServicesManager::GetForCurrentView()->CreateEditContext();

        if (_editContext != nullptr)
        {
            _editContext->InputPaneDisplayPolicy = CoreTextInputPaneDisplayPolicy::Manual;
            _editContext->InputScope = CoreTextInputScope::Text;

            auto updateToken = _editContext->TextUpdating += ref new TypedEventHandler<CoreTextEditContext^, CoreTextTextUpdatingEventArgs^>(&SoftInputPanel::OnTextUpdating);
            // EventRegistrationToken* _updateToken
            _updateToken = &(updateToken);

            auto selectionToken = _editContext->SelectionUpdating += ref new TypedEventHandler<CoreTextEditContext^, CoreTextSelectionUpdatingEventArgs^>(&SoftInputPanel::OnSelectionUpdating);
            // EventRegistrationToken* _selectionToken
            _selectionToken = &(selectionToken);

            // Platform::Agile<InputPane> _inputPane
            _inputPane = InputPane::GetForCurrentView();

            if (_inputPane != nullptr)
            {

                auto showingToken = _inputPane->Showing += ref new TypedEventHandler<InputPane^, InputPaneVisibilityEventArgs^>(&SoftInputPanel::OnShowing);
                // EventRegistrationToken* _showingToke
                _showingToken = &(showingToken);

                auto hidingToken = _inputPane->Hiding += ref new TypedEventHandler<InputPane^, InputPaneVisibilityEventArgs^>(&SoftInputPanel::OnHiding);
                // EventRegistrationToken* _hidingToken
                _hidingToken = &(hidingToken);

                _inputPane->TryShow();
            }
        }
    }));
}