UWP, XboxOne, InputFields, and onscreen keyboards (or lack thereof)

I’ve been porting a large project to the XboxOne on the UWP platform. Our game uses “D3D”, rather than “XAML”, for performance reasons. I’ve been working with Unity 5.6.3p4 mostly, though all of the above seems to be true for 2017.2 as well.

I submitted a bug for an issue I’ve been struggling with: on this platform, selecting an InputField does not launch the On-Screen Keyboard (OSK) for text entry. Unity launches a touch keyboard automatically on mobile platforms (including, I think, UWP on Windows Phone)… but it definitely does not automatically handle text entry on Xbox One.

While TouchScreenKeyboard.isSupported reports FALSE on this platform, I found that using TouchScreenKeyboard.Open() does work for opening the OSK on this platform and retrieving the entered text after the user presses the Menu/submit button.

Using this to create a workaround seems like it would be trivial; and indeed doing that is what Unity rep Tautvydas-Zilys suggests in this forum thread .

However, there are two major issues I’ve had trying this, which aren’t discussed elsewhere:

  • Cancelling the keyboard with the “B” button is completely not recognized by Unity. The keyboard disappears, and no other controller input is accepted; but the TouchScreenKeyboard instance, and the static variables, are all unchanged when this happens, so there’s no way to detect this state. (Tried the “visible”, “active”, “done”, “status”, and “wasCanceled” flags, to no success.)
  • The OSK’s text entry field is not shown (ie the white box above the keyboard that shows the text being entered). For our InputFields on the lower half of the screen, this means the player can’t read the text they’re entering.

So my question (preferably for a Unity dev) is: I successfully created a native plugin to let me fix #1. Do I have to do the same to fix #2?

More importantly, does Unity plan to implement this missing functionality? This is important UI functionality, and I assume the relatively few mentions of it on the forums only because there aren’t very many XboxOne UWP games made in Unity. The keyboard can obviously be launched and used to some extent since TouchScreenKeyboard (mostly) functions; why isn’t the full functionality there to do all this automatically, without the need for a native plugin and a workaround script?

This issue seems to imply that it’s “by design” that D3D apps in UWP are unable to support the onscreen keyboard, and that this is only supported on XAML. However we need to use D3D, and we need this functionality.

I hope a Unity dev can advise on the right way to go here, or at least someone who’s dealt with this problem and can confirm I’m going down a reasonable path with this workaround.

I wasn’t aware that this wasn’t working. The keyboard on d3d build type has been fixed a while ago - in Unity 5.5. All of this sounds very bizarre.

The answer is yes, and I strongly believe that the current behaviour is simply a bug. Could we get a bug report on that?

Thanks for the prompt response. The thread from February about this issue led me to believe that was missing functionality, and that a TouchScreenKeyboard workaround was the only solution.

The bug I submitted recently is here: https://fogbugz.unity3d.com/default.asp?978369_h663u031puqcddtr

If it makes a difference: I am testing on an xbox kit flashed with OS version 16299.3036. I can restore it to a different version of the OS if you believe this is an issue with the OS.

I also forgot to mention that I’m using InControl for gamepad interaction - but I believe this is just using standard Unity methods for activating an InputField.

I just tried making a new repro project for this. It does not appear to happen in XAML projects, but it definitely happens to this D3D test project… attached.

3324367–258907–UWPInputFieldBugRepro.zip (106 KB)

Thanks, we’ll take a look.

1 Like

(And just to be clear - the repro project I added does NOT use the InControl plugin, and still demonstrates this problem, so InControl isn’t the culprit here.)

Thanks again for looking into this @Tautvydas-Zilys !

@Tautvydas-Zilys it appears this is still broken. Obviously not a lot of people need this functionality but I just wanted to state, for the record, my desire to have it fixed. Thanks!

Yeah, I’m aware. It’s pretty high in priority to look into - I hope to get to it next week.

Good news: I fixed it last week and I’m going to be backporting the fix all the way to 2017.4 in coming weeks.

Thank you very much! \o/

I use a latest Unity 2018.1.2f1 and TMP_InputField (latest version) on Creators Xbox One and it does not bring the screen keyboard like in other platforms eg mobile. This is very important.

A year and a half after this bug was found, Unity has released new major versions (2018) but this bug is still there in D3D! This makes the Creators program from Microsoft via Unity completely useless in D3D. All Apps and Games need an onscreen keyboard.

The fix is landing to patch releases very soon. I fixed it a couple of weeks ago.

1 Like

Thats good to hear but I already made my own custom keyboard for these platforms.

Its been 2 weeks but I didnt see it in the latest patches. When do you expect it?

2 Likes

I am curious as well. My project is still using a workaround script to bring this keyboard up and dismiss it; and bringing up the keyboard this way has at least one issue (it doesn’t display the text entry, so for InputFields in the lower half of the screen, the player can’t see the text they’re entering as they’re entering it).

And the workaround doesn’t work for bringing up the OSK on Windows 10 at all. (Is that supported? It’s something we’d like to do through the OS when the player is using a gamepad.)

1 Like

The fix should work on Windows Desktop, Mobile and Xbox one.

The fix landed to 2018.1.8f1 and 2017.4.7f1.

We don’t have any support for it on the Standalone player. As for UWP, while we do have support for it, I don’t think that will work. I believe Windows itself won’t show a keyboard if a physical one is connected (at least that was the case few versions ago, I don’t know if it changed). Feel free to report a bug if you think we should investigate this scenario further.

It seems it is still not possible to detect when the “B” or “Back” button is pressed and the OSK is dismissed. When dismissed, the TSK instance still shows Active as True and Status as Visible even though the keyboard is gone. Is this desired behavior? Is there another way to detect when the OSK is dismissed on Xbox One?

That is totally unexpected. Which Unity version are you on?

Unity 2018.2.14f1