Hi all
I am developing apps on behalf of Special Effect and aiming for Switch Access to be able scan through in game buttons, as well as enabling use with TalkBack and VoiceOver.
I am unable to access any class within UnityEngine.Accessibility.
This code:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Accessibility;
public class TestingAccessibility : MonoBehaviour
{
AccessibilityHierarchy accessibilityHierarchy;
}
Gives the following error:
Assets\TestingAccessibility.cs(8,5): error CS1069: The type name ‘AccessibilityHierarchy’ could not be found in the namespace ‘UnityEngine.Accessibility’. This type has been forwarded to assembly ‘UnityEngine.AccessibilityModule, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null’ Enable the built in package ‘Accessibility’ in the Package Manager window to fix this error.
However, there is no package called Accessibility visible in the package manager.
In the above example I have attempted to use the AccessibilityHierarchy class, but the same error is given when attempting to access any class.
I’m probably doing something daft, but any support would be much appreciated!
Thanks
Harry
Hello Harry,
This is because the Accessibility module was not initially listed and added to projects by default. To add the module to your project, you need to manually add it to Packages/manifest.json:
"com.unity.modules.accessibility": "1.0.0",
The module will be listed in the Package Manager and added by default to new projects starting with Unity 2023.2.0b8.
Please note that the current set of Accessibility APIs does not fully support switch access, as they primarily focus on mobile screen reader support (TalkBack and VoiceOver). However, we are actively working on improving and extending them.
I hope this helps! We are excited to see that our Accessibility APIs are already being used. 
All the best,
Bianca
2 Likes
Hi Bianca
Thanks very much for the speedy reply. That’s interesting and good to know. I can now access those classes successfully.
I’m now running this code to get my bearings:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Accessibility;
public class Test : MonoBehaviour
{
Color[] colors = { Color.red, Color.green, Color.blue };
[SerializeField]
GameObject canvas, buttonOne, buttonTwo;
public AccessibilityNode buttonOneNode, buttonTwoNode;
public AccessibilityHierarchy accessibilityHierarchy;
void Awake()
{
accessibilityHierarchy = new AccessibilityHierarchy();
buttonOneNode = accessibilityHierarchy.AddNode("button one", buttonOneNode);
buttonTwoNode = accessibilityHierarchy.AddNode("button two", buttonTwoNode);
Debug.Log(buttonOneNode.frame);
Debug.Log(buttonTwoNode.frame);
buttonOneNode.frame = buttonOne.GetComponent<RectTransform>().rect;
buttonTwoNode.frame = buttonTwo.GetComponent<RectTransform>().rect;
buttonOneNode.role = AccessibilityRole.Button;
buttonTwoNode.role = AccessibilityRole.Button;
foreach (var node in accessibilityHierarchy.rootNodes)
{
Debug.Log(node);
node.isActive = true;
}
Debug.Log(buttonOneNode.frame);
Debug.Log(buttonTwoNode.frame);
Debug.Log(buttonOneNode.role);
Debug.Log(buttonTwoNode.role);
accessibilityHierarchy.RefreshNodeFrames();
Debug.Log("active hierarchy: " + AssistiveSupport.activeHierarchy);
}
// Update is called once per frame
void Update()
{
}
}
But the active hierarchy is returning empty, and I can see no active hierarchies during runtime or editing:
So I suppose my follow up question is, what’s the intended workflow of adding nodes?
My hope re:switch access was that once AndroidOS knows the position of each interactable item it can then use that to scan between them, as it needs that information for TalkBack anyway. Is there any Switch Access functionality, is that down the line, or possible but yet to be tested?
Thanks
Harry
For cashing purposes, you can have multiple hierarchies at once (for example, one for each screen), but only one can be active at a time. Thus, once the hierarchy is created, it needs to be set as the active one using AssistiveSupport.activeHierarchy. This has to be done both initially and when the screen reader is activated (you can listen to this using AssistiveSupport.screenReaderStatusChanged) because AssistiveSupport.activeHierarchy is set to null when the screen reader is deactivated. Additionally, the active hierarchy should be set after calling WaitForEndOfFrame() like below:
IEnumerator SetAccessibilityHierarchy()
{
yield return new WaitForEndOfFrame();
AssistiveSupport.activeHierarchy = accessibilityHierarchy;
}
We plan to publish a sample project soon to make it easier to understand how the screen reader support APIs should be used and what the workflow should look like.
Regarding switch access: What you are saying sounds right. Switch access may work using the current set of Accessibility APIs. However, please note that we did not test it and cannot guarantee full functionality, as it is possible that additional configuration may be required beyond what our current APIs offer.
Hi Bianca.
Thanks again for getting back to me. Having done some testing I can get TalkBack going, but no luck on Switch Access. Do you have a vague idea of when that functionality may be coming, or any ideas on what else I might be able to try?
Thanks
Harry
Hi Harry,
I have quickly tested Switch Access on one of our internal Android sample projects. Basic functionality appears to work for accessibility nodes that are set to listen to the AccessibilityNode.selected callback. Note that this callback should not be triggered by the application itself, as it comes from the operating system based on the user’s actions (we are currently working on making it an event). Also, this callback is linked to AccessibilityNodeInfo.ACTION_CLICK, and there is currently no API available for AccessibilityNodeInfo.ACTION_LONG_CLICK.
Unfortunately, we do not have a date for when we will be able to fully support switch access.
Let me know if you have any further questions or concerns!
Hi Bianca
That’s really helpful, thanks.
After adding that, the blue box is appearing, but only in the top left corner. I have tried different means of anchoring to no avail. I can see that the width/height/x/y are being set on the .frame correctly from the logs. Here’s my code so far:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Accessibility;
public class Test : MonoBehaviour
{
Color[] colors = { Color.red, Color.green, Color.blue };
[SerializeField]
GameObject canvas, buttonOne, buttonTwo, buttonThree;
public AccessibilityNode buttonOneNode, buttonTwoNode, buttonThreeNode;
public AccessibilityHierarchy accessibilityHierarchy;
void Awake()
{
accessibilityHierarchy = new AccessibilityHierarchy();
buttonOneNode = accessibilityHierarchy.AddNode("button one", buttonOneNode);
buttonTwoNode = accessibilityHierarchy.AddNode("button two", buttonTwoNode);
buttonThreeNode = accessibilityHierarchy.AddNode("button three", buttonThreeNode);
Debug.Log(buttonOneNode.frame);
Debug.Log(buttonTwoNode.frame);
Debug.Log(buttonThreeNode.frame);
buttonOneNode.frameGetter = GetButtonOneFrame;
buttonTwoNode.frameGetter = GetButtonTwoFrame;
buttonThreeNode.frameGetter = GetButtonThreeFrame;
buttonOneNode.role = AccessibilityRole.Button;
buttonTwoNode.role = AccessibilityRole.Button;
buttonThreeNode.role = AccessibilityRole.Button;
foreach (var node in accessibilityHierarchy.rootNodes)
{
node.isActive = true;
}
buttonOneNode.selected = LogOne;
buttonTwoNode.selected = LogTwo;
buttonThreeNode.selected = LogThree;
AssistiveSupport.activeHierarchy = accessibilityHierarchy;
StartCoroutine(SetAccessibilityHierarchy());
AssistiveSupport.screenReaderStatusChanged += OnScreenReaderStatusChanged;
accessibilityHierarchy.RefreshNodeFrames();
}
bool LogOne()
{
Debug.Log("Log One");
Debug.Log(buttonOneNode.frame);
return true;
}
bool LogTwo()
{
Debug.Log("Log Two");
Debug.Log(buttonTwoNode.frame);
return true;
}
bool LogThree()
{
Debug.Log("Log Three");
Debug.Log(buttonThreeNode.frame);
return true;
}
void OnScreenReaderStatusChanged(bool isActive)
{
Debug.Log("Screen reader status changed: " + isActive);
StartCoroutine(SetAccessibilityHierarchy());
}
IEnumerator SetAccessibilityHierarchy()
{
yield return new WaitForEndOfFrame();
AssistiveSupport.activeHierarchy = accessibilityHierarchy;
Debug.Log("active hierarchy: " + AssistiveSupport.activeHierarchy);
buttonOneNode.frame = buttonOne.GetComponent<RectTransform>().rect;
buttonTwoNode.frame = buttonTwo.GetComponent<RectTransform>().rect;
buttonThreeNode.frame = buttonThree.GetComponent<RectTransform>().rect;
}
Rect GetButtonOneFrame()
{
Debug.Log("GetButtonOneFrame");
return buttonOne.GetComponent<RectTransform>().rect;
}
Rect GetButtonTwoFrame()
{
Debug.Log("GetButtonTwoFrame");
return buttonTwo.GetComponent<RectTransform>().rect;
}
Rect GetButtonThreeFrame()
{
Debug.Log("GetButtonThreeFrame");
return buttonThree.GetComponent<RectTransform>().rect;
}
}
What is used in your test project to set the .frame correctly?
Thanks again,
Harry
Hi Harry,
Make sure that the node frames are always up-to-date and that you applied the correct scaling from world coordinates to screen coordinates. We recommend using AccessibilityNode.frameGetter rather than AccessibilityNode.frame so that the frames are automatically recalculated. If AccessibilityNode.frame is not explicitly set, it will retrieve its value from AccessibilityNode.frameGetter.
node.frameGetter = () => GetFrame(gameObject.GetComponent<RectTransform>());
...
Rect GetFrame(RectTransform rectTransform)
{
var canvas = rectTransform.GetComponentInParent<Canvas>();
var camera = canvas.renderMode == RenderMode.ScreenSpaceOverlay ? null : canvas.worldCamera;
var worldCorners = new Vector3[4];
var screenCorners = new Vector3[4];
rectTransform.GetWorldCorners(worldCorners);
for (var i = 0; i < worldCorners.Length; i++)
{
screenCorners[i] = RectTransformUtility.WorldToScreenPoint(camera, worldCorners[i]);
}
GetMinMaxX(screenCorners, out var minX, out var maxX); // custom method that outputs the minimum and maximum y value in a vector
GetMinMaxY(screenCorners, out var minY, out var maxY); // custom method that outputs the minimum and maximum x value in a vector
return new Rect(minX, Screen.height - maxY, maxX - minX, maxY - minY);
}
AccessibilityHierarchy.RefreshNodeFrames is not necessarily needed. It is a convenience method that updates the AccessibilityNode.frame of all nodes in the accessibility hierarchy based on AccessibilityNode.frameGetter and then calls AssistiveSupport.notificationDispatcher.SendLayoutChanged.
Hope this helps!
Hi Bianca
That’s tremendously helpful. This is now working 
. There’s something slightly off about the frames on the Google pixel 6a, but I think that’s to do with the resolution reported by the phone. Thanks again, this is a real win for us.
Harry
Awesome! I am genuinely happy that it works and glad to have been of assistance.
For any further questions or feedback regarding our Accessibility APIs, please do not hesitate to post on our new Accessibility Forum.
Good luck with your project!
Hi Bianca
I’ve now moved on to iOS, using the same code as for Android, Switch Control automatically goes to the equivalent of point scan. Do you have any insights into how to get it to recognise the buttons and therefore be scannable? In this case, VoiceOver does not seem to recognise the buttons either.
The code is a little harder to show now it’s implemented with our WebGL support. But you can think of switches as an array of buttons. As previously stated this is working correctly on Android.
#else
// makes use of Unity accessibility API FOR iOS and Android
accessibilityHierarchy.Clear();
foreach (Switchable s in _switches){
Rect testRect = GetFrame(s.GetComponent<RectTransform>());
AccessibilityNode tempNode = null;
if (!s.gameObject.activeInHierarchy || s.disabled || accessibilityHierarchy.TryGetNodeAt(testRect.x, testRect.y, out tempNode)) continue;
AccessibilityNode node = accessibilityHierarchy.AddNode(s.gameObject.name, null);
node.isActive = true;
node.role = AccessibilityRole.Button;
node.frameGetter = () => GetFrame(s.GetComponent<RectTransform>());
node.selected += () => Log(s.gameObject.name, s.GetComponent<StandardAccessibleButton>());
Debug.Log("Added node: " + node.label);
}
accessibilityHierarchy.RefreshNodeFrames();
StartCoroutine(SetAccessibilityHierarchy());
AssistiveSupport.screenReaderStatusChanged += OnScreenReaderStatusChanged;
#endif
Thanks
Harry
Giving this a little bump
Hi Harry,
It took me a bit of time to look into this. Due to how the iOS support is implemented, the accessibility hierarchy is only applied if VoiceOver is active.
As previously mentioned, the current set of Accessibility APIs is designed to enable screen reader support. Switch access on Android is just an unintended side effect.
I have discussed switch access with my team, and we would greatly appreciate it if you could submit a feature request on our UI roadmap.
Regarding VoiceOver not recognising your buttons – unfortunately, I cannot determine the cause of this issue based on the information provided. However, if you could submit a bug report through the Unity Bug Reporter and include a project with the relevant code, we would be happy to investigate further.
1 Like
Hi Bianca
I’m running into an issue where-in the scan pattern used by the Switch Access App on a Samsung tablet is not adhering to the hierarchy displayed within Unity.
Samsung Tablet using Switch Access scan pattern
The hierarchy is -
25: “MuteSFXButton”
26: “ReduceSFXButton”
27: “IncreaseSFXButton”
28: “MaximumSFXButton”
…followed by the next row in the correct order.
They are all root nodes.
On the Google Pixel 6a however, these nodes are scanned in the correct order using Switch Access:
Samsung’s own Switch feature does not recognise any nodes at all.
Is this is a known issue, or is there some way around this?
Thanks again,
Harry
Hi Harry,
As stated previously, the current set of Accessibility APIs focuses on screen reader support; switch access is just an unintended side effect and might not always work as expected.
Does your issue reproduce with the screen reader as well?
Hi Bianca, it did indeed reproduce with the screen reader, in testing that I realised what I’d done. The Google Pixel 6a phone apparently does it’s own grouping for Switch Access that was obfuscating the underlying issue.
Thanks
Harry
Hi Harry,
The screen reader navigates the accessibility hierarchy in a Depth First Search (DFS) order, so you should be able to determine the navigation order through how you structure your hierarchy. If all nodes are root nodes, the screen reader will navigate them in their order.
If the screen reader does not respect this order, please submit a bug report through the Unity Bug Reporter so we can investigate the issue.
I hope this helps!
Hi Bianca. Thanks for that explanation. Thought you’d like to know the apps are now published on the Play Store, all three working seamlessly with Switch Access, Special Effect will be doing a press release for them soon.
https://play.google.com/store/apps/details?id=com.EyegazeGames.SnakesAndLadders
Thanks again
Harry
Hi Harry,
Thank you for letting me know. That’s fantastic news! I will share them with my team.
