Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement support for Android TalkBack #17704

Merged
merged 38 commits into from
Jan 20, 2025

Conversation

IsaMorphic
Copy link
Contributor

What does the pull request do?

This pull request adds a much-needed implementation of UI automation / screen reader accessibility features to the Android implementation of Avalonia. It does so by implementing Android's ExploreByTouchHelper built-in as a compatibility shim between Avalonia's own AutomationPeer API and Android's AccessibilityNodeInfo API.

What is the current behavior?

The current behavior of Avalonia on Android is to essentially operate as a dynamically rendered raster SurfaceView which is completely opaque to screen readers such as Android's TalkBack or HumanWare's KeySoft. This is completely unsuitable for users who have visual and/or cognitive impairments and need a screen reader to help them understand what's on the screen.

What is the updated/expected behavior with this PR?

The new behavior that this PR implements is a fully functional prototype integration between Avalonia and Android's various accessibility APIs. This integration includes support for all relevant specialized "providers" within the AutomationPeer API and creates a performant, responsive, and verbose user experience for visually impaired users of Avalonia apps on Android.
To test, enable TalkBack on your Android mobile device and run the ControlCatalog sample. You will observe that the "explore by touch" feature now responds appropriately to user input and allows them to hover over interactive elements like buttons and text fields in order to focus them and have their attached labels read aloud.

How was the solution implemented (if it's not obvious)?

The solution was implemented by implementing a specialized AccessibilityDelegateCompat via Android's ExploreByTouch abstract. This delegate acts as a plug-in for Avalonia's existing automation features into Android's accessibility APIs designed for custom controls.

Checklist

Breaking changes

This PR only modifies internal Avalonia.Android functionality and as such does not expose any new APIs or make changes to the existing API surface.

Fixed issues

@IsaMorphic IsaMorphic force-pushed the feature/android-touch branch from cf694f6 to 89c4fff Compare December 5, 2024 23:18
@IsaMorphic IsaMorphic force-pushed the feature/android-touch branch from 89c4fff to d64e513 Compare December 5, 2024 23:29
@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053729-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@cla-avalonia
Copy link
Collaborator

cla-avalonia commented Dec 6, 2024

  • All contributors have signed the CLA.

@IsaMorphic
Copy link
Contributor Author

@cla-avalonia agree

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053735-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053745-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053747-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053749-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053751-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053863-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053865-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053891-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0053983-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054029-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054087-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054155-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054190-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054200-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054211-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.3.999-cibuild0054295-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@maxkatz6
Copy link
Member

Thank you!

@maxkatz6 maxkatz6 added this pull request to the merge queue Jan 20, 2025
@IsaMorphic
Copy link
Contributor Author

You betcha! :3

Merged via the queue into AvaloniaUI:master with commit 57c8595 Jan 20, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Accessibility support for targeted platforms needs to be considered
6 participants