2026.09.02 16:20 ~ 17:00 Panda

Real-world experience in automating full-screen TalkBack support for government apps through designer collaboration and AI agents

UI・UX・Design 日本語 Interpretation available

I am in charge of government app development and implementing WCAG 2.2 A/AA compliant TalkBack support for all components and all screens. The main challenges are scale and collaboration. Currently, correctly implementing `contentDescription`, semantics roles, focus management, and double-read prevention for 35 design system components and multiple feature screens requires a deep understanding of the Compose semantics API and a massive amount of effort. Furthermore, the initial cost of organizing with designers regarding "which speech output to assign to which component" cannot be ignored. As a solution, we built a 3-step pipeline combining Figma MCP, Claude Code custom skills, and TalkBack verification, integrating the workflow across design and implementation. On the design side, AI automatically reads the Figma repository structure and applies the initial draft of accessibility annotations, establishing a collaborative workflow where designers review and fix them. On the implementation side, a custom skill extracts the finalized Figma annotations, converts them into Markdown classification tables, and an AI agent generates implementation code for the Compose semantics API. It supports code generation not only on a screen basis but also on a component basis. We also prepared a dedicated TalkBack verification app for general-purpose components, allowing granular TalkBack testing on actual devices and speech output reviews by designers. In this session, I will also openly share bugs found during testing: A "double-reading bug" where both outer contentDescription and inner Text are read aloud, and an anti-pattern where attaching focusable() to liveRegion causes double-reading with an external keyboard—all of which were discovered for the first time during actual device TalkBack verification. By attending this session, you will find concrete answers to the question "Where should I start with TalkBack?" by learning how to build an automation pipeline from designer collaboration workflows to Compose implementation and device verification. (Translated by the DroidKaigi Committee)

Intended audience

Engineers and designers who feel "it's time to take TalkBack support seriously" Teams facing challenges with a11y collaboration workflows

Sessions in the same time slot

View all sessions