2026.09.02 11:20 ~ 12:00 Otter

Entering the era of AI agents for device operation: challenges in Android UI/E2E testing leveraging mobile-mcp

Development with AI 日本語 Interpretation available

In Android app development, automating areas covered by unit tests and AI generation of test code are very important approaches. However, in transitional projects without sufficient test code or in final integration phases, a gritty manual process of "running physical devices or emulators to perform End-to-End (E2E) verification" is still required. Of course, for routine regression testing, comprehensive automation using existing E2E testing tools is effective. However, what is truly needed on the development front is likely "lightweight verification feedback tailored to developers, allowing quick and minimal verification of specific modified features or screens." To solve this challenge, solutions that leverage MCP (Model Context Protocol) to entrust device operations to AI for autonomously driven UI/E2E testing have recently gathered attention. However, simply instructing "test this screen" presents high hurdles in real operation, such as the AI wandering into entirely unintended screens or failing to reproduce prerequisite states, causing rework. In this session, based on my actual experience, I will explain the challenges I faced when putting these "autonomous device operations by AI" into practical production use, along with four specific approaches used to overcome them. Four approaches explained in this session: 1. How to navigate to (identify) the target screen Mechanism for screen identification using a "reference file (map file)" defining screen names and launch methods Methods for monitoring and identifying currently displayed Activity information using adb How to define expected UI objects and text strings in reference files 2. How to reproduce the target state A framework for generating and executing app operation commands via adb so the AI can set up prerequisites without getting lost 3. How to verify "test success" Methods for defining and verifying whether "expected states" are met after screen transitions and operations Evidence recording through screenshot capture and history management 4. How to maintain reference files A mechanism to automatically update and synchronize reference files based on interaction details with the AI Going beyond mere theory, I will deliver practical know-how for letting AI agents operate Android, learned through actual production operations, complete with concrete examples. (Translated by the DroidKaigi Committee)

Intended audience

Managers and developers who want to overwhelmingly reduce manual UI/E2E verification costs Those interested in advanced test automation architectures leveraging MCP and AI agents

Sessions in the same time slot

View all sessions