Improving Proficiency and Efficiency of Android GUI Agents via Self-Generating Tool Actions (DroidTool)
Source: https://arxiv.org/abs/2609.06792 · platform: arxiv · authors: Juyong Lee, Woogyeol Jin, Kimin Lee · date: 2026-09-06
TL;DR（中文摘要）
Android GUI agent 的混合动作空间（GUI 动作 + 直接调应用 API/数据库的工具动作）一直受困于写工具成本太高，开源方案基本只用 GUI。DroidTool 把工具生成交给 agentic workflow：proposal → 实现 → 测试生成与执行 → repair 四阶段，并用关联测试跨多个工具构造前置条件而非单点测试。在 AndroidWorld、B-MoCA、MobileSafetyBench 三个基准上，带自生成工具的 agent 对纯 GUI 基线平均提升任务成功率并减少约 20% 交互步数。对 Android GUI agent 工程，「写工具」与「用工具」共用一套 agentic 流程是关键判断。
Summary (English)
DroidTool augments Android GUI agents with self-generated tool actions via an agentic workflow (proposal, implementation, test generation/execution, repair), using relational tests across tools instead of isolated unit tests. Across AndroidWorld, B-MoCA, and MobileSafetyBench, hybrid GUI+tool agents improve task success over GUI-only baselines while reducing interaction steps.
Abstract
Android agents using a hybrid action space that combines GUI actions and tool actions (e.g., accessing application data via APIs) remain largely underexplored, mainly due to the excessive effort required to create tools. To address this gap, we introduce DroidTool, a framework for augmenting the agents with self-generated tools, which are realized as Python functions operating on application states (e.g., a database). To create tools with minimal human labor, DroidTool employs an agentic workflow featuring stages: proposal, implementation, test generation and execution, and repair. Notably, when testing the created tools for verification, it constructs relational tests across relevant tools rather than testing each tool in isolation. Evaluation across AndroidWorld, B-MoCA, and MobileSafetyBench shows that agents equipped with DroidTool-generated tools consistently improve task success rates over GUI-only baselines while reducing interaction steps.
基本信息
| 项 | 值 | |------|------| | 论文 ID | 2609.06792 | | 发表 | 2026-09-06 | | 作者 | Juyong Lee, Woogyeol Jin, Kimin Lee | | abs | <https://arxiv.org/abs/2609.06792> | | PDF | <https://arxiv.org/pdf/2609.06792v1> |
入库依据（同日 digest 交叉验证）
论文流水线 2026-09-16 首位入选：混合动作 agent 平均比纯 GUI 基线高 4.47p、交互次数少 20.05%；代码未给链接。