Software Delegation Contracts: Measuring Reviewability in AI Coding-Agent Work
- source_url: https://arxiv.org/abs/2606.17099
- source_type: paper
- platform: arxiv
- author: Vincent Schmalbach
- original_date: 2026-06-14
- added_date: 2026-07-21
- arxiv_id: 2606.17099
- arxiv_categories: cs.SE, cs.AI
- pdf_url: https://arxiv.org/pdf/2606.17099v1
- category: coding
- tags: coding-agent, reviewability, software-delegation, evaluation, arxiv
- quality_score: 4
摘要（中文）
论文把 AI coding agent 的交付物放到“可审查性”框架下评估：任务、权限、返回工作包与验收上下文共同构成 software delegation contract。作者在一个无依赖 TypeScript API 任务环境中设计 10 个任务、64 次 agent 执行，对比普通 issue prompt、显式契约、契约加证据包三种条件，用 pilot study 测量契约对审查与验收的影响。
Summary (English)
AI coding agents increasingly accept assigned software tasks, modify repositories under bounded authority, and return work packages for review. Prior work proposed the software delegation contract, covering the task, authority, returned work package, and acceptance context, as the unit of analysis for delegated coding work, but did not measure its effects. This paper reports a controlled pilot study of explicit delegation contracts for coding agents. We built a dependency-free TypeScript API task environment with seeded defects and documentation gaps, authored ten tasks across five families, and ran 64 agent executions across two model tiers under three conditions: a realistic issue-style prompt, an explicit delegation contract, and a contract with a required evidence bundle. Each run was scored with hidden acceptance tests, mutation checks, and scope analysis, then reviewed by three independent condition-blinded model-based reviewers using a fixed rubric, for 192 reviews. Explicit contracts did not improve objective task outcomes: all 64 runs passed hidden acceptance checks, with zero scope violations. They did improve reviewability. Evidence sufficiency improved in 22 of 30 paired comparisons and worsened in none (+0.83 on a 5-point scale, p < 0.0001, Cliff's delta = 0.66); reviewer ambiguity decreased (p = 0.035); changed-file lists, known-limitations sections, residual-risk sections, and reviewer checklists appeared mostly or only when demanded by the contract. Contracts cost +13% agent tokens and +38% wall-clock time, with larger effects for the weaker model tier. On these small tasks, delegation contracts bought reviewability rather than correctness.
One-liner
Coding agent 的落地价值不只在生成代码，而在交付是否能被人类低成本审查与验收。
原文 / 元数据抓取
Software Delegation Contracts: Measuring Reviewability in AI Coding-Agent Work
作者: Vincent Schmalbach
原文链接: https://arxiv.org/abs/2606.17099
PDF: https://arxiv.org/pdf/2606.17099v1
发布时间: 2026-06-14
更新时间: 2026-06-14
分类: cs.SE, cs.AI
AI coding agents increasingly accept assigned software tasks, modify repositories under bounded authority, and return work packages for review. Prior work proposed the software delegation contract, covering the task, authority, returned work package, and acceptance context, as the unit of analysis for delegated coding work, but did not measure its effects. This paper reports a controlled pilot study of explicit delegation contracts for coding agents. We built a dependency-free TypeScript API task environment with seeded defects and documentation gaps, authored ten tasks across five families, and ran 64 agent executions across two model tiers under three conditions: a realistic issue-style prompt, an explicit delegation contract, and a contract with a required evidence bundle. Each run was scored with hidden acceptance tests, mutation checks, and scope analysis, then reviewed by three independent condition-blinded model-based reviewers using a fixed rubric, for 192 reviews. Explicit contracts did not improve objective task outcomes: all 64 runs passed hidden acceptance checks, with zero scope violations. They did improve reviewability. Evidence sufficiency improved in 22 of 30 paired comparisons and worsened in none (+0.83 on a 5-point scale, p < 0.0001, Cliff's delta = 0.66); reviewer ambiguity decreased (p = 0.035); changed-file lists, known-limitations sections, residual-risk sections, and reviewer checklists appeared mostly or only when demanded by the contract. Contracts cost +13% agent tokens and +38% wall-clock time, with larger effects for the weaker model tier. On these small tasks, delegation contracts bought reviewability rather than correctness.
Obsidian intake evidence excerpt
该内容文件由 AAIF content-fetcher 根据 active/high-score entry 与 OpenCLI arXiv 元数据补齐。
- entry_id: 8865f6e2
- title: Software Delegation Contracts: Measuring Reviewability in AI Coding-Agent Work
- source: https://arxiv.org/abs/2606.17099
- existing_summary_zh: 论文提出软件委托契约来衡量 AI coding agent 产物的可审查性，关注人类如何界定检查和验收代理完成的软件修改收录理由：它把 coding agent 的价值从能否生成代码推进到交付是否可审查可委托，对团队落地代理编程流程有方法论价值