Harness Engineering for Agentic AI Coding Tools: An Exploratory Study

ID: 1a35d706
原文链接: https://arxiv.org/abs/2602.14690
PDF: https://arxiv.org/pdf/2602.14690v5
作者: Matthias Galster, Seyedmoein Mohsenimofidi, Jai Lal Lulla, Muhammad Auwal Abubakar, Christoph Treude, Sebastian Baltes
日期: 2026-02-16
更新: 2026-06-30
分类: coding
来源类型: paper
标签: harness-engineering, coding-agent, agents-md, configuration, empirical-study
质量评分: 4/5
抓取时间: 2026-07-30T04:18:47Z

中文导读

对 2853 个 GitHub 仓库的 agentic AI 编程工具（Claude Code / Copilot / Cursor / Gemini / Codex）配置机制做了实证普查识别出 8 种配置机制（静态上下文可执行脚本外部集成）核心发现：Context Files 占主导地位，AGENTS.md 成为跨工具互操标准；极少仓库用 Skills/Subagents 等高级机制，且 Skills 多为静态指令而非可执行脚本；各工具已形成差异化配置实践，Claude Code 用户机制覆盖最广为 coding agent 配置策略建立了经验基线

为什么值得关注

2853 仓库实证：AGENTS.md 成跨工具配置互操标准，但 Skills/Subagents 采用率极低

arXiv comment: 10 pages, 7 figures, 3 tables, based on "Configuring Agentic AI Coding Tools: An Exploratory Study" published in Proceedings of the 3rd ACM/IEEE International Conference on AI-powered Software (AIware 2026)

关键信息

论文标题：Harness Engineering for Agentic AI Coding Tools: An Exploratory Study
作者：Matthias Galster, Seyedmoein Mohsenimofidi, Jai Lal Lulla, Muhammad Auwal Abubakar, Christoph Treude, Sebastian Baltes
arXiv：https://arxiv.org/abs/2602.14690
发布时间：2026-02-16
arXiv 分类：cs.SE
关联标签：harness-engineering, coding-agent, agents-md, configuration, empirical-study

English Abstract

Agentic AI coding tools increasingly automate software development tasks. Developers can configure these tools through versioned repository-level artifacts such as Markdown and JSON files. We present a systematic analysis of configuration mechanisms for agentic AI coding tools, covering Claude Code, GitHub Copilot, Cursor, Gemini, and Codex. We identify eight configuration mechanisms spanning from static context to executable and external integrations and, in an empirical study of 2,853 GitHub repositories, examine whether and how they are adopted, with a detailed analysis of Context Files, Skills, and Subagents. First, Context Files dominate the configuration landscape and are often the sole mechanism in a repository, with AGENTS$.$md emerging as an interoperable standard across tools. Second, few repositories adopt advanced mechanisms such as Skills and Subagents. Skills predominantly rely on static instructions rather than executable scripts. Third, distinct configuration practices are forming around different tools, with Claude Code users employing the broadest range of mechanisms. These findings establish an empirical baseline for understanding how developers configure agentic tools, suggest that AGENTS$.$md serves as a natural starting point, and motivate longitudinal and experimental research on how configuration strategies evolve and affect agent performance.

English Summary

A systematic empirical analysis of configuration mechanisms for agentic AI coding tools (Claude Code, GitHub Copilot, Cursor, Gemini, Codex) across 2,853 GitHub repositories. The authors identify eight configuration mechanisms spanning static context, executable scripts, and external integrations. Key findings: (1) Context Files dominate and AGENTS.md emerges as an interoperable standard; (2) few repos adopt advanced mechanisms like Skills and Subagents, and Skills mostly rely on static instructions rather than executable scripts; (3) distinct configuration practices are forming around each tool, with Claude Code users employing the broadest range. Establishes an empirical baseline for how developers configure agentic coding tools.

Obsidian Notes

内容由 opencli arxiv paper 拉取 arXiv 元数据与摘要生成。
中文导读与价值判断均锚定在条目已有摘要、论文摘要、作者、日期与分类信息上；未补充论文摘要之外的实验细节。