SWE-Shepherd: Advancing PRMs for Reinforcing Code Agents
- ID: 34f94807
- 原文链接: https://arxiv.org/abs/2604.10493
- PDF: https://arxiv.org/pdf/2604.10493v1
- 作者: Mahir Labib Dihan, Md Ashrafur Rahman Khan
- 日期 / 版本: Submitted on 2026-04-12
- 分类: coding
- 来源类型: paper
- arXiv 分类: cs.SE
- arXiv 备注: Code is available at https://github.com/mahirlabibdihan/swe-shepherd
- 标签: coding-agent, process-reward-model, reinforcement-learning, software-engineering, prm, 2604-10493
- 质量评分: 4/5
- 抓取时间: 2026-07-27T12:26:04.051482+00:00
中文导读
论文围绕 Code Agent 的强化学习与过程奖励模型（PRM）展开摘要主题是 advancing PRMs for reinforcing code agents：通过更细粒度的过程反馈来引导软件工程 Agent，而不是只依赖最终答案成败，适合跟踪 coding-agent 训练与评估方法
为什么值得关注
SWE-Shepherd 将 PRM 引入代码 Agent 强化流程，关注过程级反馈如何改进软件工程任务
这篇内容值得放进 AAIF，是因为它围绕 Agent / Coding Agent 系统中的一个具体评测或训练问题展开；本页基于 arXiv 元数据、摘要与条目已有摘要整理，未补充摘要之外的实验细节。
关键信息
- 论文标题：SWE-Shepherd: Advancing PRMs for Reinforcing Code Agents
- 作者：Mahir Labib Dihan, Md Ashrafur Rahman Khan
- arXiv：https://arxiv.org/abs/2604.10493
- 发布时间 / 修订：Submitted on 2026-04-12
- arXiv 分类：cs.SE
- arXiv 备注：Code is available at https://github.com/mahirlabibdihan/swe-shepherd
- 关联标签：coding-agent, process-reward-model, reinforcement-learning, software-engineering, prm, 2604-10493
English Abstract
Automating real-world software engineering tasks remains challenging for large language model (LLM)-based agents due to the need for long-horizon reasoning over large, evolving codebases and making consistent decisions across interdependent actions. Existing approaches typically rely on static prompting strategies or handcrafted heuristics to select actions such as code editing, file navigation, and test execution, but they lack fine-grained feedback on intermediate decisions. This leads to inefficient exploration, error propagation, and brittle solution trajectories. To address this limitation, we propose SWE-Shepherd, a framework that introduces Process Reward Models (PRMs) to provide dense, step-level supervision for repository-level code agents. Using trajectories from SWE-Bench, we construct an action-level reward dataset and train a lightweight reward model on a base LLM to estimate the usefulness of intermediate actions. During inference, the PRM evaluates candidate actions and guides the agent toward higher-reward decisions without requiring full reinforcement learning. Experiments on SWE-Bench Verified demonstrate improved interaction efficiency and action quality, while also highlighting challenges in aligning intermediate rewards with final task success.
English Summary
Automating real-world software engineering tasks remains challenging for large language model (LLM)-based agents due to the need for long-horizon reasoning over large, evolving codebases and making consistent decisions across interdependent actions. Existing approaches typically rely on static prompting strategies or handcrafted heuristics to select actions such as code editing, file navigation, and test execution, but they lack fine-grained feedback on intermediate decisions. This leads to inefficient exploration, error propagation, and brittle solution trajectories. To address this limitation, we propose SWE-Shepherd, a framework that introduces Process Reward Models (PRMs) to provide dense, step-level supervision for repository-level code agents. Using trajectories from SWE-Bench, we construct an action-level reward dataset and train a lightweight reward model on a
Obsidian Notes
- 内容获取路径：优先尝试
opencli arxiv paper 2604.10493 -f json，本页使用返回的 arXiv 元数据与摘要回填。 - 中文导读与价值判断均锚定在条目已有摘要、论文摘要、作者、日期与分类信息上。
- 现代站点生成器按
content/{entry.id}.md查找内容页；本文件写入 canonical content 目录，而不是openclaw/content/。