Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair
Source: https://arxiv.org/abs/2607.24604
arXiv: 2607.24604
Authors: Xueping Gao, Jianwei Yang, Qiang Yang
Published: 2026-07-27
Categories: cs.CL, cs.AI
PDF: https://arxiv.org/pdf/2607.24604v1
Abstract (en)
Generate--test--revise loops are common in coding agents, but repetition alone provides no reliability guarantee. We study the gap between finding a correct patch and retaining, verifying, and submitting it. A sealed five-seed study over 30 HumanEval repairs produces 900 three-revision trajectories. Under forced revision, current correctness with current traces falls from 0.820 after one revision to 0.673 after two, although ever-correct rises to 0.847. Two common-state studies use 2,430 branches from identical frozen programs to remove post-treatment risk-set bias. In a prespecified 14B replication, stale traces harm 34/135 correct starts versus 4/135 with current traces, a 22.2-point increase (task-cluster 95\% CI $[8.9,37.0]$, exact Holm $p=0.0337$). A prospective 540-rollout policy eliminates observed correct-start harm but reduces wrong-start repair and fails its joint criterion. Repository experiments over 24 bugs and four coder stacks expose floor effects and component heterogeneity without Holm-significant effects. We therefore separate admission, preservation, grounded certification, competence, and liveness. We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation that binds verifier evidence to exact code states, preserves verified checkpoints, and emits auditable admission receipts. The implementation is an executable specification and conformance artifact, not evidence of improved repair competence or calibrated verifier dependence.
Summary (zh)
论文拆开 generate–test–revise：找到正确补丁与保留/验证/提交它不是一回事。30×HumanEval×五 seed×三轮修订共 900 轨迹显示，强制修订下 current-correct 从一轮后 0.820 掉到两轮后 0.673，而 ever-correct 升到 0.847——正确补丁曾被找到又丢掉。2430 条 common-state 分支中，stale trace 伤害 34/135 正确起点，当前 trace 仅 4/135。作者分离 admission/preservation/grounded certification/competence/liveness，给出绑定 verifier 证据到确切代码状态的 typed loop contract 参考实现（conformance 工件，不宣称提升修复能力本身）。
One-liner
多跑几轮修订不等于更可靠：过期 trace 会毁掉已正确起点，需要状态绑定证据与 typed revision contract。
Obsidian evidence
OpenClaw定时任务/论文流水线/2026-07-29-论文流水线.md- opencli:
arxiv paper 2607.24604 -f json - Run date: 2026-07-29