TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning
- ID: 3ff04675
- 原文链接: https://arxiv.org/abs/2608.04007
- PDF: https://arxiv.org/pdf/2608.04007v1
- 作者: Changle Qu, Sunhao Dai, Hengyi Cai, Yuqi Zhou, Xinran Chen, Simon, Jun Xu
- 日期: 2026-08-04
- 更新: 2026-08-04
- 分类: agents
- 来源类型: paper
- 标签: tool-use, reasoning, self-distillation, reinforcement-learning
- 质量评分: 4/5
- 抓取时间: 2026-08-06T04:19:06Z
中文导读
TurnSight 面向 Tool-Integrated Reasoning 的长程工具交互信用分配问题，提出 turn-level hindsight self-distillation：教师分支使用同一轨迹的后续状态作为 hindsight context，为每一轮工具调用提供更密集监督，而不是只依赖整条轨迹成败
为什么值得关注
工具型推理 Agent 的训练可以把未来轨迹反馈下沉到每一轮工具调用
TurnSight 面向 Tool-Integrated Reasoning 的长程工具交互信用分配问题，提出 turn-level hindsight self-distillation：教师分支使用同一轨迹的后续状态作为 hindsight context，为每一轮工具调用提供更密集监督，而不是只依赖整条轨迹成败
关键信息
- 论文标题：TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning
- 作者：Changle Qu, Sunhao Dai, Hengyi Cai, Yuqi Zhou, Xinran Chen, Simon, Jun Xu
- arXiv：https://arxiv.org/abs/2608.04007
- 发布时间：2026-08-04
- arXiv 分类：cs.CL, cs.AI
- 关联标签：tool-use, reasoning, self-distillation, reinforcement-learning
English Abstract
Tool-Integrated Reasoning (TIR) enables LLMs to solve complex tasks through iterative tool interactions. However, existing reinforcement learning methods often rely on trajectory-level supervision, limiting fine-grained credit assignment in long-horizon TIR scenarios. On-policy self-distillation offers denser signals through teacher branches with privileged context, but existing approaches typically derive such context from ground-truth answers or retrieved skills, which may not reflect the states actually visited by the agent. Moreover, token-level supervision fails to capture the turn-level structure of tool interactions. To address this, we propose TurnSight, a turn-level hindsight self-distillation framework that derives supervision directly from execution-conditioned hindsight. It then constructs multiple hindsight views with different lookahead horizons and selects reliable supervision through cross-horizon directional agreement. Finally, the selected hindsight signal is normalized across sibling rollouts and used to adaptively modulate RL advantages while preserving their original optimization direction. Extensive experiments on three benchmarks demonstrate the effectiveness of TurnSight. Our codes are available at https://github.com/quchangle1/TurnSight.
English Summary
Tool-Integrated Reasoning (TIR) enables LLMs to solve complex tasks through iterative tool interactions. However, existing reinforcement learning methods often rely on trajectory-level supervision, limiting fine-grained credit assignment in long-horizon TIR scenarios. On-policy self-distillation offers denser signals through teacher branches with privileged context, but existing approaches typically derive such context from ground-truth answers or retrieved skills, which may not reflect the states actually visited by the agent. Moreover, token-level supervision fails to capture the turn-level structure of tool interactions. To address this, we propose TurnSight, a turn-level hindsight self-distillation framework that derives supervision directly from execution-conditioned hindsight....
Obsidian Notes
- 内容由
opencli arxiv paper拉取 arXiv 元数据与摘要生成。 - 中文导读与价值判断均锚定在条目已有摘要、论文摘要、作者、日期与分类信息上；未补充论文摘要之外的实验细节。