WAR: Workload-Aware Rollouts for Synchronous Agentic Reinforcement Learning
- source_url: https://arxiv.org/abs/2607.17299
- source_type: paper
- platform: arxiv
- author: Ryan Xu, Atlas Zhao, David Bao, Frank Du
- original_date: 2026-07-19
- added_date: 2026-07-22
- category: infra
- tags: agentic-rl, rollout, kv-cache, suffix-decoding, systems, arxiv
- quality_score: 4
- arxiv_id: 2607.17299
- arxiv_categories: cs.LG, cs.AI, cs.CL, cs.OS
- pdf_url: https://arxiv.org/pdf/2607.17299v1
摘要（中文）
把同步 agentic RL 的长轨迹 rollout 当作系统瓶颈：低负载用无额外 draft 模型的 SuffixDecoding 复用已完成轨迹后缀模式；高负载转向缓存感知全局调度，按 cache locality、轨迹进度与负载放置请求，减少 KV 重算与负载不均。摘要称长上下文 agentic rollout 吞吐低负载约 1.4×、高负载最高约 1.6×，不改底层 RL 算法。价值在于按 runtime load 切换解码级与系统级优化，而不是一套策略打天下。
Summary (English)
Long-horizon rollout generation is the dominant systems bottleneck in agentic RL. WAR is workload-aware: under low load it uses model-free SuffixDecoding that reuses suffix patterns from completed trajectories as speculative drafts without an extra draft model; under high load it shifts to cache-aware global scheduling across rollout replicas based on cache locality, trajectory progress, and server load. Combining decoding-level suffix reuse with system-level scheduling improves long-context agentic rollout throughput by 1.4× under low load and up to 1.6× under high load without changing the RL algorithm.
One-liner
Agentic RL 的 rollout 瓶颈要按负载切换：低载后缀推测，高载缓存感知调度。
Source body / metadata
arXiv abstract grounded intake for 2607.17299. PDF: https://arxiv.org/pdf/2607.17299v1
Long-horizon rollout generation is the dominant systems bottleneck in agentic RL. WAR is workload-aware: under low load it uses model-free SuffixDecoding that reuses suffix patterns from completed trajectories as speculative drafts without an extra draft model; under high load it shifts to cache-aware global scheduling across rollout replicas based on cache locality, trajectory progress, and server load. Combining decoding-level suffix reuse with system-level scheduling improves long-context agentic rollout throughput by 1.4× under low load and up to 1.6× under high load without changing the RL algorithm.
Obsidian evidence
- local_note: OpenClaw定时任务/论文流水线/2026-07-22-论文流水线.md
- intake_run: daily-intake-evening 2026-07-22