Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility

ID: a0dbad6a
原文链接: https://arxiv.org/abs/2608.04001
PDF: https://arxiv.org/pdf/2608.04001v1
作者: Mohsen Hariri, Weicong Chen, Nahal Shahini, Vikash Singh, Kai Ye, Amirhossein Samandar, Debargha Ganguly, Sreehari Sankar, Yanyan Zhang, Shouren Wang, Jerry Peng, Biyao Zhang, Michael Hinczewski, Vipin Chaudhary
日期: 2026-08-04
更新: 2026-08-04
分类: models
来源类型: paper
标签: test-time-scaling, reasoning-models, evaluation, reproducibility
质量评分: 5/5
抓取时间: 2026-08-06T04:19:06Z

中文导读

这篇综述把 reasoning LLM 的 test-time scaling 拆成单轨迹延长多候选投票或验证以及未完成状态搜索等不同推理制度，强调它们在统计结构算力记账和失败模式上不可互换；作者提出统一符号和报告清单，用于减少评测与复现实验中的口径混乱

为什么值得关注

Test-time scaling 需要按推理制度算力口径和失败模式分开评测

关键信息

论文标题：Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
作者：Mohsen Hariri, Weicong Chen, Nahal Shahini, Vikash Singh, Kai Ye, Amirhossein Samandar, Debargha Ganguly, Sreehari Sankar, Yanyan Zhang, Shouren Wang, Jerry Peng, Biyao Zhang, Michael Hinczewski, Vipin Chaudhary
arXiv：https://arxiv.org/abs/2608.04001
发布时间：2026-08-04
arXiv 分类：cs.LG, cs.AI
关联标签：test-time-scaling, reasoning-models, evaluation, reproducibility

English Abstract

Large language models can solve substantially harder reasoning problems with more inference-time compute. The term "test-time scaling," however, now covers diverse inference algorithms that extend deliberation along a single trajectory, sample completed candidates and aggregate them through voting or verification, or search over unfinished partial states. These algorithms differ in their statistical structure, compute accounting, and failure modes. Treating these procedures as interchangeable under a single scalar "budget," or reporting accuracy without the inference protocol that produced it, makes results difficult to compare across studies. We develop a systematic account of test-time scaling along three axes. First, we formalize test-time scaling as budgeted inference over the implicit prefix tree of an autoregressive model and distinguish three structural regimes: single-trajectory sequential scaling, leaf-level scaling with terminal reduction, and prefix-level scaling. Second, we treat the evaluated object as the entire inference system and develop evaluation principles that separate end-to-end system performance from candidate-bank diagnostics. We introduce an evaluation profile whose coordinates and simple functionals recover or bound common repeated-sampling metrics, and prescribe protocol-matched reporting of compute and uncertainty. Third, we specify reproducibility requirements for inference protocols, distinguishing exact replay from distributional reproducibility and identifying the artifacts needed to support each. We also organize the open-weight reasoning ecosystem by model-side and interface mechanisms, apply these principles to broad-knowledge, symbolic-reasoning, and competition-mathematics benchmarks, and assemble over 2 billion full reasoning traces for release with progressively richer verifier and token-level signals.

English Summary

Obsidian Notes

内容由 opencli arxiv paper 拉取 arXiv 元数据与摘要生成。
中文导读与价值判断均锚定在条目已有摘要、论文摘要、作者、日期与分类信息上；未补充论文摘要之外的实验细节。