Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation
- arXiv: 2608.20316
- Authors: Adam Fisch, Shubhendu Trivedi, Fantine Huot, William W. Cohen, Michael Kaisers, Mirella Lapata, Kate Larson, Jacob Eisenstein
- Published: 2026-08-20; categories: cs.AI
Abstract
Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing queries to the specialist who can answer most effectively at the lowest cost. Routing requires estimating each specialist's expected return, but this value estimation has a cost. Cheap estimators (e.g., embedding-based predictors) are fast but noisy, while accurate estimators (e.g., fine-tuned models with access to retrieval results or partial reasoning traces) are expensive. We formalize this tradeoff as an instance of Pandora's Box, the classical problem of optimal search with costly inspection. Under a Gaussian signal model, the resulting policies have closed-form value-of-information expressions that determine, for each specialist and input, whether refining the value estimate is worth its cost. We call the centralized policy Pandora's Router. We extend this to a decentralized setting, Pandora's Bidder, where specialists independently decide whether to invest in self-assessment before accepting an offered price to claim a query. Experiments across three domains—a standard multi-LLM benchmark, retrieval-augmented specialists, and LLMs with variable inference-time reasoning—show that Pandora's Router matches the routing quality of exhaustive estimation, while querying the expensive estimator far less often. In the decentralized setting, value-of-information reasoning improves allocative efficiency when competing estimates are accurate; when competing estimates are noisy, however, it can increase the strategic specialist's utility at the expense of others.
为什么值得读（AAIF 扫描）
把多模型系统里的查询路由形式化为经典的 Pandora's Box 代价搜索问题：便宜的价值估计器（嵌入预测器）快但有噪声，昂贵的估计器（能看检索结果/部分推理轨迹的微调模型）准但贵。在高斯信号模型下推导出闭式 value-of-information 表达式，逐 specialist、逐输入判断「再花一次精化估值是否值得」——中心化策略叫 Pandora's Router，去中心化版本 Pandora's Bidder 让 specialist 自己决定是否投资自评估再按报价竞标查询。
实验覆盖三组场景：多 LLM 基准路由、RAG specialist、可变推理时计算的 LLM。核心结果：Pandora's Router 用远少于穷尽估计的昂贵估值调用，达到与穷尽估计相当的路由质量。去中心化设定下有个值得注意的反直觉发现：估计噪声大时，VOI 推理会放大策略型 specialist 的收益、牺牲其他人。
做 router / 模型选择系统时，这篇给出了可计算的「何时值得多花一次估值调用」判据，适合直接搬进成本感知路由设计。
Source: https://arxiv.org/abs/2608.20316
Captured: 2026-08-22 (AAIF content-fetcher)