X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding
Source: https://arxiv.org/abs/2609.09166 · platform: arxiv · authors: Jaeduk Lee, Wan Choi · date: 2026-07-15
TL;DR（中文摘要）
端云协作推测解码（CoSD）通常要求设备端 SLM 与服务端 LLM 共享词表，否则残差 resampling 要来回传整段 token 分布，通信开销吃掉本地解码收益。X-CoSD 用 hybrid resampling 把残差切分成「设备侧公共词表段」+「服务端 LLM-only 段」，分布传输只在公共词表段发生；增强版 X-CoSD-E 让服务端只发替换候选+概率、由设备本地验证，传输量进一步下降。论文形式证明两个变体均无损保留服务端 LLM 输出分布，实验显示 token 生成显著加速且质量持平。「词表不匹配」不再是端云协同解码的硬约束。
Summary (English)
X-CoSD makes collaborative speculative decoding work across heterogeneous vocabularies: hybrid resampling confines distribution exchange to the shared-vocabulary segment, and X-CoSD-E sends only replacement candidates with probabilities for local device-side verification. Both variants provably preserve the server LLM distribution with significant speedup.
Abstract
This paper investigates collaborative speculative decoding (CoSD), a distributed large language model (LLM) inference framework in which an on-device small language model (SLM) drafts candidate tokens and a server LLM verifies them. Existing CoSD methods assume a shared vocabulary between the SLM and the LLM and incur substantial communication load because residual resampling requires token distribution exchange between the user device and the edge server. To address these limitations, we propose cross-vocabulary CoSD (X-CoSD), a lossless and communication-efficient CoSD framework for heterogeneous vocabularies. X-CoSD splits residual resampling into a device-side common-vocabulary segment and a server-side LLM-only segment, so distribution exchange only occurs over the shared segment. An enhanced variant, X-CoSD-E, further cuts uplink traffic by having the server send only replacement candidates with probabilities, verified locally on device. Both variants provably preserve the server LLM output distribution, and experiments show significant token-generation speedup with matched generation quality.
基本信息
| 项 | 值 | |------|------| | 论文 ID | 2609.09166 | | 发表 | 2026-07-15 | | 作者 | Jaeduk Lee, Wan Choi | | Subjects | cs.CL; cs.DC; cs.LG | | abs | <https://arxiv.org/abs/2609.09166> | | PDF | <https://arxiv.org/pdf/2609.09166v1> |
入库依据（同日 digest 交叉验证）
论文流水线 2026-09-16 入选；abs 页核实标题/作者/提交日期（opencli 429 后 fallback）。