Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation
- ID: 052f7963
- 原文链接: https://arxiv.org/abs/2608.30996
- PDF: https://arxiv.org/pdf/2608.30996v1
- 作者: Atta Ul Asad, Ahsan Bilal, Muhammad Ali, Muhammad Haseeb, Dean F. Hougen
- 日期: 2026-08-31
- 抓取时间: 2026-09-02T15:31:00Z
- 分类: infra
- 来源类型: paper
- 语言: zh
- 标签: kv-cache, quantization, rag, faithfulness, eval
- 质量评分: 4/5
中文导读
RAG 离线缓存把检索文档的 KV 预计算存储，量化能进一步省存储，但没人系统回答过：压缩是否伤害忠实度？这篇第一次把准确率和忠实度拆开审计，对 Qwen2.5-7B-Instruct 在 RGB 和 HotpotQA 上用 INT8/INT4 测，幻觉检测器NLI 蕴含LLM judge 三个口径并行结论是：INT8 接近无损；INT4 即使在仍然事实正确的答案里，超过 90% 的忠实度变化是负向的accuracy 指标对这个退化完全失明，且检索噪声越大chunk 越多伤害越大所有做 KV cache 量化上线的人都该把"忠实度审计"加入发布门禁
一句话点评
RAG 离线缓存把检索文档的 KV 预计算存储，量化能进一步省存储，但没人系统回答过：压缩是否伤害忠实度？
English Abstract / Summary
The paper audits offline KV-cache quantization for RAG systems and shows faithfulness and accuracy can diverge. On Qwen2.5-7B-Instruct with RGB and HotpotQA, INT8 is near-lossless but INT4 causes over 90% of faithfulness changes to be negative even among answers that remain factually correct; standard accuracy metrics are blind to this regression and the damage grows with retrieval noise and chunk count. The authors recommend pairing quantization releases with faithfulness audits.
Obsidian Notes
- 由
daily-intake-evening2026-09-02 cron 从当日 Obsidian 摘要（论文流水线 / AK-RSS / ClawFeed / X 书签消化）发现并入库存量阶段。 - 中文导读与判断均锚定在条目已有摘要、源页面正文、作者、日期与分类信息；未补充源页面之外的实验细节。