UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge

端侧 LLM 推理把稀疏化叠在低位量化上，但量化缩小了权重负载稀疏元数据却不按比例下降index 流量与非零提取成为 SpMM 新瓶颈论文提出 Payload-to-Metadata Ratio（PMR）度量，UnionSparse 用 Index-Efficient Bitmap Encoding（IE-BME）+ 低比特共享内存并行解码 SpMM kernel（LSPD）组合：W4A430-70% 稀疏度下超 FlashLLM/SpInfer 2.30x/1.43x，超 CUTLASS/cuBLAS Tensor Core 1.56x/3.46xESWEEK 2026 期刊轨（IEEE TCAD）录用

UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge

ID: 6a46c181
原文链接: https://arxiv.org/abs/2608.09291
PDF: https://arxiv.org/pdf/2608.09291v1
作者: Tianhao Jiang, Hang Gu, Teng Wang, Qianyu Cheng, ZhenDong Zheng, Cheng Tang, Qiyue Su, Wenqi Lou, Lei Gong, Chao Wang, Xi Li, Xuehai Zhou
日期: 2026-08-10
更新: N/A
分类: infra
来源类型: paper
标签: sparsity, quantization, edge-inference, spmm, esweek, llm
质量评分: 4/5
抓取时间: 2026-08-17T23:51:34+08:00

中文导读

端侧 LLM 推理把稀疏化叠在低位量化上，但量化缩小了权重负载、稀疏元数据却不按比例下降——index 流量与非零提取成为 SpMM 新瓶颈。论文提出 Payload-to-Metadata Ratio（PMR）度量，UnionSparse 用 Index-Efficient Bitmap Encoding（IE-BME）+ 低比特共享内存并行解码 SpMM kernel（LSPD）组合：W4A4、30-70% 稀疏度下超 FlashLLM/SpInfer 2.30x/1.43x，超 CUTLASS/cuBLAS Tensor Core 1.56x/3.46x。ESWEEK 2026 期刊轨（IEEE TCAD）录用。

为什么值得关注

量化×稀疏的下一块短板是 index 带宽：PMR 度量 + IE-BME 编码把端侧 SpMM 提速至 2.3 倍。

收录理由：指出量化后稀疏元数据成为新瓶颈并提出 PMR 度量，对移动端 SpMM index 带宽问题有直接映射

Abstract

Edge LLM inference combines sparsity and low-bit quantization to meet device memory, latency, and power limits. Yet quantization shrinks weight payloads without proportionally reducing sparse metadata, so index traffic and nonzero extraction become critical SpMM bottlenecks. We introduce the Payload-to-Metadata Ratio (PMR) and show that improving PMR raises effective compute intensity in decoding. We present UnionSparse, an index-efficient framework that combines Index-Efficient Bitmap Encoding (IE-BME) with a SpMM kernel using Low-Bit Shared-Memory Parallel Decoding (LSPD). IE-BME amortizes metadata and aligns sparse traversal with fragment assembly, while LSPD improves small-batch execution. Under W4A4 quantization and 30%--70% sparsity, UnionSparse outperforms FlashLLM and SpInfer by 2.30x and 1.43x, and CUTLASS and cuBLAS Tensor Core by 1.56x and 3.46x, respectively. These results establish payload-extraction efficiency as a first-order concern for low-bit sparse inference on edge GPUs. Source code is available at: https://github.com/Victor-Alen/UnionSparse.

元数据

arXiv ID: 2608.09291
主分类: cs.DC
分类: cs.DC
评论: 14 pages, 19 figures. Accepted via the ESWEEK 2026 Journal Track for publication in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD)

Obsidian 证据：OpenClaw定时任务/论文流水线/2026-08-17-论文流水线.md（2026-08-17 周度回顾）；元数据抓取自 opencli arxiv paper 2608.09291（2026-08-17）。