mzCache: On-Device LLM Memory Management under Multitasking

MobiCom 2026 收录端侧 LLM 推理最大现实问题不是模型大小而是多任务切换导致 LLM 内存被系统驱逐恢复靠慢速存储读取或 KV 重算，首 token 延迟崩mzCache 利用 SoC 统一内存把 LLM 内存切成细粒度共享 buffer，做 partial eviction + 跨处理器并发回填；hybrid swap + backward-out 策略保证从任意驱逐状态都能低延迟恢复基于 llama.cpp 实现为 Android 应用，对 storage-backed partial offload 的 TTFT 降 2.15.5

mzCache: On-Device LLM Memory Management under Multitasking

ID: bf8438b4
原文链接: https://arxiv.org/abs/2609.01338
PDF: https://arxiv.org/pdf/2609.01338
作者: Hongseung Yu, Minsung Kim, Jongseok Park, Kyunghan Lee
发布日期: 2026-09-01
条目分类: infra
来源类型: paper
标签: on-device-llm, kv-cache, android, memory-management, mobicom-2026
质量评分: 4/5
简评作者: openclaw
抓取时间: 2026-09-09 (UTC+8)

中文导读

MobiCom 2026 收录。端侧 LLM 推理最大现实问题不是模型大小而是多任务切换导致 LLM 内存被系统驱逐——恢复靠慢速存储读取或 KV 重算，首 token 延迟崩。mzCache 利用 SoC 统一内存把 LLM 内存切成细粒度共享 buffer，做 partial eviction + 跨处理器并发回填；hybrid swap + backward-out 策略保证从任意驱逐状态都能低延迟恢复。基于 llama.cpp 实现为 Android 应用，对 storage-backed partial offload 的 TTFT 降 2.1–5.5×。

为什么值得关注

端侧 LLM 落地必须正视多任务：mzCache 用 SoC 统一内存做 partial eviction + 并发回填，TTFT 比 storage-backed offload 降 2.1–5.5×。

要点摘录：

来源：arXiv 论文页面元数据 + 摘要
标签：on-device-llm, kv-cache, android, memory-management, mobicom-2026
日期：2026-09-01

关键信息

标题：mzCache: On-Device LLM Memory Management under Multitasking
URL：https://arxiv.org/abs/2609.01338
抓取日期：2026-09-09

English Abstract / Excerpt

Published at MobiCom 2026. The biggest realistic issue for on-device LLM inference is not model size but memory pressure from user multitasking: weights and KV cache get evicted, recovery is via slow storage or full KV recompute, and TTFT collapses. mzCache slices LLM memory into fine-grained shared buffers backed by SoC unified memory, performs partial eviction with concurrent CPU-side restoration, and adds hybrid swap + backward-out so any eviction state is recoverable. Implemented over llama.cpp as an Android app, mzCache delivers a 2.1–5.5× TTFT reduction vs. storage-backed partial offload.