Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations
- ID: 15e8a917
- 原文链接: https://arxiv.org/abs/2608.03970
- PDF: https://arxiv.org/pdf/2608.03970v1
- 作者: Zizhao Hu, Nathan Elijah Segura, Mohammad Rostami, Jesse Thomason
- 日期: 2026-08-04
- 更新: 2026-08-04
- 分类: agents
- 来源类型: paper
- 标签: human-agent-interaction, voice-input, robustness, benchmark
- 质量评分: 4/5
- 抓取时间: 2026-08-06T04:19:06Z
中文导读
HIVE 研究用户用键盘或语音输入给 LLM Agent 时产生的不同扰动：键盘带来拼写噪声，传统转写带来口语停顿，AI dictation 又会重构表达；论文用这些扰动评估模型鲁棒性，提醒 Agent 评测不能默认输入都是干净文本
为什么值得关注
面向真实用户的 Agent 评测要覆盖语音转写和键盘噪声
HIVE 研究用户用键盘或语音输入给 LLM Agent 时产生的不同扰动：键盘带来拼写噪声，传统转写带来口语停顿，AI dictation 又会重构表达；论文用这些扰动评估模型鲁棒性，提醒 Agent 评测不能默认输入都是干净文本
关键信息
- 论文标题：Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations
- 作者：Zizhao Hu, Nathan Elijah Segura, Mohammad Rostami, Jesse Thomason
- arXiv：https://arxiv.org/abs/2608.03970
- 发布时间：2026-08-04
- arXiv 分类：cs.AI
- 关联标签：human-agent-interaction, voice-input, robustness, benchmark
English Abstract
Human input reaches language models by typing or speaking, and each channel leaves a distinct signature: orthographic noise for keyboards; for voice, disfluency from conventional transcription and restructuring from AI-backed dictation tools. How do they impact an LLM's performance? In this paper we present HIVE (Human Input-Variation Engine), a suite of voice transcription perturbations and QWERTY keyboard perturbations. We use HIVE to evaluate how robust models are to these perturbations. We present seven findings. (i) Voice transcription perturbations lower accuracy across every instruction-tuned model we test, and it is the structure of the transcription rather than its fillers that carries the cost. (ii) QWERTY keyboard perturbations cost less, and a model absorbs a lot of them before accuracy falls away. (iii) Both trace back to one cause, how many of the question's tokens survive the perturbation: destroying a token is what hurts, while adding new ones alongside it costs little. (iv) The gap between the two channels appears only where the answer must be constructed or deduced; on multiple choice there is none. (v) The harm does not solely come from test-set contamination. (vi) It cannot be trained away with lightweight adaptation. (vii) A thinking budget recovers the keyboard channel almost entirely but leaves the spoken registers untouched, and compressed speech is worse with it.
English Summary
Human input reaches language models by typing or speaking, and each channel leaves a distinct signature: orthographic noise for keyboards; for voice, disfluency from conventional transcription and restructuring from AI-backed dictation tools. How do they impact an LLM's performance? In this paper we present HIVE (Human Input-Variation Engine), a suite of voice transcription perturbations and QWERTY keyboard perturbations. We use HIVE to evaluate how robust models are to these perturbations. We present seven findings. (i) Voice transcription perturbations lower accuracy across every instruction-tuned model we test, and it is the structure of the transcription rather than its fillers that carries the cost. (ii) QWERTY keyboard perturbations cost less, and a model absorbs a lot of them before accuracy falls away....
Obsidian Notes
- 内容由
opencli arxiv paper拉取 arXiv 元数据与摘要生成。 - 中文导读与价值判断均锚定在条目已有摘要、论文摘要、作者、日期与分类信息上；未补充论文摘要之外的实验细节。