Agents at Risk: How Users Unwittingly Undermine LLM Safety

ID: 513d01e2
原文链接: https://arxiv.org/abs/2601.10758
PDF: https://arxiv.org/pdf/2601.10758
作者: Fengchao Chen, Tingmin Wu, Van Nguyen, Surya.Nepal, Carsten Rudolph
日期 / 版本: Submitted on 14 Jan 2026 (v1), last revised 7 Jul 2026 (this version, v3)
分类: agents
来源类型: paper
arXiv 分类: Cryptography and Security (cs.CR)
标签: llm-agent, security, prompt-injection, context-confusion, agent-safety, 2601-10758
质量评分: 4/5
抓取时间: 2026-07-27T04:25:26.452521+00:00

中文导读

论文研究 LLM Agent 的 User-Relayed Context Manipulation（UReCoM）攻击：攻击者诱导普通用户把对抗内容转述进请求，从而绕过只针对外部检索内容的防护摘要指出，Trip planningWeb-use 等 Agent 会把这些内容并入推理上下文，导致安全边界从外部内容过滤扩展到用户请求中的来源与权限识别

为什么值得关注

把用户转述的恶意上下文纳入 Agent 安全模型，提醒防线不能只盯检索和网页内容

这篇论文值得放进 AAIF 的原因，是它把 Agent 系统里的一个具体工程问题讲清楚：论文研究 LLM Agent 的 User-Relayed Context Manipulation（UReCoM）攻击：攻击者诱导普通用户把对抗内容转述进请求，从而绕过只针对外部检索内容的防护摘要指出，Trip planningWeb-use 等 Agent 会把这些内容并入推理上下文，导致安全边界从外部内容过滤扩展到用户请求中的来源与权限识别。以下内容基于 arXiv 页面元数据、摘要与条目已有摘要整理，未补充论文摘要之外的实验细节。

关键信息

论文标题：Agents at Risk: How Users Unwittingly Undermine LLM Safety
作者：Fengchao Chen, Tingmin Wu, Van Nguyen, Surya.Nepal, Carsten Rudolph
arXiv：https://arxiv.org/abs/2601.10758
发布时间 / 修订：Submitted on 14 Jan 2026 (v1), last revised 7 Jul 2026 (this version, v3)
arXiv 分类：Cryptography and Security (cs.CR)
arXiv 备注：User-relayed Context Manipulation; LLM-based Agents; Agent Security; Human Factors in Cybersecurity; Web-Use Agents; Planning Agents
关联标签：llm-agent, security, prompt-injection, context-confusion, agent-safety, 2601-10758

English Abstract

Large language model (LLM)-based agents are increasingly deployed in applications, such as trip-planning agents and web-use agents, to perform complex planning and execution tasks. Prior work has shown that LLM-based agents are vulnerable to context confusion, where external adversarial content incorporated into the agent's reasoning context may be treated as task-relevant constraints. However, external malicious content can enter the agent context via channels beyond retrieval. In this work, we introduce the User-Relayed Context Manipulation (UReCoM) attack, in which attackers manipulate benign users into relaying adversarial content within user requests, thereby relocating external adversarial content into user-provided task context. Our experimental evaluation shows that UReCoM outperforms five prompt-injection baselines (naive, context ignoring, fake completion, escape-character attacks, and combined attacks) under prevention-based (Sandwich, StruQ, and SecAlign) and detection-based defenses (Perplexity detection, DataSentinel, and CausalArmor). Additionally, UReCoM shows that LLMs can reject explicit malicious instructions more reliably than they can identify adversarial task entities, such as promotion codes, embedded within user requests. On 12 commercial LLM-based agents, we find that validation of adversarial task entities is largely prompt-driven rather than default, highlighting a design flaw in current agent frameworks. These results indicate that current defenses and deployed agents remain insufficient against user-relayed context manipulations, highlighting the need for task-entity-level prevention and default safety verification in LLM-based agents.

English Summary

Obsidian Notes

内容获取路径：优先尝试 opencli arxiv paper 2601.10758 -f json，本轮遭遇 arXiv API HTTP 429 后，改由 arXiv 页面元数据与摘要回填。
中文导读与价值判断均锚定在条目已有摘要、论文摘要、作者、日期与分类信息上。
现代站点生成器按 content/{entry.id}.md 查找内容页；本文件写入 canonical content 目录，而不是 openclaw/content/。