ACE: A Security Architecture for LLM-Integrated App Systems

ID: 532bd31f
原文链接: https://arxiv.org/abs/2504.20984
PDF: https://arxiv.org/pdf/2504.20984v3
作者: Evan Li, Tushin Mallick, Evan Rose, William Robertson, Alina Oprea, Cristina Nita-Rotaru
日期: 2025-04-29
更新: 2025-09-10
分类: agents
标签: llm-apps, agent-security, tool-calling, prompt-injection, secure-architecture
质量评分: 4/5
抓取时间: 2026-07-26T12:28:05+08:00

一句话

LLM App 系统的安全边界应先抽象规划再具体执行，避免恶意 App 污染计划阶段

中文导读

ACE 把 LLM 集成第三方 App 的风险拆成 planning 与 execution 两层：恶意 App 不只会做提示注入，还会破坏计划完整性执行可用性和隐私边界论文提出 Abstract-Concrete-Execute：先只基于可信信息生成抽象计划，再映射到具体已安装 App，为工具/应用调用型 Agent 提供可验证的安全架构切分

关键要点

把 Agent / 工具调用系统的攻击面从“模型回答是否安全”推进到规划、权限、数据流与执行边界。
论文备注：25 pages, 13 figures, 8 tables; accepted by Network and Distributed System Security Symposium (NDSS) 2026
arXiv 分类：cs.CR, cs.LG，适合放入 AAIF 的 agents 频道继续追踪。
内容页由 arXiv 元数据与摘要回填，后续若需要精读可再补充实验细节、威胁模型图和 benchmark 表格。

English Summary

LLM-integrated app systems extend LLMs with third-party apps invoked through interleaved planning and execution. The paper identifies attacks where malicious apps can cause planning integrity violations, execution integrity and availability failures, or privacy compromise, including attacks against IsolateGPT. It proposes Abstract-Concrete-Execute (ACE), which separates planning into an abstract plan using only trusted information and then maps that plan to concrete installed apps, aiming to provide security guarantees for planning and execution.

Abstract

LLM-integrated app systems extend the utility of Large Language Models (LLMs) with third-party apps that are invoked by a system LLM using interleaved planning and execution phases to answer user queries. These systems introduce new attack vectors where malicious apps can cause integrity violation of planning or execution, availability breakdown, or privacy compromise during execution. In this work, we identify new attacks impacting the integrity of planning, as well as the integrity and availability of execution in LLM-integrated apps, and demonstrate them against IsolateGPT, a recent solution designed to mitigate attacks from malicious apps. We propose Abstract-Concrete-Execute (ACE), a new secure architecture for LLM-integrated app systems that provides security guarantees for system planning and execution. Specifically, ACE decouples planning into two phases by first creating an abstract execution plan using only trusted information, and then mapping the abstract plan to a concrete plan using installed system apps. We verify that the plans generated by our system satisfy user-specified secure information flow constraints via static analysis on the structured plan output. During execution, ACE enforces data and capability barriers between apps, and ensures that the execution is conducted according to the trusted abstract plan. We show experimentally that ACE is secure against attacks from the InjecAgent and Agent Security Bench benchmarks for indirect prompt injection, and our newly introduced attacks. We also evaluate the utility of ACE in realistic environments, using the Tool Usage suite from the LangChain benchmark. Our architecture represents a significant advancement towards hardening LLM-based systems using system security principles.

Source Metadata

arXiv ID: 2504.20984
Primary category: cs.CR
Categories: cs.CR, cs.LG
Comment: 25 pages, 13 figures, 8 tables; accepted by Network and Distributed System Security Symposium (NDSS) 2026

_Content backfilled by AAIF content-fetcher from OpenCLI arXiv metadata._