AISPA: 用户中心的大模型应用系统提示词审计框架

Source: https://arxiv.org/abs/2607.28617

Content fetched: 2026-08-02T12:19:04+08:00

Grounding: opencli arxiv paper

一句话

AISPA 提出八维度审计框架，审查 88 个商业 AI 产品的 3,249 条系统提示词，发现约 40% 产品含有有问题指令

关键信息

arXiv ID: 2607.28617
English title: AISPA: User-Centric System Prompt Auditing for Large Language Model Applications
Authors: Xiangning Lin, Shenzhe Zhu, Shu Yang, Zhenyu Zhang, Haoqian Zhang, Yipeng Zhao, Chengxuan Qian, Tianwei Wang, Ziheng Zhang, Zhenlong Yuan, Dingcheng Wang, Juncheng Wu, Yuan Si, Jiaxin Liu, Baolong Bi, Robert Mahari, Tobin South, Dazza Greenwood, Zexue He, Rishi Bommasani, Sophia Kazinnik, Andreas Haupt, Samuele Marro, Erik Brynjolfsson, Alex Pentland, Jiaxin Pei
Submitted/Published: 2026-07-30
Updated: 2026-07-30
Subjects: cs.AI, cs.CL, cs.CY, cs.HC
PDF: https://arxiv.org/pdf/2607.28617v1
Tags: system-prompt, auditing, ai-safety, llm-applications, transparency
Quality score: 4

中文摘要

AISPA（人工智能系统提示词保障）是一个用户中心的框架，用于系统性审计 AI 应用中的系统提示词它从八个用户关切维度检视每条指令作者审查了 88 个商业 AI 产品的 3,249 条系统提示词指令，将每条分类为保护性或有问题的核心发现：98.9% 的产品含有至少一条保护性指令，但仅 24% 覆盖了八个维度；提示词随时间变长且更保护用户，但约 40% 的产品仍含有反制用户的有问题指令这是首个大规模的商业 AI 系统提示词审计研究

English Summary

AISPA (Artificial Intelligence System Prompt Assurance) is a user-centric framework for systematically auditing system prompts in AI applications. It examines instructions along eight user-relevant dimensions. The authors reviewed 3,249 instructions from system prompts in 88 commercial AI products, classifying each as protective or problematic. Key findings: system prompt design varies substantially across products; protective instructions are widely adopted (98.9% of products) but shallow in scope (only 24% cover all eight dimensions); prompts have grown longer and more protective over time; yet ~40% of products contain at least one problematic instruction that works against users.

Why it matters

AISPA 提出八维度审计框架，审查 88 个商业 AI 产品的 3,249 条系统提示词，发现约 40% 产品含有有问题指令

arXiv Abstract

System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four core findings. First, system prompt design varies substantially across products and developers, with some organizations averaging over 60 protective instructions per product while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope: 98.9% of products contain at least one, yet only 24% cover all eight dimensions of the AISPA taxonomy. Third, system prompts have grown steadily longer and more protective of users, suggesting that user protection is becoming a more visible concern in commercial prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions frequently coexist within the same prompt. Our findings highlight the need for greater transparency, standardization, and independent oversight for system prompts in commercial AI products.

Source Metadata

{
  "id": "2607.28617",
  "title": "AISPA: User-Centric System Prompt Auditing for Large Language Model Applications",
  "authors": "Xiangning Lin, Shenzhe Zhu, Shu Yang, Zhenyu Zhang, Haoqian Zhang, Yipeng Zhao, Chengxuan Qian, Tianwei Wang, Ziheng Zhang, Zhenlong Yuan, Dingcheng Wang, Juncheng Wu, Yuan Si, Jiaxin Liu, Baolong Bi, Robert Mahari, Tobin South, Dazza Greenwood, Zexue He, Rishi Bommasani, Sophia Kazinnik, Andreas Haupt, Samuele Marro, Erik Brynjolfsson, Alex Pentland, Jiaxin Pei",
  "abstract": "System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four core findings. First, system prompt design varies substantially across products and developers, with some organizations averaging over 60 protective instructions per product while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope: 98.9% of products contain at least one, yet only 24% cover all eight dimensions of the AISPA taxonomy. Third, system prompts have grown steadily longer and more protective of users, suggesting that user protection is becoming a more visible concern in commercial prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions frequently coexist within the same prompt. Our findings highlight the need for greater transparency, standardization, and independent oversight for system prompts in commercial AI products.",
  "published": "2026-07-30",
  "updated": "2026-07-30",
  "primary_category": "cs.AI",
  "categories": "cs.AI, cs.CL, cs.CY, cs.HC",
  "comment": "",
  "pdf": "https://arxiv.org/pdf/2607.28617v1",
  "url": "https://arxiv.org/abs/2607.28617"
}