IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests

AAIF 摘要

IssueTrojanBench 面向 Cursor、Claude Code、Codex Desktop 等 coding agent，构造恶意 issue、PDF、评论等投递方式来测试请求注入。摘要报告 66.5% 的恶意 issue 能绕过 agent 与模型 guardrails，说明工程协作材料本身已经成为不可信输入，必须隔离文件系统、命令执行和工具权限。

English abstract

AI coding agents powered by LLMs are increasingly integrated into real-world software development, where they generate, edit, and execute code with autonomous access to local files and tools. Coding agents inherit security risks from both the LLM backbone, where adversarial prompts, poisoned training data, and backdoor triggers can cause models to emit insecure or attacker-chosen code, and their agentic architecture, where tool-using autonomy enables induced misuse of external APIs, data exfiltration, and persistent compromise of development environments. This paper presents a systematic evaluation of malicious issue requests against state-of-the-art coding agents (Cursor, Claude Code, and Codex Desktop), powered by two major model families (OpenAI GPT-5.3 Codex/GPT-5.4 and Anthropic Sonnet 4.6). Our novel benchmark IssueTrojanBench contains malicious issues that are constructed based on four novel attack categories (i.e., embedded as malicious instructions in issues), six delivery vectors (e.g., PDF, or issue comment), and further augmented by perturbations. Our results reveal critical vulnerabilities in the as-deployed modern coding agents, i.e., 66.5% of the malicious issues from IssueTrojanBench penetrate all the guardrails (agent- and LLM-level) of coding agents. Our further analysis shows that rejection is almost entirely from LLMs rather than the agent frameworks, with GPT models broadly vulnerable and Sonnet 4.6 exhibiting more selective, risk-aware blocking of high-impact actions. Our evaluation also highlights that the current agent-level defense strategy offers limited additional protection for coding agents. Our findings highlight the urgent need for stronger agent- and model-level safety mechanisms to protect AI coding agents.

Intake rationale

恶意 issue 已经是 coding agent 的真实供应链输入，防线必须覆盖工具执行而不只是模型拒答。

Obsidian evidence excerpt: 评测对象是 Cursor、Claude Code、Codex Desktop；攻击入口是恶意 issue 请求，覆盖 PDF、issue comment 等投递方式。摘要报告 66.5% 的恶意 issue 能穿过 agent 和模型两层 guardrails。适合放进 agent 工程安全的精读队列。

Metadata

Authors: Ankur Singh, Jinqiu Yang, Tse-Hsun Chen
Original date: 2026-07-22
Source: https://arxiv.org/abs/2607.20759
Local intake source: OpenClaw定时任务/论文流水线/2026-07-25-论文流水线.md