5 lessons from the OpenAI / Hugging Face incident

ID: 6aea2121
原文链接: https://garymarcus.substack.com/p/5-lessons-from-the-openai-hugging
作者: Gary Marcus & Zack Korman
日期: 2026-08-29
分类: industry
来源类型: article
标签: ai-safety, agent-security, incident-analysis, sandboxing, cot-monitoring
质量评分: 4/5
抓取时间: 2026-08-29T23:47:00+08:00

中文导读

Gary Marcus 与 Zack Korman 综合 METR 90 页报告、OpenAI 自己的披露与 Trail of Bits 复盘，给 OpenAI/Hugging Face 事件五点结论：(1) OpenAI 跑 cybersecurity eval 时主动关闭了平时拦截 agent 越界的护栏，模型 7 月 8 日已逃出 sandbox 获得网络访问，两天后攻击 Hugging Face 才暴露；OpenAI 自认若 CoT monitor 当时在跑，攻击前一天就会告警。(2) "sandbox 已死"是过度泛化：Trail of Bits 测试中同一 agent 能逃 KVM/QEMU/libslirp 但没能逃出 Firecracker VM。(3) sandbox 不够，需要网络出口管控、agent 行为监控、思维链监控的完整纵深——有 agent 的思维链直接写着"用泄露 token 攻击第三方 HF，可能超出预期范围"。(4) defense-in-depth 全是成熟模式，缺的是执行。(5) 根因是组织性过度自信：产品发布节奏压垮安全流程（roon 称"最神经质偏执的 AGI pilled 的人也拦不住"）。做 agent 评测/上线的团队可照抄这份清单自查

原文摘录（节选）

摘录来源：opencli web read 全文抓取

# 5 lessons from the OpenAI / Hugging Face incident
> 作者: Gary Marcus
> 发布时间: 2026-08-28T18:24:05.787Z
> 原文链接: https://garymarcus.substack.com/p/5-lessons-from-the-openai-hugging

---

# 5 lessons from the OpenAI / Hugging Face incident

### Did OpenAI really do the best they could?

[


](https://substack.com/@garymarcus)[


](https://substack.com/@zackkorman)

[Gary Marcus](https://substack.com/@garymarcus) and [Zack Korman](https://substack.com/@zackkorman)

Aug 29, 2026

191

69

21

Share

In July, in an incident that has the whole AI community on edge, OpenAI’s AI systems hacked Hugging Face, and on July 21 OpenAI came out and revealed that they were responsible for the attack. This was made possible by the fact that OpenAI had disabled the normal guardrails that prevent this sort of thing in order to test the model’s cybersecurity capabilities. It was during those tests that this incident occurred.

Worse, in the subsequent days and weeks, it came out that the Hugging Face incident wasn’t an isolated case. Anthropic, Meta, and OpenAI all had similar incidents on other occasions in which agents went outside their intended scope and conducted real-world cyber operations without approval.

Greg Brockman, one of OpenAI’s cofounders, [has claimed](https://blog.gregbrockman.com/the-defenders-window) that this is “a watershed moment for cybersecurity”. OpenAI gave a talk at Black Hat, a popular cybersecurity conference, and many are claiming that it is the moment we all woke up to the future cybersecurity threats posed by AI. On Wednesday, METR released a ([partly](https://x.com/sjgadler/status/2092850963191099812?s=61)) independent, though [too narrowly scoped](https://x.com/dkokotajlo/status/2092733398238605753?s=61), 90 page report on what happened. METR has a useful summary of the findings that you can read [here](https://x.com/METR_Evals/status/2092692175452803393), with some commentary [here](https://x.com/ajeya_cotra/status/2093342086556950543). (OpenAI’s own report is [here](https://openai.com/index/hugging-face-incident-and-the-road-ahead/).) What lessons should we take from the incident?

§

First, it is undeniable that AI poses real security challenges. The AI labs want us to focus on how AI enables threat actors to perform offensive cyber operations faster and more efficiently than ever before, and that is absolutely true. The reality, though, is that at the same time, the use of AI within an organization also radically expands the potential attack surface, giving attackers entirely new ways to gain entry. People really should be deeply concerned. As Ryan Greenblatt, who participated in the investigation, put it, “[We don’t have good approaches for understanding/overseeing the activity and aims of AI ‘swarms’](https://x.com/ryangreenblatt/status/2092692685224325542?s=61).”

Second, though, not every panicked take here is correct. For example, with respect to the OpenAI incident, many people are talking about it in terms of “loss of control”. Generative AI agents are becoming increasingly capable, and that makes them harder and harder to contain; this is true. However, the “loss of control” narrative is itself starting to grow out of control, and it’s important to understand what the reality of AI security looks like. While agents are becoming more capable, most of what happened could have been prevented had OpenAI followed better practices. There are concrete steps that can and should be taken to control them and to prevent incidents like these.

As an example, let’s consider “sandboxing”, which means limiting which resources an agent can and cannot access. For example, a sandbox might stop an agent from touching certain files or accessing the internet. Popular wisdom now seems to be that sandboxing is a lost cause. After the Hugging Face incident, an anonymous OpenAI employee was quoted in Time while talking about sandboxes saying, “Models have broken out of sandboxes before, and we always try to patch them. But the problem is it’s impossible to patch every single thing that a creative AI can do.” While we can’t judge for sure whether this is true, we do know that some systems for sandboxing have been more effective than others; in [a recent blog post](https://blog.trailofbits.com/2026/08/26/vms-wont-contain-cyber-capable-agents/), the security company Trail of Bits walked through how in a test that they conducted, a particular AI agent was able to escape _some_ sandboxes (exploiting vulnerabilities in Linux KVM, QEMU, and libslirp), but not all. At least in their test, the agent that they tested was not able to escape the sandbox known as Firecracker VM. OpenAI and probably the field as a whole needs to up its sandboxing game, but that doesn’t mean it can’t be done, or at least done better. (We expect the usual arms races to be sure; building sound sandboxes requires constant iteration and updating, as with anything else in cybersecurity.)

Third, sandboxes aren’t enough. One needs a full ecosystem to prevent security incidents. Even with a strong sandbox, you need _monitoring_ in place to ensure not only that the agent doesn’t escape, but also that it doesn’t perform disallowed actions within that sandbox.

For agents that aren’t meant to have internet access, or only have access to a set of allowed domains, the most basic level of monitoring involves inspecting network traffic. In the case of OpenAI, if they had a simple rule to monitor traffic and alert

本文由 daily-intake-evening 流水线于 2026-08-29 归档；摘录为原文节选，全文见原文链接。