Why don't machine learning research agents overfit?
Source: https://www.amazon.science/blog/why-dont-machine-learning-research-agents-overfit
Author: Martin Bertran Lopez, Aaron Roth
Published: 2026-09-10
Platform: blog
中文概要
Amazon Science（Martin Bertran Lopez、Aaron Roth）总结新研究：AI 科研智能体学到的数据模型本身可压缩，缺乏记忆所需的额外容量，因此在小规模任务语料上也不会过拟合，而是走向泛化。文章比喻"听者已知越多，所需信息越短；专家只需几句话，新人需要整本手册"，并指出压缩充当了自主 ML 研究者的隐式正则化器。
English Summary
Amazon Science (Martin Bertran Lopez, Aaron Roth) summarize their new ML-research finding: AI research agents learn compressible models of data, so they lack the spare capacity to memorize — that's why they generalize instead of overfitting, even on small task-specific corpora. Framing: the more the listener already knows, the shorter the message required; an expert engineer needs a few sentences, a newcomer needs the whole manual. Implication: compression acts as an implicit regularizer for autonomous ML researchers.