Compile by Training: Turning Natural-Language Specifications into Local Neural Functions (arXiv...

哈佛与 UT Austin 团队提出 compile by training：把自然语言规格编译成可复用的本地神经函数编译期由教师模型针对任务生成样本，训练紧凑解释器上的小型 adapter；产出的函数不再依赖教师，可像普通软件一样存储版本化与组合，规避逐次调用远程大模型的成本延迟与供应商依赖在 Program-as-Weights 快速编译器无法精确命中的 FuzzyBench-Hard 子集上，语义准确率达到 83.6%，代价是编译时间从秒级升到约一分钟作者部署了公开交互服务，并展示多站点网站助手语言可控 3D 头像与英-Claudish 双向翻译三个落地函数EMNLP 2026 System Demonstrations，2026-09-03 提交 cs.CL

Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

Source: https://arxiv.org/abs/2609.04199

Authors: Yuntian Deng, Pengyu Nie, Stuart Shieber

Published: 2026-09-03

Categories: cs.CL, cs.AI, cs.LG

PDF: https://arxiv.org/pdf/2609.04199

Abstract

Many recurring text functions are easy to describe but difficult to implement with rules, while calling a large remote model for every input introduces repeated cost, latency, and dependency on a provider. We present compile by training, which turns a natural-language specification into a reusable neural function. At compile time, teacher models generate task-specific examples that are used to train a small adapter for a compact interpreter. The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Hard, a subset on which the Program-as-Weights fast compiler produced no exact matches, compile by training reaches 83.6% semantic accuracy. This higher accuracy comes with a higher compile-time cost: roughly a minute rather than seconds for the fast compiler. We deploy the compiler in a public interactive service and demonstrate compiled functions in a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator.

Key Points

Problem: recurring text functions are easy to describe but hard to implement with rules, while calling a large remote model per input adds repeated cost, latency, and provider dependency.
Compile by training turns a natural-language specification into a reusable neural function: at compile time, teacher models generate task-specific examples used to train a small adapter for a compact interpreter.
The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software.
On FuzzyBench-Hard - a subset where the Program-as-Weights fast compiler produced no exact matches - compile by training reaches 83.6% semantic accuracy, at a higher compile-time cost of roughly a minute rather than seconds.
Deployed in a public interactive service with three compiled functions demonstrated: a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator.