Many recurring text functions are easy to describe but difficult to implement with rules, while calling a large remote model for every input introduces repeated cost, latency, and dependency on a provider. We present compile by training, which turns a natural-language specification into a reusable neural function. At compile time, teacher models generate task-specific examples that are used to train a small adapter for a compact interpreter. The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Hard, a subset on which the Program-as-Weights fast compiler produced no exact matches, compile by training reaches 83.6% semantic accuracy. This higher accuracy comes with a higher compile-time cost: roughly a minute rather than seconds for the fast compiler. We deploy the compiler in a public interactive service and demonstrate compiled functions in a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a paper by Deng, Nie, and Shieber proposing a method they call "compile by training" — a way to turn large language model capabilities into small, local, versioned software artifacts.
Sam: So the starting problem is that calling a massive remote model for every routine text task is expensive and slow. What's their alternative?
Alex: They reframe finetuning as a build step. You write a natural language specification for what you want — say, "extract the sender's name from an email" — and the system uses a large teacher model to generate synthetic input-output pairs. Those pairs train a small LoRA adapter on top of a frozen local interpreter. After roughly a minute of compile time, you have a lightweight artifact that runs entirely on-device.
Sam: So the large model's role is purely at compile time. It never touches production traffic.
Alex: Exactly. They call the framework Program-as-Weights, or PAW. The base interpreter stays frozen. What you're shipping is just the adapter — a tiny set of weights that specialise the model for your specific task. You can version it, cache it, diff it against previous builds. It behaves like any other compiled binary.
Sam: That's a meaningful shift. But if the teacher is generating the training data, you're inheriting whatever the teacher gets wrong. Hallucinations, systematic biases — those get baked in.
Alex: The authors flag this explicitly. Teacher drift is the central limitation they name. If the teacher is consistently wrong on some input pattern, the compiled function will be consistently wrong on that pattern too — and because it's a frozen artifact, you won't catch it at runtime the way you might catch a stochastic model failure. For high-stakes applications, that's a real concern.
Sam: What does the compile process actually look like mechanically? You mentioned a warm-start.
Alex: Right. There are two phases. First, an amortised initialisation — a fast forward pass that gives you a rough starting point for the adapter weights. Think of it as a coarse prior. Then gradient descent refines those weights against the synthetic pairs. The warm-start matters because it dramatically reduces how many gradient steps you need to reach convergence, which is what keeps the compile time down to roughly a minute rather than hours.
Sam: And the output — the dot-paw artifact — contains the adapter and the prompt scaffold, but not the base model?
Alex: Correct. The interpreter is shared infrastructure. You're only distributing the specialisation on top of it. That's what makes the versioning story credible — the artifacts are small enough to treat like ordinary software packages.
Sam: How do they evaluate whether the compiled function is actually doing the right thing? Exact string match seems obviously too strict for natural language outputs.
Alex: They use what they call LLM Exact Match — a stronger model, GPT-4.5 in their setup, acts as a semantic judge. It evaluates whether the compiled function's output matches the intent of the specification, not the literal string. They show that naive string matching misses a meaningful fraction of semantically valid outputs, so this judge-based metric is load-bearing for their accuracy claims.
Sam: So the headline number — over 83% semantic accuracy on tasks where previous fast compilers failed entirely — that's measured against that judge.
Alex: That's the main finding, yes. And to be clear about what "failed entirely" means: the comparison class is methods that try to do fast, amortised compilation without the refinement step. Those collapse on tasks with complex or ambiguous specifications. The compile-by-training approach recovers most of that gap.
Sam: Did they look at how much synthetic data you actually need? There's a real practical question about whether the teacher call costs are worth it.
Alex: They did. Scaling from roughly 1,400 unique pairs to 3,600 improved accuracy, but the gains plateaued beyond that. So you're not looking at an open-ended data collection problem — there's a practical ceiling where additional teacher calls stop paying off, and it's within a range that's economically reasonable for a one-time build step.
Sam: And the composability question — can you chain these functions, or is each adapter strictly siloed?
Alex: They demonstrate composability directly. The paper includes a website helper that routes incoming questions through a tree of specialised PAW programs, mixed with conventional procedural code. Each node in the tree is an independently compiled adapter. So the architecture supports the kind of modular decomposition you'd want for anything beyond a toy task.
Sam: The versioning angle is what I keep coming back to. Right now, prompt engineering is a runtime activity — you're negotiating with a model every time. This turns it into a build-time activity with reproducible outputs.
Alex: That's the core reframing. The authors are explicit that the goal is to make LLM-backed functions behave like ordinary software components — deterministic given fixed inputs, auditable, deployable without cloud dependency. The public playground they've released lets developers test the build pipeline directly, which is a reasonable way to stress-test whether that framing holds in practice.
Sam: The natural next question is whether you'd want a package registry for these adapters. If someone's already compiled a high-quality JSON summariser, why recompile it?
Alex: That's the logical extension, and the authors gesture at it. If the artifact is truly portable — same frozen interpreter, same adapter format — then redistribution is straightforward. Whether the ecosystem develops that way depends on how stable the interpreter layer stays across versions, which is an open engineering question they don't fully resolve.
Sam: And the teacher drift problem becomes more acute in a shared registry, because you're now trusting someone else's compile-time teacher.
Alex: Exactly. That's where the limitation bites hardest. For now, the practical scope is probably internal tooling — teams compiling functions against specifications they control, using teachers they've evaluated. The broader redistribution story needs a solution to provenance and drift that this paper doesn't yet provide.
Sam: So: a concrete, working approach to a real bottleneck, with a clear limitation that honest reviewers would push on.
Alex: That's a fair summary. The mechanism is sound, the accuracy gains over the amortised baseline are the load-bearing result, and the teacher drift issue is the constraint that most limits where you'd trust this in production. Thanks for listening to ResearchPod.