Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng
5 min
Abstract
Many everyday programming tasks resist clean rule-based implementation, such as alerting on important log lines, repairing malformed JSON, or ranking search results by intent, and are increasingly outsourced to large language model APIs at the cost of locality, reproducibility, and price. We propose fuzzy-function programming: compiling such a function from a natural-language specification into a compact, locally-executable neural artifact. We instantiate this paradigm with Program-as-Weights (PAW), in which a 4B compiler trained on FuzzyBench, a 10M-example dataset we release, emits parameter-efficient adapters for a frozen, lightweight interpreter. A 0.6B Qwen3 interpreter executing PAW programs matches the performance of direct prompting of Qwen3-32B, while using roughly one fiftieth of the inference memory and running at 30 tokens/s on a MacBook M3. PAW reframes the foundation model from a per-input problem solver into a tool builder: invoked once per function definition, it produces a small reusable artifact whose subsequent calls per function application are cheap and offline.
Alex: Precisely. The interpreter stays frozen—it never changes. You just hot-swap the LoRA adapter for whichever task you need. That's what makes the whole system efficient.
Sam: And how much more efficient are we talking?
Alex: According to the paper, a small interpreter running these PAW packages can match the performance of a model roughly fifty times larger. That difference in size translates directly to memory—which is what makes it feasible to run on a phone or laptop rather than a data center.
Sam: That's a meaningful gap. It's the difference between needing a server farm and needing something that fits in your pocket.
Alex: And that gap has real consequences. When the model runs locally, your data never leaves your device. There's no network call, no subscription cost per query, and no dependency on an internet connection.
Sam: Though I imagine there are trade-offs. You mentioned the compiler and interpreter are locked together—so if I want to upgrade to a newer, smarter interpreter, do I lose all my compiled packages?
Alex: That's a genuine limitation the paper acknowledges. Because the compiler is trained to generate weights specifically for one interpreter's architecture, changing the interpreter means retraining the compiler from scratch. Think of it like switching from a PC to a Mac—the old software simply won't run.
Sam: What about when something goes wrong? Can you actually look inside a PAW package to debug it?
Alex: Partially. The pseudo-program—the plain-language instruction set—is readable. But the LoRA weights, the part that actually shapes the model's behavior, are opaque. You can see the recipe, but not why the cook made certain choices. That makes diagnosing subtle errors genuinely difficult.
Sam: And I'm guessing it can only handle fairly simple tasks at this stage?
Alex: Currently, yes. The system handles single-step tasks—one input, one output. Multi-step reasoning, where a system needs to work through a problem in several stages, isn't yet supported. That's a meaningful boundary for more complex automation.
Sam: So it's a focused tool, not a general one. Which, in a way, is also part of the point—it can't be prompted into doing something unexpected.
Alex: That's a fair way to put it. The system is designed for specific, narrow functions rather than open-ended generation. And by separating the heavy compilation work—done once, by a large model—from the lightweight execution done locally, the paper argues you can make these fuzzy functions reliable, private, and fast. It's a practical step toward AI tools that live on your device rather than in someone else's cloud. Thanks for listening to ResearchPod.