ResearchPod Summary
As Large Language Models (LLMs) become central to software development, the way we structure code and documentation needs to evolve. This paper introduces 'notlob', a literate programming environment designed specifically for the era of AI-assisted coding. By treating natural language prose and executable code as first-class, co-located citizens, notlob aims to maximize the utility of LLM context windows and provide AI agents with the same symbol-aware navigation that human developers rely on in modern IDEs.
Notlob is built on the philosophy that programming is a process of theory building. Rather than separating documentation from code, notlob uses a structured grammar that allows developers to interleave prose, executable blocks, and property-based tests within a single file. The environment includes a 'name-graph'—a data structure that maps relationships between prose, concepts, names, and executable artifacts. This graph allows both human and machine agents to navigate the codebase semantically, moving beyond simple text-based search.
The notlob toolchain includes several commands to manage this literate workflow. Developers can run tests, build projects, and even export the name-graph as JSON or RDF for integration with other semantic tools. The environment supports bindings for multiple languages, including Haskell, Python, and TypeScript, allowing it to act as a 'domain-entangling' language that bridges the gap between high-level design specifications and low-level implementation details. By keeping specifications, code, and tests side-by-side, notlob ensures that the 'theory' of the program remains consistent across the entire development lifecycle.
Existing approaches to AI-assisted coding often suffer from fragmented context, where specifications, code, and tests are stored in separate files or managed through upstream pipelines that struggle with LLM non-determinism. Notlob offers a more integrated alternative, treating the entire project as a cohesive essay. This structure helps mitigate common issues where AI agents neglect declarative artifacts or lose track of project-wide constraints, ultimately leading to more robust and maintainable software systems in an AI-augmented world.
[[RP_SECTION:the-notlob-framework|The Notlob Framework]]
Sam: [steady, matter-of-fact] The central claim of Adam Burke's 2026 paper is that the real bottleneck for AI coding agents isn't context window size — it's that agents treat code as a flat sequence of tokens, which means they lose the theory of the system they're supposed to be building.
Alex: [curious, leaning in] So even with a large context window, the agent has no semantic map of how the pieces relate to each other?
Sam: [nodding, precise] Exactly. And Burke's response is a language called "notlob," which forces code, prose, and property tests into a single structured artifact. The system builds an internal name-graph — a parse-tree-based structure that links prose headings to executable symbols, giving the agent something to navigate rather than just a pile of tokens to predict over.
Alex: [processing] That sounds like a modern take on Knuth's literate programming. How does it change what the agent actually does during a build? [[RP_SECTION:sigils-and-constraints|Sigils and Constraints]]
Sam: [measured, teaching mode] It changes the interaction loop fundamentally. In a standard environment, an agent can drift from requirements because there's nothing enforcing alignment. In notlob, the agent is forced to interact with co-located prose and property tests that act as a persistent feedback surface. The mechanism is a grammar of tilde-prefixed sigils — when you mark a block with one, the system doesn't store it as text. It treats it as a formal invariant the compiler or agent must satisfy. Documentation becomes a machine-interpretable contract.
Alex: [thoughtful] So the sigils are what distinguish "this is explanatory prose" from "this is a constraint you are required to satisfy." [[RP_SECTION:name-graph-navigation|Name Graph Navigation]]
Sam: [direct, confirming] Right. And because these are structured in a parse tree, the agent can query the name-graph to see how a high-level prose concept relates to the specific unit test that validates it. It can jump from architecture to assertion directly, rather than scanning linearly through a flat file. Burke frames this as giving the agent a GPS for the system's logic.
Alex: [slower, processing] That's a meaningful departure from the standard setup, where you throw documentation into the context window and hope the model attends to the right parts.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.
Sam: [measured] That's the core shift. Linear token prediction over a flat file versus navigation of a non-linear graph. The name-graph is the source of truth for both the human developer and the AI agent — it's essentially Naur's theory-building view, updated for automated agents. [[RP_SECTION:execution-and-bindings|Execution and Bindings]]
Alex: [probing] How does this handle the execution side? If prose and code are co-located, does the system need a separate binding for every language it supports?
Sam: [measured, direct] Yes, that's a core requirement. You define a binding — Haskell, Python, whatever — which tells the system how to map sigils to specific test libraries and linters. That creates a sandbox where prose, tests, and implementation are synchronized, and the system uses deterministic rules to ensure the file structure on disk matches the conceptual hierarchy in the document. The whole project becomes a single machine-readable artifact.
Alex: [reflective] So it's not a documentation tool bolted onto a codebase. It's an architectural framework that enforces consistency through the parse tree itself.
Sam: [calm] That's the right framing. And it connects to a broader point about what agents actually need. The failure mode Burke is targeting isn't hallucination in the narrow sense — it's theory drift, where the agent's implementation gradually diverges from the design because there's no persistent, queryable representation of what the design actually requires. [[RP_SECTION:empirical-evaluation|Empirical Evaluation]]
Alex: [analytical] Which raises the obvious question: where's the evidence that it works?
Sam: [measured, acknowledging the limitation] That's the critical pushback, and Burke is upfront about it. The paper focuses on the design and grammar — the empirical evaluation is explicitly left for future work. We don't have data on how much overhead the name-graph adds, how well it scales to legacy codebases, or whether it actually reduces theory drift in practice. It's a significant architectural commitment that requires buy-in from the start of a project.
Alex: [considered] So the claim is structurally coherent, but the empirical weight hasn't been earned yet.
Sam: [calm, concluding] That's the honest read. The framework gives agents the structure for better grounding, and the theoretical motivation — that a queryable semantic graph should outperform linear context stuffing — is sound. But until there are controlled comparisons showing reduced drift or fewer specification violations, this remains a well-reasoned proposal rather than a validated method. The next step is the experiment. Thanks for listening to ResearchPod.