ADAM T. BURKE
4 min
As Large Language Models (LLMs) become central to software development, the way we structure code and documentation needs to evolve. This paper introduces 'notlob', a literate programming environment designed specifically for the era of AI-assisted coding. By treating natural language prose and executable code as first-class, co-located citizens, notlob aims to maximize the utility of LLM context windows and provide AI agents with the same symbol-aware navigation that human developers rely on in modern IDEs.
Notlob is built on the philosophy that programming is a process of theory building. Rather than separating documentation from code, notlob uses a structured grammar that allows developers to interleave prose, executable blocks, and property-based tests within a single file. The environment includes a 'name-graph'—a data structure that maps relationships between prose, concepts, names, and executable artifacts. This graph allows both human and machine agents to navigate the codebase semantically, moving beyond simple text-based search.
The notlob toolchain includes several commands to manage this literate workflow. Developers can run tests, build projects, and even export the name-graph as JSON or RDF for integration with other semantic tools. The environment supports bindings for multiple languages, including Haskell, Python, and TypeScript, allowing it to act as a 'domain-entangling' language that bridges the gap between high-level design specifications and low-level implementation details. By keeping specifications, code, and tests side-by-side, notlob ensures that the 'theory' of the program remains consistent across the entire development lifecycle.
Existing approaches to AI-assisted coding often suffer from fragmented context, where specifications, code, and tests are stored in separate files or managed through upstream pipelines that struggle with LLM non-determinism. Notlob offers a more integrated alternative, treating the entire project as a cohesive essay. This structure helps mitigate common issues where AI agents neglect declarative artifacts or lose track of project-wide constraints, ultimately leading to more robust and maintainable software systems in an AI-augmented world.
This paper introduces an environment for constructing literate programs in concert with language-aware machine agents. This environment includes a grammar for executable program essays, a parser that treats names as first-class objects, an internal name-graph which relates prose, names and executable artifacts, and a binding mechanism for existing languages and testing toolsets. This supports co-location of code with its most relevant natural language and structured data context, making better use of Large Language Model (LLM) context windows. It also provides LLM coding agents with a toolset more analogous to the symbol-aware search and usage information available in human programmer-facing Integrated Development Environments (IDEs). We describe a working implementation with bindings to three established programming languages, and several example programs.
Alex: [probing] How does this handle the execution side? If prose and code are co-located, does the system need a separate binding for every language it supports?
Sam: [measured, direct] Yes, that's a core requirement. You define a binding — Haskell, Python, whatever — which tells the system how to map sigils to specific test libraries and linters. That creates a sandbox where prose, tests, and implementation are synchronized, and the system uses deterministic rules to ensure the file structure on disk matches the conceptual hierarchy in the document. The whole project becomes a single machine-readable artifact.
Alex: [reflective] So it's not a documentation tool bolted onto a codebase. It's an architectural framework that enforces consistency through the parse tree itself.
Sam: [calm] That's the right framing. And it connects to a broader point about what agents actually need. The failure mode Burke is targeting isn't hallucination in the narrow sense — it's theory drift, where the agent's implementation gradually diverges from the design because there's no persistent, queryable representation of what the design actually requires. [[RP_SECTION:empirical-evaluation|Empirical Evaluation]]
Alex: [analytical] Which raises the obvious question: where's the evidence that it works?
Sam: [measured, acknowledging the limitation] That's the critical pushback, and Burke is upfront about it. The paper focuses on the design and grammar — the empirical evaluation is explicitly left for future work. We don't have data on how much overhead the name-graph adds, how well it scales to legacy codebases, or whether it actually reduces theory drift in practice. It's a significant architectural commitment that requires buy-in from the start of a project.
Alex: [considered] So the claim is structurally coherent, but the empirical weight hasn't been earned yet.
Sam: [calm, concluding] That's the honest read. The framework gives agents the structure for better grounding, and the theoretical motivation — that a queryable semantic graph should outperform linear context stuffing — is sound. But until there are controlled comparisons showing reduced drift or fewer specification violations, this remains a well-reasoned proposal rather than a validated method. The next step is the experiment. Thanks for listening to ResearchPod.