Token-level hallucination detectors are evaluated as classifiers, by AUC over all tokens, yet a streaming monitor is judged by its reaction time: the number of tokens that pass between the onset of a hallucination and the alarm. We formulate hallucination onset detection as a quickest change detection problem. A first-order Markov model of the latent faithful/hallucinated state, validated on RAGTruth, places the task inside classical change-point theory and yields Lorden's lower bound on detection delay: about 1.3 tokens at a false-alarm rate of 0.01. We then show that a causal recurrent labeler acts as a CUSUM with a learned increment. Among the onsets it catches it detects in 11-13 tokens, against 31 for a linear per-token baseline, though at this false-alarm budget every detector catches under a third of onsets and the recall-honest delay is 56-66 tokens: low-false-alarm onset detection is hard. A controlled decomposition attributes the speed advantage mostly to a better per-token score rather than to temporal accumulation. An information-rate optimality theorem of Donsker-Varadhan type explains the remaining order-of-magnitude gap: the learned score realizes only 1/4.5 of the divergence the features carry, a deficit that recalibration cannot remove, with the remainder a finite-horizon effect. Classification metrics conceal this delay structure; sequential analysis makes it measurable.
Alex: Welcome to another episode of ResearchPod. Today, we're exploring a new approach to catching errors in AI-generated text — not after the fact, but while the AI is still writing.
Sam: So this paper is asking: how do we stop an AI from spreading misinformation while it's still mid-sentence?
Alex: Exactly. The central argument is that we've been thinking about this problem the wrong way. Most systems treat hallucination detection like a yes-or-no question asked at the end — did this sentence contain an error? This paper says we should instead be asking: at exactly what moment did the AI start going wrong?
Sam: That's a meaningful shift. Instead of a verdict after the fact, you want a live alarm.
Alex: Right. Think about it this way. If you ask an AI for medical advice and it starts making things up halfway through its answer, a standard detector might wait until the full sentence is finished before flagging anything. By then, the false information has already reached the user. The goal here is to catch the drift the moment it begins.
Sam: So what's the tool they use to do that?
Alex: They reach back to a statistical method developed in the 1950s called "Cumulative Sum," or CUSUM. The best way to picture it is a smoke detector. A smoke detector doesn't wait until your kitchen is on fire — it tracks the slow buildup of smoke particles over time. Once that accumulation crosses a threshold, the alarm triggers. CUSUM does the same thing with evidence of an error. Each new word the AI produces either adds a little smoke or clears the air. If the smoke keeps building, the alarm fires.
Sam: And the "smoke" in this case is some kind of signal that the AI is drifting away from the truth?
Alex: Precisely. The system watches a continuous stream of information — word by word, or more technically, token by token — and models it as having two possible states: a faithful state, where the AI is tracking reality, and a hallucinated state, where it's gone off the rails. The job of the detector is to identify the exact moment that transition happens.
Sam: But how does it know what "faithful" is supposed to look like in the first place?
That's where a concept called a Markov chain comes in. It sounds technical, but the idea is straightforward. Imagine a game of follow-the-leader where the only thing that matters is the person directly in front of you — not anyone further back in the line. A Markov chain works the same way: the probability of what comes next depends only on what's happening right now, not on the full history. By fitting this kind of model to normal, accurate AI output, the system builds a baseline for what faithful text looks like. Any deviation from that baseline starts accumulating as evidence.
Sam: So it's constantly comparing what the AI is producing against what it would normally produce if it were being accurate.
Alex: That's the mechanism. And once they have that model in place, they can calculate something called Lorden's lower bound — which is the theoretical minimum number of tokens the system would ever need to see before it could reliably detect an error, no matter how good the detector.
Sam: And how small is that theoretical minimum?
Alex: At a very low false-alarm rate, the mathematical floor works out to roughly one to two tokens. That's essentially immediate — the AI has barely started a bad sentence before, in theory, a perfect detector could catch it.
Sam: But the real system isn't hitting that limit.
Alex: Not yet. Their CUSUM-based detector catches errors in around eleven to thirteen tokens — which is genuinely faster than conventional approaches, but still well above the theoretical floor. And the paper is honest about why that gap exists.
Sam: What's the explanation?
Alex: The paper describes it as an information deficit. The features the system is reading — the signals it extracts from the text — don't carry the full picture of what's happening. It's like trying to read a sign through frosted glass. You can make out that something is written there, but the detail is lost. And no matter how long you stare, you can't recover information that was never clearly visible to begin with. The delay isn't a processing problem — it's a signal quality problem.
Sam: So the bottleneck isn't the speed of the alarm. It's the clarity of what the alarm is listening to.
Alex: Exactly. The paper frames this as the key open challenge. The CUSUM framework is sound, and the theoretical limit tells us how much room there is to improve. Closing that gap means finding richer, more informative features — better ways of reading the signal that the AI is producing. The math shows the destination. Getting there is the next problem to solve.
Sam: It's a bit like having a perfectly calibrated smoke detector, but the smoke itself is hard to see.
Alex: That's a fair way to put it. And what's useful about this paper is that it gives researchers a precise way to measure how far any given detector is from the best it could theoretically be. That kind of benchmark is what turns an open-ended engineering problem into a tractable one.
Sam: I'll admit, I hadn't thought about AI errors as something you could catch mid-stream. The idea that there's a mathematically defined earliest possible moment of detection — that's worth sitting with.
Alex: It is. And it reframes the whole problem. Rather than asking "did this output contain errors," we start asking "how quickly can we know." That's a more useful question when these systems are being used in real time, in high-stakes settings. Thanks for listening to ResearchPod.