Igor Itkin
5 min
Abstract
Token-level hallucination detectors are evaluated as classifiers, by AUC over all tokens, yet a streaming monitor is judged by its reaction time: the number of tokens that pass between the onset of a hallucination and the alarm. We formulate hallucination onset detection as a quickest change detection problem. A first-order Markov model of the latent faithful/hallucinated state, validated on RAGTruth, places the task inside classical change-point theory and yields Lorden's lower bound on detection delay: about 1.3 tokens at a false-alarm rate of 0.01. We then show that a causal recurrent labeler acts as a CUSUM with a learned increment. Among the onsets it catches it detects in 11-13 tokens, against 31 for a linear per-token baseline, though at this false-alarm budget every detector catches under a third of onsets and the recall-honest delay is 56-66 tokens: low-false-alarm onset detection is hard. A controlled decomposition attributes the speed advantage mostly to a better per-token score rather than to temporal accumulation. An information-rate optimality theorem of Donsker-Varadhan type explains the remaining order-of-magnitude gap: the learned score realizes only 1/4.5 of the divergence the features carry, a deficit that recalibration cannot remove, with the remainder a finite-horizon effect. Classification metrics conceal this delay structure; sequential analysis makes it measurable.
Sam: So it's constantly comparing what the AI is producing against what it would normally produce if it were being accurate.
Alex: That's the mechanism. And once they have that model in place, they can calculate something called Lorden's lower bound — which is the theoretical minimum number of tokens the system would ever need to see before it could reliably detect an error, no matter how good the detector.
Sam: And how small is that theoretical minimum?
Alex: At a very low false-alarm rate, the mathematical floor works out to roughly one to two tokens. That's essentially immediate — the AI has barely started a bad sentence before, in theory, a perfect detector could catch it.
Sam: But the real system isn't hitting that limit.
Alex: Not yet. Their CUSUM-based detector catches errors in around eleven to thirteen tokens — which is genuinely faster than conventional approaches, but still well above the theoretical floor. And the paper is honest about why that gap exists.
Sam: What's the explanation?
Alex: The paper describes it as an information deficit. The features the system is reading — the signals it extracts from the text — don't carry the full picture of what's happening. It's like trying to read a sign through frosted glass. You can make out that something is written there, but the detail is lost. And no matter how long you stare, you can't recover information that was never clearly visible to begin with. The delay isn't a processing problem — it's a signal quality problem.
Sam: So the bottleneck isn't the speed of the alarm. It's the clarity of what the alarm is listening to.
Alex: Exactly. The paper frames this as the key open challenge. The CUSUM framework is sound, and the theoretical limit tells us how much room there is to improve. Closing that gap means finding richer, more informative features — better ways of reading the signal that the AI is producing. The math shows the destination. Getting there is the next problem to solve.
Sam: It's a bit like having a perfectly calibrated smoke detector, but the smoke itself is hard to see.
Alex: That's a fair way to put it. And what's useful about this paper is that it gives researchers a precise way to measure how far any given detector is from the best it could theoretically be. That kind of benchmark is what turns an open-ended engineering problem into a tractable one.
Sam: I'll admit, I hadn't thought about AI errors as something you could catch mid-stream. The idea that there's a mathematically defined earliest possible moment of detection — that's worth sitting with.
Alex: It is. And it reframes the whole problem. Rather than asking "did this output contain errors," we start asking "how quickly can we know." That's a more useful question when these systems are being used in real time, in high-stakes settings. Thanks for listening to ResearchPod.