Elisei Rykov, Timur Ionov, Nikolay Ivanov, Maksim Savkin, Maksim Makarenko, Alexander Panchenko, Vasily Konovalov, Julia Belikova
5 min
Abstract
Ensuring factuality remains a critical challenge for deploying LLMs in high-stakes settings. Existing hallucination detectors usually operate at a single level: claim-level methods provide interpretable factual units, while span-level methods localize unsupported text. Bridging these views is costly, as LLM-heavy pipelines require multiple decomposition and verification calls, and modular systems need additional claim-to-span alignment. We propose Enoki, an Open Information Extraction framework for multi-level hallucination detection. Enoki extracts text-anchored relational facts, verifies them against evidence, and projects unsupported facts back to hallucinated spans. This shared representation enables claim-level verification and span-level localization without requiring separate alignment. Enoki supports LLM-based, encoder-based, and rule-based extraction regimes, balancing accuracy and inference cost through a common interface. Experiments show that Enoki remains competitive with strong claim-level systems while using fewer resources and achieves superior performance on fine-grained span- and entity-level localization. We also release EnokiQA, a dual-granularity dataset with aligned claim-level verification and span-level localization annotations.
Alex: How does the full system hold up against existing detectors?
Sam: The headline result is that ENOKI is competitive with the strongest claim-level baselines on factual verification while being substantially faster — orders of magnitude for the encoder and rule-based backends. But the more distinctive finding is on span localization. Because the pipeline is unified rather than staged, it avoids the error propagation you get when a claim-level system hands off to a separate localization module. Each stage in a multi-stage pipeline can amplify upstream mistakes; ENOKI sidesteps that by design.
Alex: Is there a meaningful limitation the authors flag?
Sam: The honest constraint is extraction quality. The text-anchoring requirement is what gives you localization for free, but it also means the system can only represent facts that are literally expressible as spans in the source. Implicit claims, paraphrased content, or facts that emerge from combining multiple sentences are harder to capture cleanly. The LLM backend handles this better, but at the cost of latency — which partially undermines the efficiency argument.
Alex: So the trade-off between backends isn't just speed versus accuracy — it's also about what kinds of hallucinations the system can even detect.
Sam: That's a fair way to put it. And it's the kind of thing a careful referee would push on. The benchmark results probably favor cases where hallucinations are explicit and localizable, which may not reflect the distribution of errors in deployed systems. If your model is hallucinating through subtle paraphrase or multi-sentence inference, the anchoring constraint works against you.
Alex: So the contribution is real, but the scope conditions matter. A unified representation that makes localization structural rather than post-hoc — with a training fix that makes the encoder backend viable — but evaluated on a slice of the hallucination space that may be cleaner than what you'd see in production.
Sam: That's the right read. The open question is how much of the hallucination distribution actually falls within what the anchoring constraint can represent. That's not answered here, and it's probably the most important follow-on question for anyone thinking about deploying this.
Alex: Thanks for walking through it, Sam. And thanks to everyone listening — this has been ResearchPod.