Ensuring factuality remains a critical challenge for deploying LLMs in high-stakes settings. Existing hallucination detectors usually operate at a single level: claim-level methods provide interpretable factual units, while span-level methods localize unsupported text. Bridging these views is costly, as LLM-heavy pipelines require multiple decomposition and verification calls, and modular systems need additional claim-to-span alignment. We propose Enoki, an Open Information Extraction framework for multi-level hallucination detection. Enoki extracts text-anchored relational facts, verifies them against evidence, and projects unsupported facts back to hallucinated spans. This shared representation enables claim-level verification and span-level localization without requiring separate alignment. Enoki supports LLM-based, encoder-based, and rule-based extraction regimes, balancing accuracy and inference cost through a common interface. Experiments show that Enoki remains competitive with strong claim-level systems while using fewer resources and achieves superior performance on fine-grained span- and entity-level localization. We also release EnokiQA, a dual-granularity dataset with aligned claim-level verification and span-level localization annotations.
Alex: Welcome to another episode of ResearchPod. Today we're looking at ENOKI, a framework for hallucination detection in large language models — and it's attacking a problem that's been quietly frustrating the field for a while.
Sam: Right. The core tension is this: existing detectors split into two camps. Claim-level systems verify whether a proposition is factually supported, but they can't tell you which words in the output are responsible for the error. Span-level systems can localize the problem in the text, but they lack the factual grounding to say why something is wrong. You get precision or you get interpretability — not both.
Alex: And ENOKI is trying to collapse that distinction?
Sam: Exactly, and the mechanism is worth understanding carefully. They use text-anchored Open Information Extraction — OIE — to decompose generated text into subject-predicate-object triples. That part isn't novel. What's novel is the anchoring constraint: every argument in a triple must be a verbatim span from the source text. So you're not extracting abstract propositions — you're extracting relational facts that are literally grounded in specific words.
Alex: Which means when you verify a triple against evidence, you already know which tokens to flag.
Sam: Precisely. There's no separate alignment step, no downstream module trying to map a failed claim back onto the surface text. The localization is a byproduct of how the facts were extracted in the first place. Think of it as building the highlighter into the fact-checker rather than bolting it on afterward.
Alex: That's a meaningful architectural choice. What does it cost you?
Sam: The main cost is in the extraction itself, and this is where the engineering gets interesting. They offer three backends with different points on the accuracy-latency curve. The LLM-based backend gives you the highest extraction quality. The rule-based backend is fast but brittle. And then there's the encoder-based backend — the one they develop most carefully — which required a non-trivial training fix to get working properly.
Alex: What was the problem?
Sam: The issue is what they call incremental fact construction. When you extract facts from a sentence, you often get multiple valid triples that differ only in specificity — a coarser fact and a more granular one that extends it. If you train with a standard row-wise loss, the model gets penalized for predicting a correct triple in the wrong position in the output sequence. The loss function treats ordering as meaningful when it isn't.
Alex: So you're penalizing the model for a correct answer.
Sam: Exactly. Their fix is a permutation-invariant matching loss — you compute the optimal assignment between predicted triples and ground-truth triples before scoring, so the model is evaluated on what it extracted, not where it placed it in the sequence. It's a targeted fix, but it's load-bearing for the encoder backend's performance.
Alex: How does the full system hold up against existing detectors?
Sam: The headline result is that ENOKI is competitive with the strongest claim-level baselines on factual verification while being substantially faster — orders of magnitude for the encoder and rule-based backends. But the more distinctive finding is on span localization. Because the pipeline is unified rather than staged, it avoids the error propagation you get when a claim-level system hands off to a separate localization module. Each stage in a multi-stage pipeline can amplify upstream mistakes; ENOKI sidesteps that by design.
Alex: Is there a meaningful limitation the authors flag?
Sam: The honest constraint is extraction quality. The text-anchoring requirement is what gives you localization for free, but it also means the system can only represent facts that are literally expressible as spans in the source. Implicit claims, paraphrased content, or facts that emerge from combining multiple sentences are harder to capture cleanly. The LLM backend handles this better, but at the cost of latency — which partially undermines the efficiency argument.
Alex: So the trade-off between backends isn't just speed versus accuracy — it's also about what kinds of hallucinations the system can even detect.
Sam: That's a fair way to put it. And it's the kind of thing a careful referee would push on. The benchmark results probably favor cases where hallucinations are explicit and localizable, which may not reflect the distribution of errors in deployed systems. If your model is hallucinating through subtle paraphrase or multi-sentence inference, the anchoring constraint works against you.
Alex: So the contribution is real, but the scope conditions matter. A unified representation that makes localization structural rather than post-hoc — with a training fix that makes the encoder backend viable — but evaluated on a slice of the hallucination space that may be cleaner than what you'd see in production.
Sam: That's the right read. The open question is how much of the hallucination distribution actually falls within what the anchoring constraint can represent. That's not answered here, and it's probably the most important follow-on question for anyone thinking about deploying this.
Alex: Thanks for walking through it, Sam. And thanks to everyone listening — this has been ResearchPod.