Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique

Research Question and Approach

AI agents often suffer from specification-grounding failures, where they confidently execute actions not requested by the user. Existing detection methods—such as external judges or semantic entropy—are either too slow for real-time use or fail to localize the specific hallucinated parameter. The authors introduce the Latent Critic, a lightweight Low-Rank Adaptation (LoRA) module that runs concurrently with a frozen base LLM. Instead of analyzing output text, the Critic modifies the model's internal residual stream to amplify latent uncertainty signals, allowing it to identify and verbalize the exact source of a hallucination within the same generation sequence.

The Latent Critic demonstrates superior performance in isolating hallucinations compared to standard baselines. On Qwen-based models, it achieves an AUROC of 0.966 and over 80% accuracy in localizing specific ungrounded parameters. Mechanistic analysis reveals that the adapter restructures the model's internal geometry, transforming entangled uncertainty signals into a linearly separable representation. This approach is highly robust; while passive internal probes collapse under out-of-distribution (OOD) shifts, the Latent Critic maintains high performance by exposing a more stable, shift-invariant geometric signal.

This work bridges the gap between mechanistic interpretability and practical agent safety. By embedding detection directly into the generation process, the Latent Critic acts as a negligible-latency guardrail. This allows agents to intercept undesired actions before they are executed, enabling efficient self-correction without the prohibitive computational overhead of secondary inference loops or external verification models. It provides a scalable path toward deploying reliable AI agents in high-stakes environments where specification adherence is critical.