Jacky Kwok, Shulu Li, Pranav Atreya, Yuejiang Liu, Yixing Jiang, Chelsea Finn, Marco Pavone, Ion Stoica, Azalia Mirhoseini
5 min
As large language models (LLMs) scale, generation capabilities have improved significantly, but verification—the ability to accurately determine the correctness of a solution—remains a bottleneck. Standard approaches often rely on discrete scores from LLM judges, which suffer from high tie rates and poor discrimination. This paper introduces LLM-as-a-Verifier, a framework that treats verification as a distinct scaling axis. Instead of prompting for a single discrete score, the framework computes a continuous reward by taking the expectation over the distribution of scoring-token logits. This probabilistic approach allows for scaling along three dimensions: score granularity (using more tokens to represent finer levels), repeated evaluation (averaging multiple passes to reduce variance), and criteria decomposition (breaking complex tasks into simpler, verifiable sub-factors).
The authors demonstrate that scaling these three dimensions consistently improves verification accuracy. Finer score granularity improves the signal-to-noise ratio, repeated evaluations average out individual biases, and criteria decomposition reduces prompt bias. The framework is evaluated as a trajectory reward model across four domains: coding (Terminal-Bench V2, SWE-Bench Verified), robotics (RoboRewardBench), and medical (MedAgentBench). It achieves state-of-the-art results in all these benchmarks without requiring any domain-specific fine-tuning. Furthermore, the framework provides a dense reward signal that improves the sample efficiency of both off-policy (SAC) and on-policy (GRPO) reinforcement learning algorithms.
This work shifts the paradigm of agentic evaluation from coarse, discrete judgments to fine-grained, probabilistic verification. By providing a scalable, training-free mechanism for evaluating long-horizon trajectories, LLM-as-a-Verifier enables more reliable monitoring of autonomous agents and provides a robust dense reward signal for reinforcement learning. The ability to use these scores as a proxy for task progress also offers a practical tool for developers to track agent behavior in real-time, potentially preventing failures before they occur.
Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM-as-a-Verifier computes the expectation over the distribution of scoring token logits to generate continuous scores. This probabilistic formulation enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scaling repeated evaluation and criteria decomposition consistently lead to additional gains in verification accuracy through variance and complexity reduction. We further introduce a cost-efficient ranking algorithm for selecting the best solution among candidates using the verifier's continuous scores. LLM-as-a-Verifier achieves state-of-the-art performance on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine-grained signals from LLM-as-a-Verifier can also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show that LLM-as-a-Verifier can provide dense feedback for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.
Alex: That's a reasonable heuristic, though I'd want to see how sensitive it is to the initial ring ordering.
Sam: That's a fair concern the paper doesn't fully resolve. And it connects to the deeper limitation here — the verifier is only as good as the base model's internal calibration. If the model lacks domain-specific knowledge to distinguish a correct trajectory from a subtle failure, scaling granularity or repeated sampling will amplify noise rather than signal. You're extracting latent confidence, but you cannot manufacture knowledge that isn't there.
Alex: So in domains where the base model is systematically miscalibrated — certain edge cases in medical reasoning, or low-resource code libraries — the method could confidently endorse wrong answers.
Sam: That's exactly the failure mode the authors flag. The verifier can be confidently wrong when the base model itself is wrong in a consistent direction. It's an amplifier of the model's internal state, not a correction mechanism. Which is worth keeping in mind when you look at the benchmark results — those are evaluated on tasks where the base model has reasonable coverage. The method looks good precisely where the underlying model is already reasonably well-calibrated.
Alex: Given that constraint, where does the practical value actually land?
Sam: The near-term application the authors point to is something like a background verification process running alongside agentic tasks — continuously monitoring trajectories and providing dense reward signals rather than waiting for a terminal success or failure. The idea is that you catch and roll back errors before they compound, which matters a lot for long-horizon tasks where a single bad step early can corrupt everything downstream.
Alex: That shifts the architecture from episodic evaluation to something more like continuous oversight.
Sam: And that reframing is arguably the paper's most useful contribution — not any single technique, but the argument that verification deserves its own scaling roadmap. We've had years of work on how to make generation better. This is a case for investing equivalent effort on the critic side of the loop. Whether the specific methods here are the right ones is an open question, but the framing seems worth taking seriously.
Alex: A measured but meaningful shift in where we direct compute. Thanks for walking through the mechanics, Sam. That's our look at LLM-as-a-Verifier — thanks for listening to ResearchPod.