Calibrated e-CUSUM Decoding for Quantized Reasoning Models: Why Token Log-Probability Is the Wrong Observable for Decoding Monitors

As low-bit quantization becomes a standard method for deploying reasoning models, these models often suffer from degraded chain-of-thought performance. The authors investigate whether it is possible to build a decoder-side monitor that detects when a model is going off-track during generation, allowing for real-time intervention before the final answer is produced.

The authors first analyze the theoretical properties of using token log-probabilities as a monitoring signal. They show that the standard approach—centering log-probabilities by subtracting entropy—creates a martingale that measures sampling self-consistency rather than the health of the reasoning trajectory. Because this signal is near zero during confident, repetitive loops, it fails to detect the very errors it is meant to catch.

To address this, they develop a two-part controller:

1. Degeneration-aware alarm score: A metric that combines normalized token entropy with an explicit verbatim-repetition signal, allowing the monitor to detect loops that log-probability-based methods miss.
2. Calibrated e-CUSUM detector: A sequential change-detection mechanism that uses a betting-based construction to monitor the alarm score, calibrated against healthy reference traces to ensure selectivity.

The study finds that the proposed controller successfully shifts the monitor from an indiscriminate alarm (firing on 93-95% of traces) to a selective detector of failing trajectories. While the controller effectively reduces verbatim repetition and non-termination, the impact on final accuracy remains statistically inconclusive in this pilot study. Furthermore, the authors identify that for the tested quantized models, the primary failure mode is not repetitive looping, but rather non-termination—where the model exhausts its token budget without reaching a conclusion.