SafeSpec: Fast and Safe LLM via Dynamic Reflective Sampling

As large language models (LLMs) are increasingly deployed with speculative decoding to reduce latency, a critical gap has emerged: existing safety defenses are either too computationally expensive or incompatible with the draft-verify mechanism. The authors investigate whether it is possible to achieve both high-speed speculative inference and robust safety guardrails simultaneously, rather than treating them as competing objectives.

The authors propose SafeSpec, a framework that embeds safety verification directly into the speculative decoding loop. Instead of using external, high-latency safety filters, SafeSpec attaches a lightweight, two-layer MLP safety head to the target model's final layer. This head evaluates the safety of candidate segments in a single forward pass by probing latent representations, which are shown to be linearly separable for safety concepts. When the system detects an unsafe segment, it triggers a 'Safety Mode' that performs a rollback to a previous state, injects a reflection prompt to re-orient the model, and utilizes multi-sampling to find a safe continuation. This approach treats jailbreak attempts as a shift in probability mass, where the goal is to recover safe trajectories rather than simply terminating the generation.

SafeSpec demonstrates that safety and speed can be jointly optimized. On the Qwen3-32B model, the framework achieved a 2.06x speedup on benign workloads while simultaneously reducing attack success rates (ASR) by 15% compared to baseline speculative decoding. By framing safety intervention as a probabilistic recovery task rather than a hard block, the system maintains high utility and coherence even when faced with adversarial prompts that attempt to bypass safety alignment.

This research provides a practical path for deploying safe LLMs in latency-sensitive environments. By moving safety checks into the latent space and integrating them into the speculative decoding process, developers can avoid the 'alignment tax'—the performance degradation often associated with aggressive safety tuning—while ensuring that the model remains resilient against sophisticated, trajectory-based jailbreak attacks.