DBLast: Dependent Block Drafting for Stochastic Speculative Decoding

Speculative decoding relies on a lightweight 'drafter' model to propose tokens for a larger target model to verify. Most current block-based drafters assume that tokens within a proposed block are conditionally independent to maintain parallel generation speed. However, this assumption fails in stochastic, non-greedy decoding scenarios where the target model's output is diverse, leading to poor acceptance rates. This paper investigates how to introduce token dependencies into block-based drafters without sacrificing the efficiency of parallel prediction.

The authors introduce DBLast, which modifies existing block-diffusion drafters in two ways. First, they add a categorical latent variable to the drafter architecture. This allows the model to predict a 'branch' or 'mode' of continuation; conditioned on this latent category, all tokens in the block are still generated in a single parallel pass, but the marginalization over categories induces dependencies between the tokens. Second, they propose an 'acceptance-oriented' training objective. Instead of standard negative log-likelihood (NLL), which only measures token probability, this objective uses a surrogate loss that directly rewards the drafter for producing sequences that the verifier is likely to accept.

Experiments using Qwen3-4B and Qwen3-8B models across various benchmarks (GSM8K, MT-Bench, HumanEval) demonstrate that DBLast consistently outperforms independent block-sampling baselines. The improvements are particularly significant in high-entropy, non-deterministic decoding regimes, where the gap between independent and dependent modeling is widest. The authors show that both the latent-variable dependency modeling and the acceptance-oriented training objective contribute independently to performance, with their combination yielding the highest accepted lengths—achieving macro-average gains of over 12% in high-entropy settings.