DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

Research Question and Approach

Large language model inference is often bottlenecked by sequential autoregressive generation. While speculative decoding uses a draft model to propose tokens in parallel, block-parallel drafters like DFlash often struggle with intra-block causal consistency—tokens within a block are predicted largely independently. Existing solutions to this problem, such as Domino or DSpark, typically require expensive joint training or retraining of the draft model from scratch.

DeLS-Spec addresses this by decoupling the drafting process into two experts: a fixed, pre-trained DFlash model acting as a long-context expert, and a lightweight local head acting as a short-context expert. The local head is trained independently using a standard next-token prediction objective on plain text, requiring no access to the target model or the DFlash backbone. During inference, the system fuses the logits from both experts, corrected by a unigram prior to prevent frequency bias.

Experiments on Qwen3-4B and Qwen3-8B models across math, code, and dialogue benchmarks demonstrate that DeLS-Spec consistently outperforms the baseline DFlash model. By introducing explicit causal dependencies within the draft block, DeLS-Spec increases the average acceptance length of tokens and improves overall decoding speedup. The modular design allows the same trained local head to be transferred across different DFlash checkpoints, offering significant flexibility compared to methods that require end-to-end retraining.

This approach provides a highly efficient and modular way to enhance existing speculative decoding pipelines. By avoiding the need for joint training, researchers can improve the performance of already deployed draft models with minimal computational overhead. This makes it a practical solution for accelerating inference in resource-constrained environments where retraining large draft models is prohibitive.