TIGER: Text-conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding

Multimodal speculative decoding often underperforms compared to text-only models because lightweight drafters frequently diverge from the verifier when processing vision-critical content. The authors investigate how to improve the efficiency of these systems by addressing two primary bottlenecks: the exposure of the drafter to irrelevant visual information and the misalignment between standard imitation-based training and the runtime metric that actually governs speedup—the verifier-accepted prefix length.

The authors propose TIGER (Text-conditioned vIsual GatEd Routing), a framework that introduces two main innovations. First, instead of providing the drafter with the full set of visual tokens, TIGER uses a text-conditioned routing mechanism that projects the current textual hidden state to score and select a sparse, context-relevant subset of visual tokens for each speculative block. Second, the authors introduce an acceptance-aligned training objective. After a standard distillation warm-start, the drafter is trained using a group-based policy optimization (GRPO) approach where rewards are derived directly from the number of tokens the verifier accepts, rather than simple token-level imitation. This encourages the drafter to generate sequences that are more likely to survive the verifier's acceptance test.

Experiments across various benchmarks (including MMBench, ScienceQA, and TextVQA) demonstrate that TIGER consistently increases the average verifier-accepted prefix length compared to existing multimodal speculative decoding baselines. By aligning the drafter's visual interface and training objective with the verifier's acceptance mechanism, TIGER achieves improved end-to-end speculative speedups while maintaining comparable downstream task accuracy. The authors show that these gains are robust across different model architectures, including the LLaVA and Qwen-VL families.