SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation

Autoregressive models treat images as flattened 1D sequences of tokens, a design choice inherited from language modeling that ignores the intrinsic 2D spatial geometry of visual data. This mismatch creates a severe inference bottleneck, as generating an image requires thousands of sequential forward passes. The authors investigate whether aligning the predictive objective with 2D spatial locality can overcome this bottleneck and enable faster, high-resolution visual generation.

The authors introduce Spatially Speculative Decoding (SSD), a framework that replaces standard 1D next-token prediction with 2D spatial anticipation. Instead of predicting only the next token in a raster-scan sequence, SSD uses lightweight auxiliary heads to simultaneously predict adjacent horizontal tokens and tokens in the row directly below. By factorizing 2D anticipation into orthogonal horizontal and vertical streams, the model drafts entire spatial blocks in parallel. To improve accuracy, the authors predict continuous latent features rather than discrete tokens, and they implement a verification-as-auto-correction mechanism that allows the model to repair local discrepancies within a drafted block without discarding the entire sequence.

SSD significantly accelerates inference across multiple state-of-the-art autoregressive models, including Janus-Pro, Lumina-mGPT, and Emu3. By shifting from $O(n^2)$ sequential complexity to $O(n)$ for an $n \times n$ image, the approach achieves speedups of up to 13.3x in wall-clock time. Crucially, this acceleration is achieved while maintaining high fidelity on benchmarks like DPG-Bench and GenEval, demonstrating that respecting the 2D geometry of vision is a more efficient strategy than forcing images into a 1D linguistic structure.