Where Quality Breaks in Compressed Short-Text Generation: Staged Bottleneck Localization

Compressed latent pipelines are increasingly popular for short-text generation because they enable parallel decoding and reduce the categorical burden of long sequences. However, these systems often fail in two distinct ways: the codec may discard information during compression, or the latent generator may produce poor-quality codes. This paper investigates which of these stages sets the practical quality ceiling in a hierarchical VQ-VAE-2 and masked discrete diffusion (MDLM) pipeline.

The authors propose a staged validation protocol to isolate failure modes. By using a shared external GPT-2 scorer, they evaluate the system in three distinct phases: codec reconstruction fidelity, latent generation quality, and auxiliary latent diagnostics. This approach allows researchers to distinguish between representational loss (codec) and generation loss (latent model). The study uses a 64-to-16 compression ratio on the TinyStories dataset to compare autoregressive baselines, token-space diffusion, and code-space diffusion.

The study reveals that the codec is the dominant source of quality loss. Codec reconstruction alone increases median external perplexity by 80.4% and p95 by 294.1%, indicating that the system's fidelity is severely limited before the generator even begins its work. While code-space MDLM outperforms token-space diffusion—reducing mean perplexity by 32.9%—it cannot recover the information already lost during the initial compression. Furthermore, the authors find that geometry-aware regularization improves local latent proxies but fails to translate into measurable gains in decoded text quality, serving as a cautionary example against over-relying on latent-space diagnostics.

This research provides a clear engineering roadmap for developers of compressed text generators. It demonstrates that optimizing latent generators is often futile if the underlying codec is the primary bottleneck. By emphasizing the importance of paired reconstruction statistics and tail-aware metrics (like p95), the authors provide a reusable diagnostic framework that helps teams avoid spending compute on the wrong pipeline component.