MITIGATING COMPOUNDING ERROR VIA VIDEO REPRESENTATION REGULARIZATION

Autoregressive video diffusion models often suffer from 'compounding error' or 'drift,' where small inaccuracies in early generated frames accumulate, leading to rapid visual degradation during long-horizon generation. This paper investigates the underlying mechanism of this phenomenon and explores why standard scaling paradigms—such as increasing training data—fail to resolve it.

The authors analyze the internal dynamics of video world models by tracking the 'effective rank' (erank) of hidden representations within the Diffusion Transformer (DiT) backbone. They discover that the onset of visual drift is tightly coupled with a sharp decrease in this effective rank, indicating a collapse in the model's representational capacity. To address this, they propose Video Representation Regularization (VRR), a training-time constraint that forces the model to maintain more expressive and uniform latent representations, preventing the model from relying on trivial 'shortcut' behaviors like frame-copying.

The study reveals that increasing the amount of training data does not improve resistance to error accumulation; in fact, it often accelerates the degradation of internal representations. By applying VRR, the authors demonstrate that they can maintain higher effective rank throughout the training process. This leads to substantial improvements in long-term generation stability, outperforming existing methods like Diffusion Forcing and Frame Anchoring on VBench metrics, specifically in Aesthetic and Imaging Quality.

This work provides a new, quantitative diagnostic tool (erank) for identifying why video models fail during long-term rollout. By shifting the focus from data quantity to the quality of internal representations, the authors offer a practical architectural intervention that challenges the conventional wisdom that more data is always better for robustness in generative world models.