ResearchPod Summary
Video compression is essential for managing Internet traffic, but traditional codecs and standard neural video compression methods typically optimize for pixel-fidelity metrics like PSNR or mean squared error. This focus frequently results in blurry structures and visually unsatisfactory artifacts at low bitrates. To bridge the gap between distortion minimization and human perception, diffusion-based generative video compression (GVC) has emerged as a promising paradigm. These methods use generative priors to reconstruct realistic details, transmitting compressed latent representations that guide generative reconstruction at the decoder.
Despite their potential, existing diffusion-based GVC techniques often suffer from suboptimal latent organization. They typically rely on simple frame-by-frame reference chains and uniform quality allocation, or introduce quality variations without coupling them to reference roles. Consequently, important reference frames may not be preserved with high enough quality, and unrestricted cross-frame interactions during denoising can spread coding artifacts. This paper introduces GVCHR (Generative Video Compression based on Hierarchical Referencing), a unified framework that extends hierarchical referencing from latent coding to generative reconstruction.
The GVCHR framework begins by splitting a source video into keyframes and grayscale intermediate frames, reducing transmission overhead while retaining essential structural information. These frames are mapped into latent representations via a 3D VAE encoder. To efficiently compress these latents, GVCHR couples a hierarchical reference structure with a hierarchical quality structure. In this scheme, lower-layer latent frames are allocated more bits and are reused more frequently as references, whereas higher-layer frames are coded more compactly.
To exploit this structure, the authors introduce Hierarchical Temporal Context Mining (HTCM). HTCM extracts and fuses complementary short- and long-term temporal contexts from previously decoded features. The short-term reference captures local details from the nearest decoded latent frame at the same or lower layers, while the long-term memory summarizes global temporal semantics from the lowest-layer reference frames. This dual context significantly enhances conditional coding efficiency.
Once the latent frames are decoded, they serve as conditional guidance for a video diffusion transformer (DiT) during generative reconstruction. Instead of using full cross-frame attention, which can propagate errors from less reliable higher-layer frames, GVCHR introduces a Hierarchical Attentive Adapter (HA-Adapter).
The HA-Adapter enforces the coding-side hierarchy directly into the denoising process via hierarchical attention. Each latent frame is restricted to attend only to references from the same layer or lower layers. This constraint prevents higher-layer frames—which are compressed with fewer bits—from contaminating critical lower-layer reference frames, thereby suppressing artifact propagation and preserving structural fidelity.
Experimental evaluations on multiple benchmarks demonstrate that GVCHR achieves state-of-the-art performance in diffusion-based generative video compression. Compared with the previous leading method, GVCHR achieves substantial BD-rate reductions of 50.5% in LPIPS and 54.0% in DISTS, while delivering noticeably improved visual quality and reduced temporal flickering.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.