Struct-GStream: Towards Efficient Free-Viewpoint Video Streaming at Low-Bitrates with Structured 3D Gaussians

Struct-GStream is a novel framework designed for the efficient online construction and streaming of Free-Viewpoint Videos (FVVs). While existing neural rendering methods often struggle with the trade-off between high-fidelity rendering and the computational demands of online training, Struct-GStream leverages 3D Gaussian Splatting (3DGS) to achieve a balance of low-bitrate storage, fast training, and high rendering quality.

The core innovation lies in a dual-representation strategy. First, the system uses dynamic anchor points to generate "structured 3D Gaussians," which model the foundational, roughly rigid structure of the scene. By anchoring these Gaussians, the model significantly reduces the number of parameters needed to represent the scene's basic movement. Second, the framework employs a "global free 3DGs patching strategy." This mechanism continuously monitors the reconstruction quality; if a region is deficient or if new objects emerge in the scene, the system generates, prunes, and optimizes additional "free" 3D Gaussians to fill in the gaps. This allows the model to adapt to temporal changes without requiring a complete re-training of the entire scene.

Traditional FVV methods often require complete video sequences for training or suffer from slow rendering speeds and massive storage requirements. By enabling online training—where the model learns frame-by-frame—Struct-GStream is significantly more practical for real-time applications like virtual reality (VR) and augmented reality (AR). The experimental results demonstrate that this approach outperforms existing online methods in training time and storage efficiency while maintaining competitive rendering speeds, making it a viable candidate for bandwidth-constrained streaming environments.