GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis

GrainGS is a dynamic 3D Gaussian Splatting (3D-GS) framework designed to reconstruct and render dynamic scenes with high fidelity and efficiency. Traditional dynamic 3D-GS methods often struggle with 'gradient entanglement,' where the optimization of frame-specific motion and appearance inadvertently corrupts the time-invariant canonical geometry. GrainGS addresses this by decoupling these processes through architectural and procedural constraints.

The framework operates in two distinct phases. First, a static warm-up stage optimizes a hierarchical anchor scaffold across all timestamps without any deformation, establishing a stable, time-invariant geometric reference. Second, during joint training, the model employs a stop-gradient operator that blocks the backpropagation of deformation-related gradients into the canonical scaffold. This ensures that the canonical geometry is updated solely by the reconstruction objective, while a separate DeformNet learns independent temporal offsets for each Gaussian. Additionally, the paper introduces a canonical-residual appearance decomposition, which models frame-dependent photometric changes (like lighting or shadows) separately from geometric deformation, further reducing the pressure on the deformation field.

By isolating the canonical representation from the temporal deformation field, GrainGS achieves a superior balance between motion expressiveness and structural stability. This decoupling allows the model to maintain a compact representation while accurately capturing fine-grained local motion. Experimental results demonstrate that GrainGS achieves state-of-the-art performance on synthetic and real-world benchmarks, offering high peak signal-to-noise ratios (PSNR) and real-time rendering speeds (over 400 FPS) with significantly lower storage requirements compared to existing dynamic 3D-GS methods.