4DHumanDiff: Direct Text-to-4DGS Generation for Consistent 360-Degree Dynamic Humans

Generating high-quality, 360-degree dynamic human assets from text prompts is a significant challenge in computer vision. Traditional methods typically rely on a two-stage pipeline: first synthesizing monocular or multi-view videos, and then performing a separate, computationally intensive 4D reconstruction process. This approach is often slow, prone to view-inconsistent renderings, and struggles to recover complete 360-degree geometry. The authors seek to determine if dynamic humans can be generated more efficiently and consistently by modeling the 4D representation space directly.

The authors introduce 4DHumanDiff, a diffusion framework that generates dynamic humans represented by 4D Gaussian Splatting (4DGS) end-to-end. To enable this, they constructed a large-scale dataset of 60,000 text-to-4DGS pairs. They developed a structured 4DGS representation that establishes a fixed spatial index for Gaussians at the first timestamp, which is then maintained across all subsequent frames to preserve temporal correspondence. The model utilizes a 3D U-Net backbone augmented with temporal attention layers to capture motion dynamics. Furthermore, the authors incorporate a 2D regularization term during training and a training-free 4D interpolation strategy to ensure high visual quality and smooth motion.

4DHumanDiff significantly outperforms existing video-first pipelines in both efficiency and consistency. By eliminating the need for intermediate video synthesis and per-scene optimization, the model reduces inference time by more than 10x, enabling the generation of 360-degree dynamic human assets in under one minute. Quantitative evaluations using VBench metrics demonstrate that the direct 4D generation approach yields superior temporal coherence and multi-view consistency compared to state-of-the-art methods that rely on video-to-4D reconstruction.