PRG-Fusion: Orchestrating Generative Priors with Reconstruction Evidence for Driving View Synthesis

Synthesizing photorealistic driving videos along novel trajectories is critical for closed-loop autonomous driving simulation. However, existing methods struggle to balance the geometric consistency of reconstruction-based techniques (like 3DGS) with the generalization capabilities of generative models. When viewpoints deviate from training data, 3DGS often produces artifacts or holes, while generative models may suffer from temporal drift and structural inconsistency. This paper asks: can we adaptively coordinate these two paradigms based on the local reliability of reconstruction evidence?

The authors propose PRG-Fusion, which introduces a region-aware routing policy. First, the framework extracts reconstruction evidence from the 3DGS scene to generate pixel-wise labels: Preserve (reliable regions), Repair (degraded but structurally supported regions), and Generate (unsupported regions). These labels act as a routing policy for a dual-branch control architecture. A visual branch encodes 3DGS appearance, while a geometry branch encodes LiDAR-based structural constraints. The model is trained in two stages: first establishing geometric control via LiDAR, then learning appearance control from 3DGS renderings. This staged approach prevents the model from over-relying on visual artifacts.

PRG-Fusion achieves state-of-the-art performance on the Waymo Open Dataset for novel trajectory synthesis. Quantitative results show superior visual quality (FID) and geometric fidelity (NTA-IoU and NTL-IoU) compared to existing reconstruction-based, generation-based, and restoration-based methods. Crucially, the PRG-guided fusion strategy effectively maintains scene semantics and temporal consistency, even under large lateral trajectory shifts (up to 6 meters), where competing methods often exhibit flickering or structural drift.