PanoImager: Geometry-Guided Novel View Synthesis and Reconstruction from Sparse Panoramic Views

3D reconstruction from sparse panoramic images is notoriously difficult due to rotation-dominant motion and weak parallax, which often cause traditional Structure-from-Motion (SfM) or SLAM pipelines to fail during initialization. PanoImager addresses this by developing a framework that does not rely on traditional feature-based triangulation, instead leveraging generative priors to densify sparse observations and stabilize 3D Gaussian optimization.

The authors propose a multi-stage pipeline that avoids the pitfalls of global panoramic distortion. First, the system decomposes panoramic images into a set of overlapping local perspective views (tangent-space charts), which are more compatible with standard pinhole camera models. A visual foundation model then predicts initial camera poses and depth maps for these charts.

To overcome the lack of viewpoint diversity, the framework employs a geometry-conditioned diffusion model. This model synthesizes auxiliary views based on warped RGB-D priors and visibility masks, effectively filling in unobserved regions. Finally, these synthesized views are integrated into a 3D Gaussian Splatting (3DGS) optimization process. Crucially, the synthesized data is treated as a reliability-weighted soft prior rather than a hard measurement, which helps mitigate the impact of potential hallucinations or geometric inaccuracies.

Experiments across multiple benchmarks demonstrate that PanoImager significantly improves reconstruction stability and visual quality in extreme sparsity regimes where conventional methods fail to initialize. The use of tangent-space decomposition and reliability-aware soft supervision allows the model to maintain structural integrity and spatial consistency even when input data is highly limited. The results suggest that PanoImager is a viable offline or background component for map refinement in robotic applications where traditional SLAM initialization is ill-conditioned.