Immediate 3D Gaussian Splat Reconstruction of Unordered Input with Global Consistency

Overview

This paper presents a novel approach to 3D Gaussian Splatting (3DGS) that enables immediate, globally consistent reconstruction from unordered image sequences. Traditional 3DGS methods typically require pre-computed camera poses from offline Structure from Motion (SfM) pipelines or assume sequential input, which limits their utility for interactive capture. This work addresses these limitations by integrating fast pose estimation, loop closure, and a hierarchical Gaussian representation into a single, real-time pipeline.

The core of the system is a two-level matching strategy that allows the system to handle images arriving in any order. First, it uses a visual place recognition model (MixVPR) to identify candidate keyframes, followed by a more precise geometric verification using local features (XFeat and LightGlue). This allows the system to maintain a weighted covisibility graph that tracks geometric relationships between all registered frames.

To ensure global consistency, the authors implement a cluster-based loop closure mechanism that detects when a user returns to a previously captured area. This mechanism updates poses and Gaussian primitives without requiring expensive, iterative global optimization. Finally, to scale to large environments, the authors introduce a progressive hierarchy that dynamically selects relevant keyframes and Gaussians, ensuring that GPU memory usage remains efficient while maintaining high-quality rendering.

By enabling immediate feedback for unordered captures, this method significantly lowers the barrier for high-quality 3D reconstruction. It allows users to verify the completeness and quality of their data in real-time, preventing the common issue of discovering missing coverage only after a long, offline processing session. The ability to handle thousands of images while maintaining global consistency makes this approach suitable for complex, large-scale scenes in fields like virtual reality, architecture, and visual effects.