NeoMap: Training-free Novel-View Synthesis from Single Images and Videos

Novel view synthesis (NVS) from monocular inputs is a fundamental challenge in computer vision, complicated by occlusions, depth ambiguity, and the need for temporal consistency. Existing methods often rely on task-specific fine-tuning or complex, step-wise guidance, which can introduce artifacts or overfit to specific camera trajectories. NeoMap challenges this paradigm by proposing that high-quality novel view solutions are already encoded within the latent space of pre-trained video generation models. Instead of modifying the model or the denoising trajectory, NeoMap treats NVS as a search problem: finding the optimal initial noise latent that, when processed by the pre-trained model, naturally decodes into a geometrically consistent and photorealistic video.

NeoMap introduces a manifold alternating projection mechanism to discover this optimal initial noise. The framework consists of three primary modules:

• Novel View Prior Module: Uses depth and pose estimation to warp source content into the target view, creating a geometric prior that serves as a base constraint.
• Anchored Manifold Projection (AMP): Iteratively pushes the latent state toward the natural video data manifold while anchoring reliable visible features, allowing the model to hallucinate plausible content in occluded regions.
• Pixel-Constrained Projection (PCP): Enforces strict geometric boundaries by decoding the latent state to pixel space, blending it with the warped prior, and re-encoding it. This ensures that visible regions remain geometrically immutable, correcting spatial bleeding induced by latent-level operations.

By alternating between these projections, the framework mathematically guarantees convergence to an optimal latent state that satisfies both global semantic realism and precise 3D view alignment.

NeoMap provides a robust, training-free solution that significantly outperforms existing methods across standard benchmarks like Tanks-and-Temples, LLFF, and DAVIS. By avoiding task-specific retraining, it maintains the generalization capabilities of foundational video models while achieving state-of-the-art generation fidelity and view consistency. This approach offers a scalable path for 3D/4D content creation and AR/VR applications without the heavy computational burden of fine-tuning.