View-adaptive Renderer for View-consistent 2D-to-3D Generation

How can we improve the accuracy of 3D reconstruction from a single 2D image when the intermediate multi-view images generated by current models contain visual inconsistencies and projective ambiguities? The authors seek a way to produce geometrically coherent 3D models without the heavy computational cost of traditional diffusion-based regularization techniques.

The authors propose a view-adaptive neural rendering framework that modifies the standard Neural Radiance Field (NeRF) pipeline. Instead of using a single global MLP to represent the entire scene, the model decomposes the network into two parts: a shared global feature backbone and specialized view-adaptive neural renderers. Each viewpoint is assigned its own dedicated MLP branch, allowing the model to learn and correct viewpoint-specific errors independently. To ensure these individual corrections do not lead to structural divergence, a self-attention fusion module is employed. This module aggregates features across multiple views, dynamically weighting their contributions to maintain global geometric coherence. The framework is trained using photometric rendering loss combined with lightweight attention regularizers, such as entropy and consistency losses, to stabilize the learning process.

The proposed method demonstrates that decoupling viewpoint-specific corrections from global scene features significantly improves 3D reconstruction fidelity. By allowing each view to adaptively rectify its own inconsistencies, the model produces sharper surfaces and more accurate geometry compared to standard NeRF baselines. Furthermore, the framework achieves performance competitive with state-of-the-art methods while avoiding the high computational overhead associated with Score Distillation Sampling (SDS) or other intensive indirect supervision techniques. This makes the approach more efficient and practical for real-world 3D content creation.