Current visual generation models are capable of producing high-quality content, yet they lack a coherent perception of the spatial structure. Existing generative novel view synthesis methods typically introduce explicit geometry priors, which enforce spatial consistency but inherently restrict generalization in large view changes. In contrast, recent interactive generative methods favor implicit scene modeling, offering greater flexibility at the cost of precise camera control and geometry consistency. In this paper, we propose MetaView, a diffusion-based monocular novel view synthesis framework that enables rendering under large view changes from a single image. Our key insight is to combine implicit geometry modeling with minimal yet essential explicit 3D cues: we incorporate implicit geometry priors from a feed-forward geometry perception network to regularize structure without imposing restrictive reconstruction pipelines, while leveraging metric depth to anchor the generation to a metric scale. This design allows MetaView to achieve both geometry consistency and precise controllability. Extensive experiments demonstrate that, under challenging monocular large viewpoint changes, MetaView significantly outperforms existing methods and exhibits superior generalization. Our code is publicly available at https://github.com/KlingAIResearch/MetaView.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a challenge that has long frustrated AI researchers: why do AI-generated 3D views often look like melting wax?
Sam: It's a notable problem. When you generate a new view of a scene from just a single photo, AI models often struggle to keep objects looking solid. If you move the camera, walls warp or furniture stretches like taffy. This paper, "MetaView," explores a way to fix that by changing how the AI understands the geometry of a room.
Alex: So the core question is: how do we get an AI to "see" the 3D structure of a space without it falling apart the moment we shift the camera angle?
Sam: Exactly. Previous methods either try to build a rigid 3D map—which is brittle and often fails—or they guess the structure purely from pixels. That second approach leads to what researchers call "scale drift."
Alex: "Scale drift"—that's when the size of objects changes inconsistently as you move around, right?
Sam: Precisely. If you walk toward a chair, it should get bigger in a predictable, steady way. With scale drift, the AI might make the chair grow too fast, or suddenly shrink it, so the whole scene feels like a funhouse mirror. The authors argue for a middle ground: don't force the AI to build a perfect 3D model, but don't let it guess blindly, either.
Alex: So how do they find that middle ground?
Sam: They use what they call an "implicit geometry prior." Think of it like giving a painter a rough pencil sketch of a room's depth before they start painting. The painter uses that sketch to get the perspective right, but they're still free to paint all the details naturally. In this case, the "sketch" comes from a separate, pre-trained AI that's already quite good at estimating depth from a single image.
Alex: So instead of the model building the 3D map itself, it's being "hinted" at by another system that already understands depth?
Sam: That's the core mechanism. They feed these depth hints into the generation model as tokens—think of them as little sticky notes attached to different parts of the image, saying "this patch is far away" or "this patch is close." That guidance helps the model maintain the relative structure of the room without needing to construct a rigid, fragile 3D frame from scratch.
Alex: But how does it know how to shift those hints if I want to see around a corner? How does camera movement get handled?
Sam: They encode camera information directly into the model's internal math. The camera's position and angle get injected into what are called the model's "attention layers"—the parts of the system that decide which pieces of information to relate to each other. They also add a dedicated channel specifically for depth, so that as the camera moves, the scale of objects stays consistent rather than drifting.
Alex: So it's like giving the AI two things at once: a set of instructions for how to transform the scene as the camera moves, and a depth map that keeps the objects from melting in the process.
Sam: You've got the logic exactly right. And the reason that combination works is that neither piece has to do the whole job alone. The depth hints don't need to be perfectly accurate—they just need to be good enough to keep the model honest about which parts of the scene are near and which are far. The camera encoding handles the transformation. Together, they produce much better consistency than either approach on its own.
Alex: And the underlying engine here is a "diffusion" model?
Sam: Yes—specifically, a Diffusion Transformer. Here's the intuition: imagine starting with a completely static-filled image, like a TV with no signal, and then gradually clearing away the noise until a coherent picture emerges. That's roughly how diffusion models work. They learn to reverse a process of adding random noise, step by step, until they recover a clean image. The "Transformer" part is an architecture that's particularly good at tracking relationships across a whole image at once, which is what you need to keep a scene coherent when the viewpoint changes drastically.
Alex: So the secret isn't some single clever trick—it's more about giving the model the right kind of guidance at the right level of detail.
Sam: That's the key insight. By treating geometry as a flexible hint rather than a strict rule the model must obey, MetaView stays adaptable enough to handle scenes it's never seen before, without turning furniture into wax. It's a meaningful step toward AI that can genuinely reason about three-dimensional space from a flat photograph. Thanks for listening to ResearchPod.