Text-driven 3D generation has advanced rapidly in creating large-scale outdoor environments and detailed indoor scenes, but these domains are usually synthesized independently, lacking the correspondence required for a coherent urban world. We present HoloWorld, a unified indoor-outdoor urban world generation framework built on a continuously updated cross-scale world context. Initializing from a user description, HoloWorld progressively represents and updates the diverse world information, from city-scale planning to individual buildings, allowing generated interiors to maintain explicit correspondence with their associated exterior buildings. Conditioned on the evolving context and previously generated neighboring blocks, HoloWorld autoregressively generates urban exteriors with consistent spatial organization and visual identity across blocks. The generated exterior representations are further grounded in 3D building instances and footprints, enabling building-specific indoor generation with geometry-constrained layouts and inherited appearance characteristics. To our knowledge, HoloWorld is the first framework to unify indoor and outdoor generation within a coherent 3D urban world. Extensive experiments demonstrate that HoloWorld achieves superior urban exterior generation performance, improving the average AQS score over the SOTA by 7.68\% and obtaining the highest average RDR score, while maintaining strong building-level indoor-outdoor correspondence and cross-block continuity within a unified 3D urban world. Our project page: https://huangxb326.github.io/HoloWorld/.
Alex: Welcome to another episode of ResearchPod. Today we're looking at HoloWorld, a framework that takes on a long-standing structural gap in 3D generative modeling: the disconnect between urban-scale exterior generation and detailed indoor layouts.
Sam: So we can generate convincing city blocks, and we can generate intricate room layouts — but those two pipelines don't talk to each other?
Alex: Right. Current approaches treat them as independent tasks. You can generate a city block, but the buildings are hollow shells — the exterior geometry carries no information about what the interior should look like, and vice versa. There's no shared representation enforcing consistency across scales.
Sam: And HoloWorld's answer is to build that shared representation explicitly — a persistent context that flows from the city plan down to the individual room?
Alex: That's the central architectural claim. They call it a cross-scale world context — a running state that encodes geometric constraints at every level of the hierarchy, from block-level urban planning down to interior room layout. Critically, this context isn't just passed top-down once. It's maintained and updated as generation proceeds, so each new element is conditioned on everything that came before it.
Sam: How does that actually work when you get to a specific building? The footprint is a hard geometric constraint — the interior has to fit inside it.
Alex: That's exactly where their instance grounding mechanism does the work. Rather than treating the global context as a soft prior, they project it onto each specific building instance and treat the 2D footprint as a non-negotiable boundary condition. The interior layout is then conditioned on that footprint's identity — not on a generic building type. So you're not generating "a plausible apartment." You're generating the apartment that belongs inside this particular shell.
Sam: And that's what drives the geometric alignment numbers?
Alex: It's the load-bearing mechanism behind their headline result. They report a Shape IoU of just under 1.0 between exterior footprint and interior layout — essentially perfect overlap. That figure is the direct consequence of treating the footprint as a hard constraint rather than a soft one, and it's the result the paper's central claim rests on.
Sam: That's a strong number — but I'd want to know what it costs in diversity. If everything is tightly conditioned on a global context, are you just generating the same interior over and over?
Alex: That's the obvious referee objection, and the authors address it through autoregressive conditioning on neighboring blocks. Rather than conditioning each building only on the top-level city plan, the system also conditions on its spatial neighbors — so local variation emerges from local context, while global coherence is enforced by the shared state. It's structurally similar to how language models maintain long-range dependencies while still generating locally varied text.
Sam: So diversity isn't sacrificed — it's operating at a different scale. The city-level plan constrains the broad strokes, but neighbor-to-neighbor conditioning keeps individual buildings from being identical.
Alex: That's the argument. Whether it fully holds up empirically is a fair question — the paper doesn't exhaustively characterize the diversity-coherence trade-off across a large sample of generated cities, which is the gap a careful reviewer would flag.
Sam: What's the practical ceiling on this approach? Where does it break down?
Alex: The most plausible bottleneck is the context bank itself. As you scale to larger cities, maintaining and querying that shared state grows in complexity. The authors manage this with grid partitioning — dividing the city into spatial chunks and propagating context only within and between adjacent chunks. That keeps the problem tractable, but it also means truly long-range dependencies across distant city blocks may not be fully captured. It's a reasonable engineering trade-off, but it's worth being explicit that cross-scale coherence has a spatial radius, not an infinite reach.
Sam: So the framework is genuinely tackling a real structural gap — the hollow-building problem is a meaningful limitation in current pipelines — but the coherence guarantee is local-to-regional rather than city-wide in the strictest sense.
Alex: That's a fair summary of where the evidence lands. The instance grounding result is solid, the alignment metric is strong, and the autoregressive neighbor conditioning is a principled way to handle diversity. The open question is how the system behaves at scale — larger cities, more varied building typologies, longer-range spatial dependencies. That's the natural next test for this kind of framework, and it's notably absent from the current evaluation.
Sam: A meaningful step forward for simulation, urban modeling, and synthetic data generation — with the scalability question still open.
Alex: Exactly. HoloWorld makes a clear case that cross-scale coherence is achievable in principle. The instance grounding mechanism is the key contribution — it's what turns a soft architectural prior into a hard geometric guarantee. Whether that holds as cities get larger and more complex is the work still ahead. Thanks for listening to ResearchPod.