Chunchao Guo, Jinpeng Li, Yang Li, Zilong Huang
6 min
Abstract
Generating large-scale, freely explorable 3D worlds from open-ended text remains challenging because a system must jointly maintain global spatial coherence, rich local content, and explicit assets suitable for downstream editing and reuse. We present WorldClaw, a fully agentic, coarse-to-fine framework for open-world 3D scene generation. Planning agents translate a text prompt into a structured specification of regions, terrain, assets, materials, and spatial relations. WorldClaw then builds a globally coherent terrain foundation from semantic layouts, reusable assets, generative or procedural materials, and a region-aware height field. For detail-demanding regions, it generates terrain-conditioned compositions, reconstructs editable textured meshes, and recovers their placement on the terrain; render-based agents further refine terrain, objects, appearance, and contacts. Across diverse open-world prompts, WorldClaw produces large-scale scenes with coherent spatial organization, visually compelling local content, and editable instance-level assets while preserving a consistent global terrain structure.
Alex: And once it's built something, does it just hope for the best?
Sam: No — and this is one of the more practical parts of the design. The system runs what the paper calls an "agentic refinement loop." After placing an object, it checks its own work. Is this tree floating above the ground? Is this building scaled correctly for the surrounding terrain? If something is off, it fixes it automatically. It's essentially the system grading its own homework and correcting the errors before anyone else sees them.
Alex: That's a much more dependable approach than just hoping the AI gets it right on the first attempt.
Sam: It is. And the practical payoff is that because the terrain and the objects are kept separate throughout the process, a game designer or filmmaker could actually go in afterward and edit individual elements — move a building, change the shape of a hill — without the whole scene falling apart. It bridges the gap between a one-time generated image and something you could genuinely use in production.
Alex: How does it actually build the terrain geometry, though? Does it just estimate the heights?
Sam: It uses something called a "height field." Think of it as a grid — like graph paper laid over the landscape — where every single square on that grid has one number assigned to it, representing how high the ground is at that point. To fill in those numbers in a way that looks natural, the system combines mathematical noise patterns — which produce the kind of irregular, organic variation you see in real landscapes — with specific landform operators. Those are essentially shape templates: "peak," "dune," "plateau." Mixing them together produces terrain that feels continuous and varied rather than obviously computer-generated.
Alex: So it's drawing a topographic map from scratch, mathematically.
Sam: Exactly. And once the shape is there, the system uses a rendering method that simulates how light actually behaves in the physical world. Rather than just painting a surface to look like rock or water, it calculates how light would genuinely bounce off those materials. The result is that the scene holds up visually — it doesn't have that telltale flat, digital-painting quality.
Alex: And placement — how does it make sure a building is actually sitting on the ground rather than floating above it?
Sam: It uses a technique called ray-casting. The system fires an invisible line straight down from the base of the object toward the terrain surface, then calculates the exact point where that line makes contact. The object is then moved to sit precisely at that intersection. If there's still a mismatch — a small gap or an awkward angle — it either adjusts the object's position or smooths out the terrain underneath it. The end result is that every object sits in a way that's physically plausible.
Alex: So the whole system is really about keeping two things in their proper order — the big picture first, the fine details second — and that discipline is what prevents the scene from breaking down.
Sam: That's the core insight. By anchoring every local object to a pre-planned global terrain, the system avoids the drift that tends to undermine large-scale generation in other approaches. The result isn't just a picture — it's a structured, editable environment. Whether that ultimately proves useful across a wide range of production contexts is still an open question, but as a framework for thinking about the problem, it represents a meaningful step forward.
Alex: Thanks for walking us through that, Sam. And thanks to everyone listening to ResearchPod.