Web agents require massive trajectories to generalize, yet real-world training is constrained by network latency, rate limits, and safety risks. We introduce \textbf{WebWorld} series, the first open-web simulator trained at scale. While existing simulators are restricted to closed environments with thousands of trajectories, WebWorld leverages a scalable data pipeline to train on 1M+ open-web interactions, supporting reasoning, multi-format data, and long-horizon simulations of 30+ steps. For intrinsic evaluation, we introduce WebWorld-Bench with dual metrics spanning nine dimensions, where WebWorld achieves simulation performance comparable to Gemini-3-Pro. For extrinsic evaluation, Qwen3-14B trained on WebWorld-synthesized trajectories improves by +9.2\% on WebArena, reaching performance comparable to GPT-4o. WebWorld enables effective inference-time search, outperforming GPT-5 as a world model. Beyond web simulation, WebWorld exhibits cross-domain generalization to code, GUI, and game environments, providing a replicable recipe for world model construction.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at training computer programs to handle tasks on websites, like shopping or searching—things people do every day online.
Sam: These programs, often called web agents, use large language models to think through steps and take actions, such as clicking buttons or filling forms. But training them effectively requires lots of examples of real interactions—what researchers call trajectories, which are just sequences of what the agent sees, does, and what happens next. The problem is gathering those examples from actual websites.
Alex: Sequences like paths through a site? So why not just let the agents practice on live sites as much as needed?
Sam: Real-world practice runs into serious roadblocks. Websites load slowly because of network delays, many impose rate limits to stop too many requests, and some actions carry risks—like accidentally buying something or submitting personal info. Previous efforts were stuck with just thousands of examples from controlled test sites, lacking the variety of the open internet.
Alex: That sounds limiting. So these agents couldn't learn to handle long, realistic sessions across different sites?
Sam: Exactly. The paper introduces WebWorld, a simulator trained on over a million real-world trajectories—far more than before—using a scalable pipeline to collect data safely and efficiently. It predicts what a webpage will look like after an action, like forecasting the next frame in a video game based on your move. This world model supports long sequences of 30 or more steps and works with various page formats.
Alex: A simulator that mimics the whole web without the risks. How did they gather that massive data without hitting those same walls?
Sam: They built a three-level collection process. First, random crawling explores broadly, like scanning a city map. Second, agents create their own tasks for natural discovery. Third, targeted tasks ensure focus. This hierarchical approach pulls from open-web sources aligned with the model's prior knowledge, yielding diverse, high-quality trajectories at 100 times prior scales.
Alex: Those three levels make sense for coverage. But how do they capture the actual pages without overwhelming detail—like, what exactly gets recorded from each site?
Sam: They record a simplified map of the clickable and fillable parts of a page, ignoring images or styling that don't affect actions. This map lists elements like buttons or forms with their roles and positions, making it compact yet useful for predicting changes. Researchers call this an 'A11y Tree,' pulled from the browser using a tool called Playwright. It works across different sites because it focuses on what's interactive, not the full visual mess.
Alex: So a clean blueprint of the controls. And they had to filter this data somehow to keep only the good stuff, right?
Sam: Yes, a two-step filter first checks sites for accessibility and blocks unsafe content like gambling pages using rules and keyword scans. Trajectories get pruned if actions cause no change or exceed length limits, all with simple rules to avoid bias. They then enrich the data by converting trajectories into formats like text summaries or other structures, plus adding general chat examples to prevent forgetting basic skills.
Alex: Rule-based cleaning, then mixing in variety. That keeps quality high without over-relying on judgment calls. Training follows a two-part plan—how does that work?
Sam: First, the model learns basic page changes from the full million-plus trajectories. Second, they add a small set—about a thousand—of step-by-step reasoning examples, where the model describes why an action leads to a certain outcome before predicting it. This 'Chain-of-Thought' injection activates deeper thinking patterns already in the model. The paper shows this small dose works better than ten times more without the base training.
Alex: Oh—so reasoning builds on the dynamics they've already learned. Does this synthetic data actually help real agents perform better?
Sam: It does. They generated new trajectories using an abstract-then-concretize method: start with vague goals in the simulator, run them, then make specific versions and keep successes. Fine-tuning smaller models on this boosted success rates on benchmarks—a clear improvement from safer, scaled practice.
Alex: Notable gains there. How crucial was that small set of reasoning examples—did they test varying amounts?
Sam: They did an ablation study, testing different quantities of those step-by-step reasoning samples on the model's prediction accuracy. After the initial training on over a million basic interaction examples, just a thousand reasoning samples boosted the total score notably. The paper suggests this works because the base training first teaches the model real-world patterns, then the reasoning data activates its built-in thinking skills without overwhelming it.
Alex: So a foundation first, then targeted activation. And larger models? Does throwing more power at it keep improving things?
Sam: Yes, they trained across six model sizes and found a scaling law: as compute increases, prediction errors drop predictably in a power-law pattern, like how more practice steadily sharpens a skill without plateauing soon. Extrapolating suggests even bigger models could improve substantially further. No signs of hitting a wall yet.
Alex: Steady gains from size and compute. Does this setup transfer to other digital worlds, like apps or games?
Sam: It does. Fine-tuning on examples from areas like code editing or desktop interfaces gave about twice the performance of baselines on average, using the same prediction checks. This shows the approach adapts well across environments.
Alex: Practical, but any downsides in the predictions?
Sam: The paper notes limitations: the simulator sometimes shows sycophancy bias, generating overly optimistic results that favor the agent's choices. It also struggles with creating high-quality detailed content, like full articles. These are areas for future work.
Alex: Those biases make sense for a simulator pushing boundaries. But to get solid predictions despite that, they tuned the training carefully—why split it into stages?
Sam: They used a two-stage setup to build skills step by step. In the first stage, the model practices predicting what a webpage looks like after an action, using over a million real examples—think of it as drilling basic patterns, like how clicking a button usually loads a new form. This teaches the core rules of how web pages change without any extra thinking. The second stage adds a small batch of about a thousand examples where the model first explains its reasoning before predicting, like pausing to say 'clicking here should show results because...' This activates deeper cause-and-effect understanding already lurking in the model.
Alex: Foundation of raw patterns, then sparking the 'why' with targeted examples. Did they tweak settings between stages?
Sam: Starting with basics avoids confusing the model early on; jumping straight to reasoning without that groundwork leads to weaker results, as their tests showed. For stage one, they ramped up data flow with a higher learning speed—twice as fast as stage two—and packed examples tightly to cover ground quickly. Stage two slows things down with a lower speed and extra safeguards, like masking old history so it focuses only on reasoning parts, preventing it from forgetting the dynamics it learned first.
Alex: That conservative tweak in stage two sounds key to stability. Pulling it all together, what stands out as the real contribution here?
Sam: The core advance is a scalable way to gather and process over a million real-web trajectories safely, using structured page summaries like the A11y Tree—essentially a compact list of interactive elements pulled via Playwright. This feeds a two-stage training that first builds knowledge of page changes, then injects reasoning patterns with targeted Chain-of-Thought examples, proving essential for accurate predictions. It enables simulators for any digital interface, from web to apps, allowing safe, large-scale agent practice without live-site risks. The paper highlights sycophancy and content generation limits as areas for future work, but shows meaningful gains in generalization.
Alex: Balanced scaling with known edges to sharpen. This feels like a solid step for training agents practically.
Sam: It is. By focusing on quality trajectories through filtering, augmentation into varied formats, and alignment with model priors, it sets a path for universal digital simulators. That's the measured progress from this work. Thanks for the discussion.