SEE: Structure-aware Exploring & Exploiting for Long-horizon GUI Agent Trajectory Synthesis

Modern GUI agents struggle with long-horizon tasks in complex, dynamic mobile environments. Existing datasets are often limited by high costs (human demonstrations) or poor coverage (on-policy exploration that favors common, short-horizon paths). The authors ask: can we automate the synthesis of diverse, long-horizon, and element-rich interaction trajectories to improve GUI agent robustness?

To address this, the authors propose SEE (Structure-aware Exploring & Exploiting), a two-stage framework that decouples data generation into exploration and synthesis:

1. Structure-aware Exploration: The agent interacts with an app to build an explicit transition graph where nodes represent screen states and elements, and edges represent executable actions. This stage uses visual parsing and semantic descriptions to identify meaningful transitions, including handling system interruptions like pop-ups.
2. Graph-based Synthesis: Instead of relying on trial-and-error, the framework plans paths across the constructed graph. It generates multi-step subgoals and uses graph search (e.g., BFS) to find executable action sequences. This allows for precise control over trajectory length and task complexity, while avoiding spurious loops.

SEE successfully generates trajectories with an average length of 14.8 steps, significantly outperforming existing datasets in both trajectory duration and screen complexity (measured by the number of elements per screen). By training on this synthetic data, GUI agents demonstrate improved task success rates and better generalization to unseen screens, proving that structured, graph-based data synthesis is a viable alternative to expensive human-annotated datasets.

This work provides a scalable, cost-effective way to generate high-quality training data for GUI agents. By moving away from simple, short-horizon interactions toward complex, multi-step planning, the SEE framework helps bridge the gap between current agent capabilities and the requirements of real-world mobile applications.