Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization for Task Planning

Multimodal Large Language Models (MLLMs) are increasingly used as GUI agents to automate web tasks. However, smaller, open-source models often struggle with complex task planning and cross-website generalization. This paper investigates how to improve these agents' planning capabilities without relying on massive commercial models, specifically focusing on whether training on high-level task experiences is more effective than training on atomic-level skills.

The authors introduce the Planning Experience Exploration and Utilization (PEEU) method, which consists of two stages. First, a planning tree exploration stage uses an agent to autonomously set goals and explore websites, building a tree of observations and actions. Second, a planning experience utilization stage extracts these trajectories and uses them to synthesize high-level, strictly aligned training data. To evaluate this, the authors propose the Task Decomposition Hierarchical Analysis Framework (TDHAF), which categorizes tasks into low, middle, and high levels to measure how different training granularities affect compositional generalization.

The study demonstrates that mastering low-level atomic skills (e.g., individual clicks or typing) does not guarantee competence in high-level planning. Instead, training on high-level, constrained tasks leads to significantly stronger out-of-domain (OOD) generalization. When applied to real-world benchmarks, a 7B parameter model trained with PEEU achieved 30.6% accuracy, outperforming the much larger 32B parameter Qwen2.5-VL model. The authors also found that for small models, direct fine-tuning using PEEU-generated data is far more effective than prompt-based retrieval methods, which often fail due to the models' limited reasoning capabilities.

This research provides a scalable path for deploying efficient, privacy-preserving GUI agents. By showing that high-level task training is superior to atomic-level training, the authors offer a clear strategy for improving small MLLMs. The PEEU method effectively turns raw environment exploration into high-quality training signals, bridging the performance gap between small open-source models and massive commercial ones in complex, real-world web navigation scenarios.