Self-Evolving World Models for LLM Agent Planning

Large Language Model (LLM) agents often struggle with long-horizon planning due to unreliable foresight. While world models can predict action consequences to guide agents, they often suffer from distribution shifts during deployment. This paper asks: can we improve world model foresight and agent planning through continuous, test-time adaptation without the high computational cost of retraining model parameters?

The authors introduce WorldEvolver, a self-evolving framework that keeps the downstream agent and world model parameters frozen. It instead evolves the deployment-time context using three modules:

• Episodic Memory: Stores and retrieves realized action-transition experiences to ground future predictions in past successes.
• Semantic Memory: Extracts persistent heuristic rules from prediction-observation mismatches, converting failures into reusable knowledge.
• Selective Foresight: Filters out low-confidence predictions using token-level probability scores to prevent the agent from being misled by unreliable simulations.

By updating these memory stores online, the system adapts to the environment without requiring gradient-based weight updates.

WorldEvolver consistently achieves the highest prediction accuracy across three different model backbones (Gemma-4-26B, Qwen3.5-9B, and Gemma-4-31B) on the Word2World benchmark. In downstream planning tasks on ALFWorld and ScienceWorld, WorldEvolver outperforms existing world-model baselines, demonstrating that test-time memory revision effectively enhances both predictive fidelity and agent success rates. The study also shows that selective foresight is crucial, as it mitigates the negative impact of noisy predictions, particularly for weaker model backbones.

This work provides a scalable, training-free path for improving agentic foresight. By treating world model adaptation as a memory-management problem rather than a parameter-optimization problem, WorldEvolver offers a more flexible and efficient way for agents to learn from their environment in real-time. This is particularly relevant for long-horizon tasks where agents must navigate evolving environments.