LeCropFollow: Latent Space Planning for Navigation in Unstructured Crop Fields

Autonomous robots in agriculture often fail when navigating unstructured environments, such as fields with irregular planting or significant vegetation gaps. Traditional navigation methods typically rely on geometric models that compress high-dimensional visual data into simple, deterministic spatial references (like keypoints). The authors argue that this over-compression discards critical semantic context and uncertainty, leading to failures when the geometric assumptions are violated. To overcome this, the authors introduce LeCropFollow, a framework that replaces explicit geometric modeling with a learned latent representation. By integrating a self-supervised semantic heatmap extractor with a Model-Based Reinforcement Learning (MBRL) planner (TD-MPC2), the system optimizes trajectories directly within a latent manifold, preserving the full semantic signal of the environment.

LeCropFollow demonstrates significant robustness in unstructured agricultural settings. In field experiments conducted in late-stage corn fields, the framework achieved a 2.4x reduction in semantic failures compared to keypoint-based methods. While traditional geometric approaches often struggle to maintain a consistent path when row references vanish or become ambiguous, LeCropFollow uses the latent world model to plan trajectories that account for the uncertainty inherent in the heatmap signal. The authors show that this representational shift enables zero-shot transfer from simplified simulations to real-world deployment without the need for fine-tuning. In comparative trials, LeCropFollow matched the performance of state-of-the-art baselines in structured rows and significantly outperformed them in plantation gaps.

This research highlights a shift in agricultural robotics from rigid, geometry-dependent pipelines to flexible, learning-based representations. By demonstrating that latent planning can effectively handle the ambiguity of unstructured fields, the study provides a scalable path toward more reliable autonomous phenotyping and weeding in heterogeneous environments. The ability to perform zero-shot transfer from simple simulations to complex, real-world crop fields is a major step forward for the practical deployment of under-canopy robots.