Learning to Fold: prizewinning solution at LeHome Challenge 2026

This paper details the winning entry for the LeHome Challenge 2026, a robotics competition focused on bimanual garment folding. The author presents an engineering-heavy approach that treats a vision-language-action (VLA) policy as its own value function. By integrating auxiliary heads into the policy network, the system predicts success probability, task progress, and future keypoint distances. These predictions are used to calculate advantages, detect failures in real-time, and select the best candidate actions during inference.

The core of the system is an asynchronous training loop that connects three independent components via the HuggingFace Hub: a training worker, multiple rollout workers, and a manual DAgger (Dataset Aggregation) station. This architecture allows for continuous, non-blocking data collection and training. The author employs a combination of Advantage Weighted Regression (AWR) and RECAP-style advantage conditioning. This combination allows the model to reweight training data toward high-advantage frames while simultaneously using advantage as a conditioning input to enable classifier-free guidance at inference time.

To bridge the gap between simulation and the real-world final, the author implemented a robust sim-to-real recipe. This included heavy domain randomization, camera-alignment tooling, and a DAgger-like human-in-the-loop data collection process. Furthermore, the author optimized inference-time hyperparameters—such as execution length and noise temperature—using Thompson sampling to find the best configurations for different garment types without requiring manual tuning for every scenario.