Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation

Diffusion-based text-to-motion (T2M) models often suffer from semantic drift, where the generated motion fails to accurately reflect the input text, particularly in long-duration or complex sequences. The authors investigate whether the initial Gaussian noise, typically treated as a random seed, can be strategically selected and refined to act as a latent semantic prior that guides the denoising process toward better alignment with the text prompt.

Building on the Lottery Ticket Hypothesis, the authors introduce WINRO (WInning Noise Retrieval and Optimization). The framework consists of three main components:

1. Winning Noise Retrieval: The authors construct a dictionary of noise-motion pairs by generating motions from random noise under null prompts and encoding them into a shared text-motion embedding space. At inference, the system retrieves the noise seed whose induced motion feature is most similar to the input text.
2. Noise Refinement: The retrieved noise is further optimized using a KL-regularized objective that minimizes the discrepancy between the generated motion and the text prompt while ensuring the noise remains within the Gaussian distribution.
3. Amortized Refinement: To avoid the computational cost of iterative optimization, the authors provide an optional LoRA-based adapter that learns to predict the necessary noise correction in a single forward pass.

The study demonstrates that initial noise significantly influences the semantic outcome of motion generation. By retrieving and refining these winning noise tickets, WINRO consistently improves text-motion fidelity across different diffusion backbones (MDM and MotionLCM) on the HumanML3D benchmark. The approach is model-agnostic and training-free, and it shows improved robustness in long-horizon generation and generalization to tasks like motion stylization and spatial constraint satisfaction.