Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence

Modern image-to-video (I2V) models are powerful but notoriously difficult to control, often requiring inefficient, brute-force trial-and-error to achieve desired results. The authors ask: can we replace this manual, speculative process with an automated, agentic framework that treats video synthesis as a closed-loop, goal-directed optimization problem?

The authors introduce a two-stage "Agentic Self-Improvement" framework. First, a prompt optimization loop uses a multimodal Large Language Model (mLLM) to iteratively refine input prompts. This stage uses Davidsonian Scene Graph (DSG) queries to ensure semantic adherence and Common Mistake Questions (CMQ) to detect visual artifacts. Second, the framework employs Bayesian optimization to co-optimize stochastic seeds and classifier-free guidance (CFG) scales. This process is guided by a novel Video-Text Adherence (VTA) score, which aggregates the mLLM's binary evaluations into a single metric to steer the optimization toward high-quality, intent-aligned outputs.

The proposed framework demonstrates a clear advantage over unguided generation methods. In human preference studies, videos produced by the agentic approach were strongly preferred, achieving win rates of up to 69%. The authors validated their automated evaluation engine (Gemini 2.5 Pro) against human annotators, finding 92% accuracy on semantic queries and 82% on artifact detection, confirming that the mLLM serves as a reliable proxy for human judgment in this pipeline.

This work shifts the paradigm of video generation from "lucky" generation to systematic engineering. By automating the feedback loop, the framework reduces the computational and human labor required for professional video production, making state-of-the-art generative models more predictable and reliable for real-world applications.