Offline-to-Online Creative Optimization with Generative Models and Adaptive Testing

This paper addresses the bottleneck in digital advertising where generative models can easily produce endless ad variants, but identifying the best one remains constrained by the high cost and noise of online A/B testing. The authors propose an offline-to-online workflow that separates the generation of candidates from their final selection. In the offline phase, a predictive model trained on historical experimental data acts as an inference-time critic, guiding a generative model to refine and rank potential ad creatives. In the online phase, these candidates are deployed in a batched adaptive experiment (a multi-armed bandit) to identify the winner while minimizing traffic exposure to underperforming variants.

The study reveals a critical design principle: predictive models do not need to be accurate enough to pick the single best creative; they only need to be accurate enough to ensure the generated slate contains high-performing candidates. In a 50-arm field experiment, the best AI-refined creative achieved a 45.1% higher engagement rate than the best human-authored creative. The authors observed that while the predictive model was unreliable for direct selection—often ranking the best creative poorly—it was highly effective at filtering the search space to ensure the final test slate included top-tier performers. Adaptive experimentation then provided the necessary precision to select the winner from this high-recall slate.

This work shifts the focus of generative AI in marketing from direct optimization to candidate-set generation. By treating the predictive model as a guide for exploration rather than a final arbiter, organizations can leverage historical data to improve creative performance without needing a perfect predictive model. This approach effectively mitigates the risk of false positives inherent in AI-generated content and provides a scalable framework for optimizing creative assets in high-dimensional spaces.