Ruchit Rawal, Reza Shirkavand, Sayak Paul, Yuxin Wen, Heng Huang, Yizheng Chen, Tom Goldstein, Gowthami Somepalli
5 min
Inference-time scaling for diffusion models has traditionally focused on guided search methods that prune or steer trajectories during denoising. However, these methods often ignore the computational overhead of frequent verifier calls, which can distort efficiency rankings. This paper investigates whether compute is better spent on broader exploration of candidate trajectories rather than repeated intermediate verification, and how to optimize generation costs to maximize performance under strict wall-clock constraints.
The authors introduce Flash-BoN, a pipeline that generates a large pool of inexpensive draft candidates by combining three acceleration knobs: timestep truncation, layer skipping, and activation proxies. These knobs are jointly optimized once per model to find a configuration that balances speed and fidelity. The pipeline follows a three-stage process: (1) generating multiple low-cost drafts, (2) using a multi-stage verification procedure (pointwise pruning followed by targeted pairwise comparisons) to select the most promising candidate, and (3) refining only the selected candidate at full quality. The authors also demonstrate that this draft-and-select strategy can be integrated into existing methods like Reflection-Tuning and RL post-training (Flow-GRPO).
This work challenges the prevailing focus on intermediate verification in inference-time scaling, suggesting that generation efficiency is a more critical lever for performance. By providing a modular, hardware-aware framework that scales effectively with model size, Flash-BoN offers a practical solution for improving image generation quality within realistic latency budgets.
Inference-time scaling for text-to-image generation has progressed from simple Best-of-$N$ (BoN) sampling to guided search methods that verify and steer candidate trajectories at intermediate denoising steps. These approaches focus on when and how often to verify during denoising but largely treat the cost of generation itself as fixed. Moreover, the standard practice of comparing methods by number of function evaluations (NFEs) counts only denoising forward passes and ignores verifier overhead, which can distort efficiency rankings. We show that under wall-clock evaluation, simple BoN already matches or outperforms several guided search techniques, suggesting that compute is better spent on broader exploration than on repeated intermediate verification. This motivates Flash-BoN, which generates a large pool of inexpensive draft candidates by combining three complementary acceleration knobs: timestep truncation, layer skipping, and activation proxies into a single configuration optimized once per model. An efficient multi-stage verification procedure then identifies the most promising draft, which is refined at full quality. Across three benchmarks and three model scales, Flash-BoN consistently outperforms all baselines under fixed wall-clock budgets, with gains that grow at larger model scales (+8% AUC). We further show that our strategy combines well and improves existing orthogonal techniques such as reflection-based prompt optimization (+16% AUC). The gains correlate with increased candidate diversity, which also enables draft-guided selection to accelerate RL post-training convergence.
Alex: That sounds like it would hurt the quality, though. If you're cutting corners on every draft, how do you end up with a good final image?
Sam: That's the key design insight. The drafts are never meant to be final products. They only need to be good enough for the verifier to make a reliable judgment about which one has the most promise. Once the system identifies the best candidate, it goes back and generates that one at full quality, with no shortcuts. So you're spending your careful, expensive work only on the one image that's already been selected as the most likely winner.
Alex: So it's like a talent show with a very cheap audition round. You don't give everyone a full orchestra — you just listen for thirty seconds to figure out who's worth the investment.
Sam: That's a good way to put it. And they take the selection process seriously too. Rather than giving each draft a single score and picking the highest number, they use a ranking system borrowed from competitive chess — called Elo ranking. The verifier compares images head-to-head, two at a time, which tends to produce more reliable judgments than any single absolute score.
Alex: So it's a tournament bracket, essentially.
Sam: Essentially, yes. And it's structured in stages. First, a quick scan filters out the clearly weak candidates. Then the survivors go through head-to-head comparisons. This is much more efficient than comparing every image against every other image, which would get very expensive very quickly as the pool grows.
Alex: Does this approach actually produce better images in the end?
Sam: The paper reports that it outperformed other methods across every model they tested. They also applied the same draft-and-select logic to a different problem — training AI systems through trial and error, which researchers call reinforcement learning. There, the approach reached standard performance benchmarks significantly faster than conventional methods.
Alex: So the insight generalises beyond just image generation. It's really a broader argument about how you allocate your computing budget.
Sam: That's the deeper point the authors are making. By shifting the focus from how carefully you run each individual step to how wisely you spend your total time budget, you can get meaningfully better results without retraining the underlying model at all. The model itself doesn't change — only the strategy for using it does.
Alex: That's a useful reminder. In engineering, the question of how you use a tool is often just as important as how powerful the tool is.
Sam: And that's what makes this approach practically relevant. For anyone deploying these systems in the real world — where users are waiting for results — shaving time off the process without sacrificing quality is a meaningful improvement. Flash-BoN suggests that a lot of that efficiency was already available. It just required a different way of thinking about where the time was actually going.
Alex: Thanks for walking us through that, Sam. And thanks to everyone listening — this has been ResearchPod.