Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

This paper investigates test-time scaling (TTS) in open-ended language model generation tasks across medicine, law, finance, general chat, and creative writing. While TTS techniques have achieved large performance gains on verifiable tasks like mathematics and code, their effectiveness on open-ended tasks remains unstudied. The authors evaluate five TTS families—Best-of-N, Beam Search, Particle Filtering, Sequential Refinement, and Fusion—at matched compute budgets using a unified framework that decomposes the token budget into exploration and exploitation.

The central finding is that exploration works well—the best candidate in a generated pool consistently improves with compute—whereas exploitation breaks down entirely. External reward models correlate at only about 0.12 with true output quality on open-ended tasks, rendering candidate selection near-random regardless of budget. Furthermore, tree search methods actively harm performance due to diversity collapse, while sequential refinement is largely unreliable. Only candidate synthesis, or Fusion, consistently improves over single-sample baselines, yet it still captures only about 40 percent of the available quality headroom.

These results reveal a fundamental limitation in current test-time scaling paradigms, showing that simply allocating more compute to inference will not improve open-ended generation without dramatic improvements in verifier and selection capabilities. The findings redirect the focus of alignment and scaling research away from raw candidate generation and toward robust evaluation and synthesis.