Why the Third Axis Is Freedom

The paper investigates the theoretical foundations of Explorative Modeling (XM), a training paradigm that draws multiple candidates per input and updates on the best one. While previous literature frames the benefits of this approach as 'generative expressivity'—a measure of mode count—the author argues that this metric is a proxy for a more fundamental property: the freedom of the model's function.

The author utilizes 'Stack Theory' to define freedom as the weakness of the constraints implied by a model's behavior. By modeling the hard best-of-K training process as a search for the 'weakest correct policy,' the paper derives a formal identity for the risk of XM. This identity links the probability of hitting an acceptable output to the number of candidates (K) and the model's 'freedom' (the size of its extension in an embodied language). The author then empirically tests this framework across 144 neural runs, comparing standard validation selection against a 'freedom selector' that evaluates models based on their unlabelled parent contexts.

The study demonstrates that XM effectively optimizes for freedom, which acts as the true driver of generalization. The author proves that the probability of success in best-of-K training increases with local freedom for any K > 1. Empirically, a freedom-based selector outperformed standard validation selection in 29 out of 30 cases. The paper concludes that generative expressivity is merely a mode-count proxy that discards the structural information necessary to understand why certain models generalize better than others under distribution shift.

This work provides a formal, function-level explanation for why scaling the number of candidates in generative training improves performance. By shifting the focus from parameter-based complexity or simple mode counts to the 'freedom' of the resulting policy, researchers can better predict which models will generalize to unseen tasks. It establishes that exploration is a budgetary means, while freedom is the ultimate end for robust machine learning.