The Degeneracy Distillery

Scientific models often suffer from degeneracies, where different parameter combinations produce indistinguishable data. This makes statistical inference difficult, as the Fisher information matrix—which describes the sensitivity of the model to parameter changes—becomes highly anisotropic and ill-conditioned. The authors seek a systematic, data-driven way to identify these degeneracies and reparametrize models into coordinates where the Fisher information is approximately isotropic (flat), thereby simplifying downstream inference and providing physical insight.

The authors introduce the "Degeneracy Distillery," a three-stage pipeline that operates on parameter-data pairs without requiring real-world observations:

1. Information Geometry Mapping: Using "Fishnets," an ensemble of neural networks estimates the Fisher information matrix across the parameter space.
2. Neural Coordinate Learning: A "flattener" network is trained to find a coordinate transformation that maps the original parameters to a new space where the Fisher matrix is as close to the identity as possible.
3. Symbolic Distillation: Symbolic regression is applied to the neural coordinates to derive human-readable, nonlinear expressions that define the new, degeneracy-free parameter space.

Findings and Impact

The method was validated on synthetic problems (Rosenbrock function and Gaussian models) and applied to complex scientific tasks, including SIR epidemic dynamics, gravitational-wave waveforms, and weak-lensing cosmology. The results show that the distilled coordinates significantly improve the efficiency of neural posterior estimation (NPE). By aligning the coordinate system with the model's intrinsic sensitivity, the authors achieved matched validation calibration with up to 10 times fewer simulations compared to using raw, degenerate parameters. This approach not only reduces computational costs but also reveals the physically meaningful combinations of parameters that actually control model observables.