Certification of Machine Learning Models via Directional Sharpness

How can we reliably certify the generalization quality of machine learning models? Traditional metrics like test accuracy are vulnerable to training deviations, while existing sharpness-based metrics are computationally expensive and often fail to detect models that are only superficially flat or contain hidden vulnerabilities like backdoors.

The authors propose directional sharpness, a metric that shifts from measuring static loss landscape geometry to evaluating dynamic stability. Instead of calculating sharpness at a single point, the method applies a sequence of stochastic, SAM-style perturbations to the model. A model that is truly in a wide, flat basin will maintain stable sharpness values across these perturbations, whereas models in narrow or unstable regions will exhibit detectable fluctuations. This approach is designed to be efficient enough for both direct auditing (with access to training data) and cryptographic certification (using zero-knowledge proofs).

Directional sharpness outperforms existing static metrics in correlating with generalization performance. Crucially, it remains predictive even when models are trained with intentional deviations, such as injected backdoors, or unintentional ones, such as overfitting. The authors demonstrate that this metric is highly efficient: in auditing settings, it is 4x faster to compute than test accuracy, and in certification settings, the zero-knowledge proof is up to 80,000x faster than verifying the entire training process.

This work provides a practical, robust framework for model certification. By enabling auditors to verify model quality without relying on potentially misleading test sets or computationally prohibitive training proofs, it offers a scalable way to ensure that deployed models are not just accurate on specific test data, but are fundamentally robust and well-generalized.