TUBULAR NEIGHBOURHOODS OF PFAFFIAN SETS AND APPLICATIONS TO NEURAL NETWORKS

This paper investigates the geometric complexity of decision boundaries in neural networks that use Pfaffian activation functions (such as the logistic sigmoid or tanh). Specifically, it seeks to quantify the robustness of these classifiers by bounding the volume of the tubular neighbourhood around the decision boundary, which serves as a proxy for the probability that a small input perturbation leads to misclassification.

The authors utilize the theory of Pfaffian functions—a class of transcendental functions that satisfy triangular systems of first-order partial differential equations—to generalize classical results from algebraic geometry. By applying Khovanskii’s theorem, they bound the degrees of the Gauss maps of Pfaffian hypersurfaces. They then translate these geometric bounds into probabilistic tail bounds for the condition number of neural network classifiers. A key technical innovation is the use of a multiplicative chart to bypass the exponential dependence on network width typically associated with Pfaffian intersection bounds, allowing for a polynomial-in-width bound for single-hidden-layer networks.

The study provides a general formula for the volume of tubular neighbourhoods of smooth Pfaffian hypersurfaces, which is then applied to neural networks. For general Pfaffian networks, the authors derive tail bounds on the condition number that depend on the network's depth and width. Crucially, for single-hidden-layer sigmoid networks with rational weights, the authors prove that the probability of a high condition number is bounded by a term polynomial in the network width, effectively eliminating the exponential "Khovanskii factor" found in more general settings. This result provides a theoretical foundation for understanding the robustness of shallow neural networks in terms of their geometric structure.