EML Trees Are Universal Approximators

Overview

This paper establishes a rigorous theoretical foundation for using tree-structured compositions of Exp-Minus-Log (EML) functions as a universal approximation framework. By generalizing the EML function to include six learnable parameters per unit, the authors demonstrate that these structures can represent polynomials and approximate smooth functions within Sobolev spaces ($W^{k, \infty}$) to arbitrary precision.

Theoretical Approach

The authors prove universal approximation by explicitly constructing EML trees that mimic polynomial representations and approximate partitions of unity. The proof strategy involves partitioning the domain into small cubes, creating local polynomial approximations, and then stitching these together using smooth transition functions built from EML atoms. A key challenge addressed is the logarithmic singularity inherent in the EML function; the authors resolve this by restricting the initial proof to the half-open domain $(0, 1]^d$ and subsequently extending the result to the closed cube $[0, 1]^d$ via affine contraction mappings.

Practical Implementation

Beyond the theoretical proof, the authors investigate the feasibility of training these EML trees using gradient-based optimization. They introduce a generalized EML atom with learnable parameters and test it on several one-dimensional benchmark functions. To handle numerical stability during training, they employ a softplus-smoothed logarithm surrogate. The empirical results demonstrate that EML trees can achieve low relative error on various functions, though the authors note that the optimization landscape and interpretability remain areas for further development.

This work bridges the gap between symbolic regression and neural network architectures. By providing a theoretical guarantee for EML-based models, the paper positions them as a viable alternative to standard activation-based neural networks, particularly in applications where compact, compositional, or potentially interpretable functional representations are desired.