It’s Much Easier for Neural Networks to Learn Game of Life Dynamics with the Right Activation Function: Polynomial Kolmogorov-Arnold Networks

Research Context

Conway's Game of Life is a classic cellular automaton that has historically proven difficult for standard neural networks to learn. Previous research suggested that because minimal networks can theoretically represent these rules, the failure of standard ReLU-based networks to converge is a search problem—implying that large, over-parameterized networks are required to find a 'winning ticket' or lucky initialization. This study challenges that assumption by re-evaluating the problem as one of inductive bias rather than search scale.

The authors propose 'PolyKAN,' a minimal network architecture that replaces standard synaptic weight learning with learnable 2nd-degree polynomial activation functions, inspired by the Kolmogorov-Arnold Representation Theorem. By testing various activation functions on the single-step prediction task of the Game of Life, the researchers compared the success rates of these minimal models against the standard ReLU baseline. They also performed ablation studies to determine whether the networks could learn the dynamics by adjusting only the activation function coefficients, only the synaptic weights, or both.

The study demonstrates that the choice of activation function is critical for learning cellular automata dynamics. PolyKAN consistently learned the Game of Life rules with a minimal parameter count, significantly outperforming ReLU networks, which rarely converged. Notably, PolyKAN achieved perfect success rates even when synaptic weights were frozen and only the polynomial coefficients were trained. The results suggest that the difficulty in learning such systems is often an inductive bias mismatch between the piecewise linear nature of ReLU and the interval-based logic of the target dynamics.