MODULARITY-FREE CONFLICT-AVERSE TRAINING FOR GENERALIZED PINNS

Physics-Informed Neural Networks (PINNs) are powerful tools for solving partial differential equations (PDEs), but their training is notoriously fragile. While recent conflict-averse optimization techniques aim to mitigate gradient interference between residual and boundary losses, the authors observe that these methods often fail as model capacity increases. This paper investigates why larger PINNs struggle to converge and proposes a structural solution to maintain training stability.

The researchers identify a phenomenon called functional modularity, where overparameterized networks self-partition into task-exclusive modules. This segregation suppresses the necessary cross-objective interaction required to solve the PDE effectively. To combat this, they introduce Modular-Sparsity Synchronization (ModSync). ModSync integrates structural optimization into the training process by using dynamic binary masks to prune connections that are exclusive to a single objective, while simultaneously using a reweighting function to protect connections that are shared across objectives.

The study demonstrates that as model width increases, the gradients of the residual and boundary losses tend toward orthogonality, signaling a breakdown in cross-objective coupling. Existing conflict-averse methods often exacerbate this issue. By incorporating ModSync, the authors show that networks can maintain robust cross-objective coupling even at higher capacities. Extensive experiments across diverse PDE benchmarks—including Helmholtz, Klein-Gordon, and Burgers’ equations—show that ModSync consistently prevents capacity-driven failures and achieves state-of-the-art accuracy compared to standard PINNs and existing gradient-manipulation baselines.