Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients

Physics-Informed Neural Networks (PINNs) often rely on domain decomposition to scale to complex geometries. However, partitioning a problem into subdomains introduces multiple loss terms—residual, boundary, and interface—that frequently produce conflicting gradients during training. This paper investigates how to mitigate these conflicts to improve training stability and accuracy in 2D and 3D settings.

The authors evaluate existing gradient surgery techniques, such as PCGrad and ConFIG, and identify that they often fail in complex 3D domain decomposition scenarios. To address this, they propose Norm-PCGrad, a variant that normalizes projected gradients before aggregation. This normalization prevents the magnitude bias inherent in standard PCGrad, ensuring that interface coupling between subdomains is respected. The authors also explore combining this with curvature-aware quasi-Newton optimizers (SSBroyden) and extend the framework to Kolmogorov-Arnold Networks (KANs) using feature-enriched basis functions.

Norm-PCGrad consistently outperforms existing gradient surgery methods across various 2D and 3D benchmarks, including the Helmholtz, Poisson, and sine-Gordon equations. By normalizing projected gradients, the method effectively balances the influence of different loss terms, leading to superior solution accuracy and better interface continuity. Furthermore, the authors demonstrate that this approach scales effectively to KAN-based architectures, providing a robust alternative to standard multi-layer perceptrons for solving PDEs in 3D.

Domain decomposition is essential for scaling scientific machine learning to large-scale engineering problems. By providing a principled, conflict-free gradient aggregation method, this work enables more reliable training of decomposed models. The results show that researchers can achieve state-of-the-art accuracy without the need for complex, manually tuned loss weights, significantly simplifying the deployment of physics-informed models in complex geometries.