A neural operator view on U-Nets for inverse imaging problems

This paper investigates the behavior of U-Net architectures when applied to ill-posed inverse problems in imaging, specifically focusing on how these networks perform when the resolution of the discretization changes. While deep learning models are often trained on fixed-resolution data, the underlying physical inverse problems are continuous. The authors evaluate whether U-shaped neural operator (UNO) architectures—designed to be resolution-invariant—outperform classical U-Nets when tested on resolutions different from their training data.

The authors review the integral operator interpretation of convolutional layers, which underpins Fourier Neural Operators (FNOs). They contrast this with the classical U-Net, which is typically optimized for a fixed input size. By analyzing the discrete convolution theorem and trigonometric interpolation, they explore how U-Nets can be adapted to handle varying resolutions. The study includes a 1D toy example for interpretability and extensive numerical experiments on limited-angle Computed Tomography (CT) reconstruction to test generalization across different discretization levels.

The research demonstrates that although U-shaped neural operators are theoretically superior for resolution-independent tasks, the classical U-Net architecture is unexpectedly robust. The authors find that standard U-Nets, despite lacking explicit resolution-invariance by design, perform well when evaluated on resolutions other than those used during training. This suggests that the practical utility of complex neural operator architectures for certain imaging tasks may be less significant than previously assumed, as simpler, widely-used architectures already capture the necessary features to handle resolution shifts effectively.