Implicit differentiation of tensor network algorithms

Variational optimization of Projected Entangled-Pair States (PEPS) is a cornerstone of modern quantum many-body physics, allowing for the simulation of two-dimensional lattice systems. However, the state-of-the-art approach—gradient-based optimization—is hindered by the computational cost and numerical instability of calculating energy gradients. Because the energy is evaluated using approximate contraction environments (like CTMRG or boundary MPS), the gradient must account for the sensitivity of these iterative environments to the variational parameters. Standard automatic differentiation (AD) often struggles with these nested iterative subroutines, leading to significant memory overhead and artifacts that destabilize the optimization process.

This paper proposes a shift from standard AD to implicit differentiation. Instead of backpropagating through every step of the contraction algorithm, the authors reformulate the converged contraction environment as the root of a specific algebraic characteristic equation. By applying the implicit function theorem to this equation, the gradient computation is reduced to solving a single, un-nested linear system. This transformation effectively "forgets" the iterative history of the contraction, focusing only on the optimality conditions that define the final environment.

By carefully choosing the parametrization of the characteristic equations—specifically by exploiting the intrinsic symmetries of the contraction environment—the authors demonstrate that they can bypass the numerical instabilities that typically arise from differentiating through subroutines like singular value decompositions or eigensolvers. This not only improves the robustness of the optimization but also simplifies the implementation, as it removes the requirement that every primitive operation in the contraction algorithm must be natively differentiable by an AD engine.