HESSIAN-AUGMENTED SUPERVISED LEARNING FOR HAMILTON-JACOBI-BELLMAN PDES

This paper addresses the computational challenge of synthesizing optimal feedback control laws for nonlinear, control-affine systems. The core difficulty lies in solving the Hamilton-Jacobi-Bellman (HJB) equation, which typically suffers from the curse of dimensionality, making grid-based methods intractable for systems with more than a few state variables. To overcome this, the authors propose a supervised learning framework that approximates the value function using sparse polynomial bases.

Instead of relying solely on value function samples, the authors generate a dataset consisting of values, gradients, and Hessians of the value function. These are obtained by solving the Pontryagin Maximum Principle (PMP) optimality system from various initial conditions. While gradients are identified with the adjoint variables of the PMP, the Hessian information is extracted by integrating a matrix Riccati equation along the optimal trajectories. This derivative-augmented data is then used in a weighted least-squares regression to determine the coefficients of the polynomial approximation.

The inclusion of second-order (Hessian) information provides a substantial reduction in the number of training samples required to achieve a target approximation accuracy. By incorporating gradient and Hessian data, the authors transform the regression into a system with significantly more linear equations per sample, which allows for accurate recovery of the feedback law with far fewer trajectories than value-only methods.

To manage the computational cost of generating Hessian data in high-dimensional state spaces, the authors introduce a partial Hessian strategy. This allows researchers to include second-order information for only a fraction of the training points, providing a tunable trade-off between data generation costs and approximation performance. Numerical benchmarks demonstrate that this approach achieves up to an order-of-magnitude reduction in sample complexity compared to lower-order methods.

This work provides a scalable alternative to traditional grid-based solvers for high-dimensional optimal control problems. By leveraging the geometric structure of optimal trajectories—specifically the relationship between the value function, its derivatives, and the PMP—the method enables the construction of robust, real-time feedback laws for complex systems where traditional discretization fails. The flexibility of the polynomial basis and the efficiency of the derivative-augmented regression make this a promising tool for applications in robotics, aerospace, and finance.