Learning Dynamical Systems from Multiple Sparse Datasets: A Hierarchical Bayesian Modeling Approach

Estimating parameters for dynamical systems—often represented by ordinary differential equations (ODEs)—is notoriously difficult when data is sparse, noisy, or irregularly sampled. This paper addresses the challenge of "practical non-identifiability," where individual datasets lack sufficient information to reliably estimate model parameters. The authors investigate whether pooling information across a collection of related datasets, rather than treating them independently, can stabilize parameter estimation and improve predictive performance.

The researchers employ a Hierarchical Bayesian Modeling (HBM) framework to perform probabilistic meta-learning. In this setup, dataset-specific parameters are modeled as draws from a shared population distribution, with hyperparameters that are inferred jointly alongside the specific parameters. To handle the computational complexity of the resulting high-dimensional posterior, the authors integrate a differentiable numerical ODE solver directly into a gradient-based Markov Chain Monte Carlo (MCMC) sampler, specifically the No-U-Turn Sampler (NUTS). This allows gradients of the likelihood to propagate through the numerical integration, enabling efficient exploration of the parameter space.

The study demonstrates that by systematically pooling information across sparse trajectories, the hierarchical approach achieves significantly better predictive accuracy and parameter identifiability than unpooled Bayesian or nonlinear least-squares baselines. The method successfully recovers the underlying shared structure of the dynamical systems, even when individual datasets are so sparse that they would be otherwise unidentifiable. The authors validate this on a Lotka–Volterra benchmark, showing that the learned population-level prior effectively acts as a regularizer for future, unseen datasets.