Xinxin Li, Xingyu Cui, Jin Qi, Juan Zhang, Da Li, Junping Yin
10 min
Abstract
Discovering governing Partial Differential Equations (PDEs) from sparse and noisy data is a challenging issue in data-driven scientific computing. Conventional sparse regression methods often suffer from two major limitations: (i) the instability of numerical differentiation under sparse and noisy data, and (ii) the restricted flexibility of a pre-defined candidate library. We propose Weak-PDE-Net, an end-to-end differentiable framework that can robustly identify open-form PDEs. Weak-PDE-Net consists of two interconnected modules: a forward response learner and a weak-form PDE generator. The learner embeds learnable Gaussian kernels within a lightweight MLP, serving as a surrogate model that adaptively captures system dynamics from sparse observations. Meanwhile, the generator integrates a symbolic network with an integral module to construct weak-form PDEs, avoiding explicit numerical differentiation and improving robustness to noise. To relax the constraints of the pre-defined library, we leverage Differentiable Neural Architecture Search strategy during training to explore the functional space, which enables the efficient discovery of open-form PDEs. The capability of Weak-PDE-Net in multivariable systems discovery is further enhanced by incorporating Galilean Invariance constraints and symmetry equivariance hypotheses to ensure physical consistency. Experiments on several challenging PDE benchmarks demonstrate that Weak-PDE-Net accurately recovers governing equations, even under highly sparse and noisy observations.
Sam: The core is a network that combines basic math operations like addition, multiplication, or sine waves applied to the reconstructed field, building candidate terms layer by layer. To find the best combination without predefining it, they use a search method where all possible operator mixes get soft weights that can be optimized via gradients—like voting on architectures continuously instead of trying them one by one. Researchers call this *differentiable neural architecture search*. It explores endless functional forms end-to-end.
Alex: So it's jointly tuning the structure, derivatives, and numbers all at once? Walk me through the training stages—does it evolve in phases?
Sam: Yes, three phases run in parallel with field reconstruction: first, *searching* adapts the network depth and operator counts softly via those weights, finding lightweight fits. Then *pruning* adds a penalty to shrink redundant links and coefficients for sparsity, like trimming a bush. Finally, *tuning* refines the fixed structure's numbers precisely. Losses balance data fit, weak-form match, and optional regularization—data loss pins predictions to observations, weak loss enforces the integral equation.
Alex: Huh, so the search prevents overfitting by favoring simple winners. That's a clear path from mess to exact physics.
Sam: In the pruning step, with the network fixed, they add a penalty during training that pushes small connection weights toward zero—like gently squeezing out extra branches on a decision tree until only the vital paths remain. This creates a sparse setup where only the most relevant math terms survive, keeping the equation concise. Once pruned, those terms form the candidate library for the final step.
Alex: Got it—that favors simplicity. Then in tuning, it sounds like they switch to straightforward math to nail the numbers on those terms?
Sam: Yes. They use the response learner to predict the full field on a regular grid from sparse points. Then they build a data matrix of the pruned terms' values and a target from the time changes, solving a basic linear fit—like matching puzzle pieces' sizes to find which ones belong. This gives exact coefficients via least-squares, yielding the full PDE.
Alex: Okay, efficient cleanup. But for multi-variable fluids or quantum-like waves, how do they avoid inconsistent equations across parts of the system?
Sam: For fluid flows, they apply rules from Galilean invariance—the idea that true physics laws don't change if you switch to a steadily moving viewpoint, like describing a river from a boat drifting at constant speed. Terms must cancel out any fake shifts from that motion: keep advective ones like flow carrying itself, drop absolute speed ones that vary by frame. This filters the library during tuning, ensuring consistency. In complex systems like quantum fluids, they use symmetry equivariance as a flexible guess: the equations for real and imaginary parts should mirror each other under phase twists, like clock hands linked oppositely. They pair terms anti-symmetrically in the library, letting data decide if it fits—narrowing options without forcing it. The paper suggests this boosts reliability on coupled variables.
Alex: So those physics checks like symmetry keep things consistent across variables. But how does it hold up in actual tests—does it really nail the equations on messy data?
Sam: The paper tests it on several standard equations, starting with the Burgers equation—a simple model for fluid shocks with diffusion. It recovers the exact structure every time, even from just 2.5% of the data points or with noise as strong as the signal itself.
Alex: Right, perfect hits on Burgers. What about tougher ones like wave patterns or chaotic flows?
Sam: Similar results on the Korteweg-de Vries for shallow water waves and Kuramoto-Sivashinsky for flame instabilities—exact recovery down to 2.5% data and up to 100% noise. The two-dimensional wave equation goes even lower, to one percent sampling while staying stable until very high noise. But the paper flags something on the Chafee-Infante reaction-diffusion case. At higher noise, the smoothing in the integrals acts like a low-pass filter—averaging out wiggles so much that sharp diffusion signals get lost amid the randomness. It picks wrong terms, like odd powers of spatial changes, highlighting a limit: the method trades some high-frequency detail for noise resistance.
Alex: So not flawless on every type, but a meaningful advance for sparse sensor data like oceans. The evidence points to reliable discovery where others falter.
Alex: That reliability on sparse data stands out. But to gauge how much each part contributes—like those adaptive kernels or the search—did they test versions without them?
Sam: Yes, they ran ablation studies, which means creating simplified versions of the full system by removing one key piece at a time, then comparing performance across the benchmark equations. Without the adaptive kernels, reconstruction errors rise because plain networks struggle with sharp changes like shock waves in fluids—they miss sudden jumps by focusing too much on smooth, slow-varying parts. Skipping the architecture search leaves extra, useless terms in the equation, bloating it with junk that hurts accuracy. The full setup gets perfect term recovery where the others drop off, with parameter errors about one to two orders of magnitude lower—a clear sign both pieces pull their weight.
Alex: Huh, so the kernels sharpen focus on tricky spots, and the search keeps equations lean. How does it stack up against close rivals, like that Weak-PDE-LEARN method?
Sam: They directly compared on Burgers, KdV, and KS equations with the same noisy datasets. Weak-PDE-Net holds lower parameter errors as noise climbs to 100%, and uniquely nails the full KS structure there while the other fails. This edge comes from the open search avoiding fixed libraries that might miss terms, plus better noise handling in the integrals. A notable limit is handling nested derivatives, like in the Porous Medium Equation where change over time equals the spatial shift of a product involving the field and its own change—like diffusion speed depending on density itself. The current setup can't build those layered derivative terms without tweaks, as the weak form shifts derivatives to test functions but leaves inner ones unresolved. They also point out the averaging smooths tiny-scale wiggles, which can blur sharp interfaces in some reactions.
Alex: Fair limits, then. Overall, a solid step for messy real-world physics like ocean flows.
Sam: Yes, the evidence shows it recovers exact terms consistently across those tests, even under heavy noise, pointing to practical use in updating weather or climate models in real time from IoT networks. That could mean engineering systems, like fluid flows in pipes or turbines, get self-correcting math without full simulations. The paper notes future steps like layering in more physics rules to handle complex operators, though that raises compute needs. Overall, it's a meaningful advance in letting data drive equation discovery end-to-end, without fixed assumptions.
Alex: That's a solid foundation. Thanks for walking through this, Sam—it's clearer now how this tackles real sensor challenges.
Sam: My pleasure, Alex. This work pushes toward more autonomous physics from the wild.
Alex: That's it for this look at Weak-PDE-Net. Thanks for listening.