Sobolev training for physics-informed neural networks

DOI10.4310/CMS.2023.V21.N6.A11MaRDI QIDQ6071750zbMATH OpenFDO

Authors Hwijae Son, Jin Woo Jang, Woo Jin Han, Hyung Ju Hwang

Publication date 28 November 2023

Published in Communications in Mathematical Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2101.08932

zbMATH Keywords

neural networkspartial differential equationsphysics-informed neural networksSobolev training

Mathematics Subject Classification ID

Fokker-Planck equations (35Q84) Numerical methods for partial differential equations, initial value and time-dependent initial-boundary value problems (65M99) Artificial neural networks and deep learning (68T07)

Abstract: Physics Informed Neural Networks (PINNs) is a promising application of deep learning. The smooth architecture of a fully connected neural network is appropriate for finding the solutions of PDEs; the corresponding loss function can also be intuitively designed and guarantees the convergence for various kinds of PDEs. However, the rate of convergence has been considered as a weakness of this approach. This paper proposes Sobolev-PINNs, a novel loss function for the training of PINNs, making the training substantially efficient. Inspired by the recent studies that incorporate derivative information for the training of neural networks, we develop a loss function that guides a neural network to reduce the error in the corresponding Sobolev space. Surprisingly, a simple modification of the loss function can make the training process similar to extit{Sobolev Training} although PINNs is not a fully supervised learning task. We provide several theoretical justifications that the proposed loss functions upper bound the error in the corresponding Sobolev spaces for the viscous Burgers equation and the kinetic Fokker--Planck equation. We also present several simulation results, which show that compared with the traditional $L^2$ loss function, the proposed loss function guides the neural network to a significantly faster convergence. Moreover, we provide the empirical evidence that shows that the proposed loss function, together with the iterative sampling techniques, performs better in solving high dimensional PDEs.

Recommendations

• On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
• Improved training of physics-informed neural networks for parabolic differential equations with sharply perturbed initial conditions
• Optimally weighted loss functions for solving PDEs with neural networks
• Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning

Cited in

(10)

• Physics-informed neural networks for the shallow-water equations on the sphere
• Meta-learning PINN loss functions
• Kolmogorov n-width and Lagrangian physics-informed neural networks: a causality-conforming manifold for convection-dominated PDEs
• Efficient physics-informed neural networks using hash encoding
• PINN training using biobjective optimization: the trade-off between data loss and residual loss
• Robust variational physics-informed neural networks
• A stabilized physics informed neural networks method for wave equations
• Sobolev training for operator learning
• Two-scale neural networks for partial differential equations with small parameters
• A conformal prediction framework for uncertainty quantification in physics-informed neural networks

This page was built for publication: Sobolev training for physics-informed neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6071750)