Three ways to solve partial differential equations with neural networks — A review

DOI10.1002/GAMM.202100006zbMATH Open1530.65137arXiv2102.11802OpenAlexW3171202467WikidataQ115406123 ScholiaQ115406123MaRDI QIDQ6068232FDOQ6068232

Authors: Jan Blechschmidt, Oliver G. Ernst 

Publication date: 15 December 2023

Published in: GAMM-Mitteilungen (Search for Journal in Brave)

Abstract: Neural networks are increasingly used to construct numerical solution methods for partial differential equations. In this expository review, we introduce and contrast three important recent approaches attractive in their simplicity and their suitability for high-dimensional problems: physics-informed neural networks, methods based on the Feynman-Kac formula and methods based on the solution of backward stochastic differential equations. The article is accompanied by a suite of expository software in the form of Jupyter notebooks in which each basic methodology is explained step by step, allowing for a quick assimilation and experimentation. An extensive bibliography summarizes the state of the art.

Full work available at URL: https://arxiv.org/abs/2102.11802

Recommendations

• DGM: a deep learning algorithm for solving partial differential equations
• Deep neural networks based temporal-difference methods for high-dimensional parabolic partial differential equations
• Adaptive deep neural networks methods for high-dimensional partial differential equations
• Convolution-Based Model-Solving Method for Three-Dimensional, Unsteady, Partial Differential Equations
• Neural networks-based backward scheme for fully nonlinear PDEs

zbMATH Keywords

curse of dimensionalityHamilton-Jacobi-Bellman equationsneural networkspartial differential equationstochastic processFeynman-Kacbackward differential equationPINN

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Research exposition (monographs, survey articles) pertaining to numerical analysis (65-02) Computational learning theory (68Q32) Second-order parabolic equations (35K10) PDEs with randomness, stochastic partial differential equations (35R60) Multistep, Runge-Kutta and extrapolation methods for ordinary differential equations (65L06) Optimal stochastic control (93E20) Numerical methods for partial differential equations, initial value and time-dependent initial-boundary value problems (65M99)

Cites Work

• Deep ReLU networks and high-order finite element methods
• Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• New error bounds for deep ReLU networks using sparse grids
• Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions
• Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks
• Multi-scale deep neural network (MscaleDNN) for solving Poisson-Boltzmann equation in complex domains
• A first-order numerical scheme for forward-backward stochastic differential equations in bounded domains
• A multi-scale DNN algorithm for nonlinear elliptic equations with multiple scales
• Multi-Scale Deep Neural Network (MscaleDNN) Methods for Oscillatory Stokes Flows in Complex Domains
• A dual algorithm for stochastic control problems: applications to uncertain volatility models and CVA
• Convergence analysis of a finite element method for second order non-variational elliptic problems
• Julia: a fresh approach to numerical computing
• Adaptive subgradient methods for online learning and stochastic optimization
• DGM: a deep learning algorithm for solving partial differential equations
• fPINNs: Fractional Physics-Informed Neural Networks
• Learning deep architectures for AI
• On the limited memory BFGS method for large scale optimization
• Title not available (Why is that?)
• Large Sample Properties of Simulations Using Latin Hypercube Sampling
• Deep learning
• Understanding machine learning. From theory to algorithms
• Elliptic partial differential equations of second order
• Continuous-time stochastic control and optimization with financial applications
• Higher-order implicit strong numerical schemes for stochastic differential equations
• Forward-backward stochastic differential equations and quasilinear parabolic PDEs
• Backwards SDE with random terminal time and applications to semilinear elliptic PDE
• Solving Ordinary Differential Equations I
• Multilevel Monte Carlo Path Simulation
• Second-order backward stochastic differential equations and fully nonlinear parabolic PDEs
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Backward Stochastic Differential Equations in Finance
• Linear multistep schemes for BSDEs
• The Mathematical Theory of Finite Element Methods
• Discontinuous Galerkin finite element approximation of Hamilton-Jacobi-Bellman equations with Cordes coefficients
• Adapted solution of a backward stochastic differential equation
• Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations
• Numerical simulation of quadratic BSDEs
• Title not available (Why is that?)
• Discretization and simulation of stochastic differential equations
• Spectral and finite difference solutions of the Burgers equations
• Adaptive importance sampling in least-squares Monte Carlo algorithms for backward stochastic differential equations
• Approximation by superpositions of a sigmoidal function
• A numerical algorithm for a class of BSDEs via the branching process
• The Diffuse Interface Approach in Materials Science
• Counterparty credit risk, collateral and funding. With pricing cases for all asset classes
• Title not available (Why is that?)
• Numerical solution for high order differential equations using a hybrid neural network-optimization method
• Reinforcement learning. An introduction
• On Distributions of Certain Wiener Functionals
• The numerical solution of linear ordinary differential equations by feedforward neural networks
• Stochastic integral
• Algorithms for overcoming the curse of dimensionality for certain Hamilton-Jacobi equations arising in control theory and elsewhere
• A forward scheme for backward SDEs
• What you should know about approximate dynamic programming
• A Survey of Some Applications of Probability and Stochastic Control Theory to Finite Difference Methods for Degenerate Elliptic and Parabolic Equations
• Bilateral counterparty risk under funding constraints. I: Pricing
• Continuous Markov processes and stochastic equations
• Neural algorithm for solving differential equations
• Weak approximation of a diffusion process in a bounded domain
• Stochastic ordinary differential equations in applied and computational mathematics
• Two numerical methods for the elliptic Monge-Ampère equation
• Advanced Lectures on Machine Learning
• Stochastic simulation and Monte Carlo methods. Mathematical foundations of stochastic simulation
• Mixed finite element methods for the fully nonlinear Monge-Ampère equation based on the vanishing moment method
• On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations
• Branching diffusion representation of semilinear PDEs and Monte Carlo approximation
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Solving high-dimensional partial differential equations using deep learning
• A primal-dual algorithm for BSDEs
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Finite element approximations of the three dimensional Monge-Ampère equation
• Hidden physics models: machine learning of nonlinear partial differential equations
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• Neural-net-induced Gaussian process regression for function approximation and PDE solution
• Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data
• Error bounds for approximations with deep ReLU networks
• Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Deep learning observables in computational fluid dynamics
• Constraint-aware neural networks for Riemann problems
• Machine learning of linear differential equations using Gaussian processes
• Constructive deep ReLU neural network approximation
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Asymptotic expansion as prior knowledge in deep learning method for high dimensional BSDEs
• Numerical Gaussian processes for time-dependent and nonlinear partial differential equations
• Deep hidden physics models: deep learning of nonlinear partial differential equations
• Optimal approximation with sparsely connected deep neural networks
• Physics-informed generative adversarial networks for stochastic differential equations
• Combining machine learning and domain decomposition methods for the solution of partial differential equations—A review
• Overcoming the curse of dimensionality for some Hamilton-Jacobi partial differential equations via neural network architectures
• Overcoming the curse of dimensionality in the numerical approximation of Allen-Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations
• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton-Jacobi partial differential equations
• Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks
• Monte-Carlo methods and stochastic processes. From linear to non-linear
• Title not available (Why is that?)
• Optimization methods for large-scale machine learning
• Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis
• Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations
• Finite difference methods for the weak solutions of the Kolmogorov equation for the density of both diffusion and conditional diffusion processes
• A phase shift deep neural network for high frequency approximation and wave problems
• Deep vs. shallow networks: an approximation theory perspective
• Physics-informed neural networks for high-speed flows
• A deep energy method for finite deformation hyperelasticity
• The neural particle method - an updated Lagrangian physics informed neural network for computational fluid dynamics
• Efficient approximation of solutions of parametric linear transport equations by ReLU DNNs
• Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems
• A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems

Cited In (13)

• Solving the discretised neutron diffusion equations using neural networks
• Semi-analytic PINN methods for boundary layer problems in a rectangular domain
• An introduction to kernel and operator learning methods for homogenization by self-consistent clustering analysis
• Cubic and quartic hyperbolic B-splines comparison for coupled Navier Stokes equation via differential quadrature method -- a statistical aspect
• Deep neural networks for probability of default modelling
• Solving American option optimal control problems in financial markets using a novel neural network
• JAX-DIPS: neural bootstrapping of finite discretization methods and application to elliptic problems with discontinuities
• On quadrature rules for solving partial differential equations using neural networks
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Physics-informed ConvNet: learning physical field from a shallow neural network
• Deep learning approximations for non-local nonlinear PDEs with Neumann boundary conditions
• A decoupled physics-informed neural network for recovering a space-dependent force function in the wave equation from integral overdetermination data
• Time integration schemes based on neural networks for solving partial differential equations on coarse grids

This page was built for publication: Three ways to solve partial differential equations with neural networks — A review

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6068232)