A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations

DOI10.1007/S42985-019-0006-9zbMath1455.65200arXiv1901.10854OpenAlexW3023241366MaRDI QIDQ2216499

Martin Hutzenthaler, Thomas Kruse, Tuan Anh Nguyen, Arnulf Jentzen

Publication date: 16 December 2020

Published in: SN Partial Differential Equations and Applications (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1901.10854

zbMATH Keywords

curse of dimensionalityinformation based complexityhigh-dimensional PDEsdeep neural networksmultilevel Picard approximations

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Learning and adaptive systems in artificial intelligence (68T05) Probabilistic methods, stochastic differential equations (65C99) Probabilistic methods, particle methods, etc. for initial value and initial-boundary value problems involving PDEs (65M75)

Related Items (58)

Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black--Scholes Partial Differential Equations ⋮ Stationary Density Estimation of Itô Diffusions Using Deep Learning ⋮ Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space ⋮ Convergence of a Robust Deep FBSDE Method for Stochastic Control ⋮ A Proof that Artificial Neural Networks Overcome the Curse of Dimensionality in the Numerical Approximation of Black–Scholes Partial Differential Equations ⋮ SelectNet: self-paced learning for high-dimensional partial differential equations ⋮ MIM: a deep mixed residual method for solving high-order partial differential equations ⋮ Extensions of the deep Galerkin method ⋮ DeepSets and their derivative networks for solving symmetric PDEs ⋮ Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms ⋮ The deep parametric PDE method and applications to option pricing ⋮ Approximations with deep neural networks in Sobolev time-space ⋮ Full error analysis for the training of deep neural networks ⋮ Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data ⋮ Unbiased Deep Solvers for Linear Parametric PDEs ⋮ Deep Neural Network Surrogates for Nonsmooth Quantities of Interest in Shape Uncertainty Quantification ⋮ Deep ReLU neural network approximation in Bochner spaces and applications to parametric PDEs ⋮ DEEP EQUILIBRIUM NETS ⋮ Three ways to solve partial differential equations with neural networks — A review ⋮ Deep learning methods for partial differential equations and related parameter identification problems ⋮ Neural networks in Fréchet spaces ⋮ Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes ⋮ Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation ⋮ Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning ⋮ The Kolmogorov infinite dimensional equation in a Hilbert space via deep learning methods ⋮ Space-time error estimates for deep neural network approximations for differential equations ⋮ An extreme learning machine-based method for computational PDEs in higher dimensions ⋮ Lower bounds for artificial neural network approximations: a proof that shallow neural networks fail to overcome the curse of dimensionality ⋮ Numerical methods for backward stochastic differential equations: a survey ⋮ Adaptive Learning Rate Residual Network Based on Physics-Informed for Solving Partial Differential Equations ⋮ Solving Kolmogorov PDEs without the curse of dimensionality via deep learning and asymptotic expansion with Malliavin calculus ⋮ Approximation properties of residual neural networks for fractional differential equations ⋮ Mobility Estimation for Langevin Dynamics Using Control Variates ⋮ Pricing options on flow forwards by neural networks in a Hilbert space ⋮ The use of physics-informed neural network approach to image restoration via nonlinear PDE tools ⋮ Neural Control of Parametric Solutions for High-Dimensional Evolution PDEs ⋮ Designing universal causal deep learning models: The geometric (Hyper)transformer ⋮ Deep learning algorithms for solving high-dimensional nonlinear backward stochastic differential equations ⋮ Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks ⋮ An overview on deep learning-based approximation methods for partial differential equations ⋮ Convergence of the deep BSDE method for coupled FBSDEs ⋮ Deep Network Approximation Characterized by Number of Neurons ⋮ Numerical Simulations for Full History Recursive Multilevel Picard Approximations for Systems of High-Dimensional Partial Differential Equations ⋮ Overcoming the curse of dimensionality in the numerical approximation of Allen-Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations ⋮ Efficient approximation of solutions of parametric linear transport equations by ReLU DNNs ⋮ Deep Splitting Method for Parabolic PDEs ⋮ Deep backward schemes for high-dimensional nonlinear PDEs ⋮ Computing Lyapunov functions using deep neural networks ⋮ Numerical solution of the parametric diffusion equation by deep neural networks ⋮ Variational Monte Carlo -- bridging concepts of machine learning and high-dimensional partial differential equations ⋮ Multilevel Picard iterations for solving smooth semilinear parabolic heat equations ⋮ Tensor Decomposition Methods for High-dimensional Hamilton--Jacobi--Bellman Equations ⋮ Overcoming the curse of dimensionality for some Hamilton-Jacobi partial differential equations via neural network architectures ⋮ Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning ⋮ Data-driven soliton solutions and model parameters of nonlinear wave models via the conservation-law constrained neural network method ⋮ Deep ReLU neural networks overcome the curse of dimensionality for partial integrodifferential equations ⋮ DNN expression rate analysis of high-dimensional PDEs: application to option pricing ⋮ A theoretical analysis of deep neural networks and parametric PDEs

Uses Software

• DGM

Cites Work

• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• The Deep Ritz Method: a deep learning-based numerical algorithm for solving variational problems
• DGM: a deep learning algorithm for solving partial differential equations
• On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations
• Asymptotic expansion as prior knowledge in deep learning method for high dimensional BSDEs
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Solving high-dimensional partial differential equations using deep learning
• Deep optimal stopping

This page was built for publication: A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations