Uniform error estimates for artificial neural network approximations for heat equations

DOI10.1093/IMANUM/DRAB027zbMATH Open1502.65171arXiv1911.09647OpenAlexW2991444716MaRDI QIDQ5093099FDOQ5093099

Authors: Lukas Gonon, Arnulf Jentzen, David Kofler, David Šiška, Philipp Grohs 

Publication date: 26 July 2022

Published in: IMA Journal of Numerical Analysis (Search for Journal in Brave)

Abstract: Recently, artificial neural networks (ANNs) in conjunction with stochastic gradient descent optimization methods have been employed to approximately compute solutions of possibly rather high-dimensional partial differential equations (PDEs). Very recently, there have also been a number of rigorous mathematical results in the scientific literature which examine the approximation capabilities of such deep learning based approximation algorithms for PDEs. These mathematical results from the scientific literature prove in part that algorithms based on ANNs are capable of overcoming the curse of dimensionality in the numerical approximation of high-dimensional PDEs. In these mathematical results from the scientific literature usually the error between the solution of the PDE and the approximating ANN is measured in the $L^p$-sense with respect to some $pin[1,infty)$ and some probability measure. In many applications it is, however, also important to control the error in a uniform $L^{i}nfty$-sense. The key contribution of the main result of this article is to develop the techniques to obtain error estimates between solutions of PDEs and approximating ANNs in the uniform $L^{i}nfty$-sense. In particular, we prove that the number of parameters of an ANN to uniformly approximate the classical solution of the heat equation in a region $[a,b{]}^d$ for a fixed time point $Tin(0,infty)$ grows at most polynomially in the dimension $dinmathbbN$ and the reciprocal of the approximation precision $varepsilon>0$. This shows that ANNs can overcome the curse of dimensionality in the numerical approximation of the heat equation when the error is measured in the uniform $L^{i}nfty$-norm.

Full work available at URL: https://arxiv.org/abs/1911.09647

Recommendations

• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms
• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs
• Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations

zbMATH Keywords

curse of dimensionalitydeep neural networkinformation based complexityhigh-dimensional PDEstractability of multivariate problems

Mathematics Subject Classification ID

Complexity and performance of numerical algorithms (65Y20) Artificial neural networks and deep learning (68T07) Probabilistic models, generic numerical methods in probability and statistics (65C20) Heat equation (35K05) Computational methods for stochastic equations (aspects of stochastic analysis) (60H35) Error bounds for initial value and initial-boundary value problems involving PDEs (65M15) PDEs in connection with classical thermodynamics and heat transfer (35Q79) Numerical methods for partial differential equations, initial value and time-dependent initial-boundary value problems (65M99)

Cited In (16)

• Deep ReLU network expression rates for option prices in high-dimensional, exponential Lévy models
• Deep learning methods for partial differential equations and related parameter identification problems
• Deep neural network expressivity for optimal stopping problems
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation
• Lower bounds for artificial neural network approximations: a proof that shallow neural networks fail to overcome the curse of dimensionality
• The modified MSA, a gradient flow and convergence
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
• Deep ReLU neural networks overcome the curse of dimensionality for partial integrodifferential equations
• Numerical methods for backward stochastic differential equations: a survey
• Solving Kolmogorov PDEs without the curse of dimensionality via deep learning and asymptotic expansion with Malliavin calculus
• Approximations with deep neural networks in Sobolev time-space
• Approximation properties of residual neural networks for fractional differential equations

This page was built for publication: Uniform error estimates for artificial neural network approximations for heat equations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5093099)