Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations

DOI10.1137/19M125649XMaRDI QIDQ5037569zbMATH OpenFDO

Authors Julius Berner, Arnulf Jentzen, Philipp Grohs

Publication date 1 March 2022

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1809.03062

zbMATH Keywords

curse of dimensionalitydeep learningempirical risk minimizationgeneralization errorKolmogorov equation

Mathematics Subject Classification ID

Applications of stochastic analysis (to PDEs, etc.) (60H30) Neural nets and related approaches to inference from stochastic processes (62M45) Numerical solutions to stochastic differential and integral equations (65C30) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: The development of new classification and regression algorithms based on empirical risk minimization (ERM) over deep neural network hypothesis classes, coined deep learning, revolutionized the area of artificial intelligence, machine learning, and data analysis. In particular, these methods have been applied to the numerical solution of high-dimensional partial differential equations with great success. Recent simulations indicate that deep learning-based algorithms are capable of overcoming the curse of dimensionality for the numerical solution of Kolmogorov equations, which are widely used in models from engineering, finance, and the natural sciences. The present paper considers under which conditions ERM over a deep neural network hypothesis class approximates the solution of a $d$-dimensional Kolmogorov equation with affine drift and diffusion coefficients and typical initial values arising from problems in computational finance up to error $varepsilon$. We establish that, with high probability over draws of training samples, such an approximation can be achieved with both the size of the hypothesis class and the number of training samples scaling only polynomially in $d$ and $varepsilo{n}^{-1}$. It can be concluded that ERM over deep neural network hypothesis classes overcomes the curse of dimensionality for the numerical solution of linear Kolmogorov equations with affine coefficients.

Recommendations

• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Solving the Kolmogorov PDE by means of deep learning
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms

Cites work

• A distribution-free theory of nonparametric regression
• A mean field view of the landscape of two-layer neural networks
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Asymptotic expansion as prior knowledge in deep learning method for high dimensional BSDEs
• Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
• Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Deep Neural Network Approximation Theory
• Deep optimal stopping
• DGM: a deep learning algorithm for solving partial differential equations
• Error bounds for approximation with neural networks
• Error bounds for approximations with deep ReLU networks
• Foundations of machine learning
• scientific article; zbMATH DE number 3438157 (Why is no real title available?)
• scientific article; zbMATH DE number 1420699 (Why is no real title available?)
• scientific article; zbMATH DE number 5681750 (Why is no real title available?)
• scientific article; zbMATH DE number 7064043 (Why is no real title available?)
• Infinite dimensional analysis. A hitchhiker's guide.
• Learning Theory
• Local Rademacher complexities
• Loss of regularity for Kolmogorov equations
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Neural Network Learning
• On the mathematical foundations of learning
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• Optimal approximation with sparsely connected deep neural networks
• Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Provable approximation properties for deep neural networks
• Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
• Solving high-dimensional partial differential equations using deep learning
• Solving the Kolmogorov PDE by means of deep learning
• Stochastic simulation and Monte Carlo methods. Mathematical foundations of stochastic simulation
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• Tools for computational finance.
• Topological properties of the set of functions generated by neural networks of fixed size

Cited in

(84)

• Computing Lyapunov functions using deep neural networks
• Solving the Kolmogorov PDE by means of deep learning
• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Optimally weighted loss functions for solving PDEs with neural networks
• RICAM, the Johann Radon Institute for Computational and Applied Mathematics
• Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs
• Stable recovery of entangled weights: towards robust identification of deep neural networks from minimal samples
• A measure theoretical approach to the mean-field maximum principle for training NeurODEs
• DNN expression rate analysis of high-dimensional PDEs: application to option pricing
• A theoretical analysis of deep neural networks and parametric PDEs
• A new efficient approximation scheme for solving high-dimensional semilinear PDEs: control variate method for deep BSDE solver
• Metric entropy limits on recurrent neural network learning of linear dynamical systems
• Error estimates for deep learning methods in fluid dynamics
• Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms
• Overcoming the curse of dimensionality in the numerical approximation of parabolic partial differential equations with gradient-dependent nonlinearities
• Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Deep ReLU network expression rates for option prices in high-dimensional, exponential Lévy models
• Variational Monte Carlo -- bridging concepts of machine learning and high-dimensional partial differential equations
• Asymptotic expansion as prior knowledge in deep learning method for high dimensional BSDEs
• Towards fast weak adversarial training to solve high dimensional parabolic partial differential equations using XNODE-WAN
• Physics and equality constrained artificial neural networks: application to forward and inverse problems with multi-fidelity data fusion
• Sparse polynomial approximations for affine parametric saddle point problems
• Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes
• Active learning based sampling for high-dimensional nonlinear partial differential equations
• Space-time error estimates for deep neural network approximations for differential equations
• Adaptive neural network surrogate model for solving the implied volatility of time-dependent American option via Bayesian inference
• An overview on deep learning-based approximation methods for partial differential equations
• Greedy training algorithms for neural networks and applications to PDEs
• Deep splitting method for parabolic PDEs
• Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
• Convergence bounds for empirical nonlinear least-squares
• Quantitative Approximation Results for Complex-Valued Neural Networks
• Approximations with deep neural networks in Sobolev time-space
• Imaging conductivity from current density magnitude using neural networks
• Full error analysis for the training of deep neural networks
• Uniform error estimates for artificial neural network approximations for heat equations
• Unbiased deep solvers for linear parametric PDEs
• Deep neural network surrogates for nonsmooth quantities of interest in shape uncertainty quantification
• Enhancing the momentum strategy through deep regression
• Non-linear interactions and exchange rate prediction: empirical evidence using support vector regression
• Convergence of a Robust Deep FBSDE Method for Stochastic Control
• Convergence of Physics-Informed Neural Networks Applied to Linear Second-Order Elliptic Interface Problems
• Simultaneous neural network approximation for smooth functions
• Deep empirical risk minimization in finance: Looking into the future
• Three ways to solve partial differential equations with neural networks — A review
• Monte Carlo simulation of SDEs using GANs
• Physics-informed neural networks for approximating dynamic (hyperbolic) PDEs of second order in time: error analysis and algorithms
• Exponential ReLU neural network approximation rates for point and edge singularities
• Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation
• An extreme learning machine-based method for computational PDEs in higher dimensions
• Lower bounds for artificial neural network approximations: a proof that shallow neural networks fail to overcome the curse of dimensionality
• Numerical methods for backward stochastic differential equations: a survey
• Neural network approximation and estimation of classifiers with classification boundary in a Barron class
• Deep Weak Approximation of SDEs: A Spatial Approximation Scheme for Solving Kolmogorov Equations
• Solving Kolmogorov PDEs without the curse of dimensionality via deep learning and asymptotic expansion with Malliavin calculus
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Solving PDEs on unknown manifolds with machine learning
• Deep neural network expressivity for optimal stopping problems
• Deep learning based on randomized quasi-Monte Carlo method for solving linear Kolmogorov partial differential equation
• Error analysis for deep neural network approximations of parametric hyperbolic conservation laws
• Proof of the theory-to-practice gap in deep learning via sampling complexity bounds for neural network approximation spaces
• Generalization of PINNs for elliptic interface problems
• Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning
• Strong overall error analysis for the training of artificial neural networks via random initializations
• Parameter identification by deep learning of a material model for granular media
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• Sampling complexity of deep approximation spaces
• Error analysis for empirical risk minimization over clipped ReLU networks in solving linear Kolmogorov partial differential equations
• Generalization error estimates of a machine learning method for solving high-dimensional Schrödinger eigenvalue problems
• Generalization error analysis of deep backward dynamic programming for solving nonlinear PDEs
• Bounds on the approximation error for deep neural networks applied to dispersive models: nonlinear waves
• Adaptive neural network basis methods for partial differential equations with low-regular solutions
• Interpolating between BSDEs and PINNs: deep learning for elliptic and parabolic boundary value problems
• Error analysis and numerical algorithm for PDE approximation with hidden-layer concatenated physics informed neural networks
• Towards continuous mathematical models for the analysis of classes of deep neural networks
• The neural network basis method for nonlinear partial differential equations and its Gauss-Newton optimizer
• Compressive Fourier collocation methods for high-dimensional diffusion equations with periodic boundary conditions
• Walk-on-Spheres Monte Carlo and deep neural network approximations of elliptic PDEs with drift and killing
• Curse of dimensionality in neural network optimization
• Global convergence of deep Galerkin and PINN methods for solving partial differential equations
• Stable learning using spiking neural networks equipped with affine encoders and decoders
• Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation
• Non intrusive reduced order modeling of parametrized PDEs by kernel POD and neural networks

Describes a project that uses

Uses Software

• MNIST
• Adam
• DGM

This page was built for publication: Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5037569)