Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations

DOI10.1137/19M125649XzbMATH Open1480.60191arXiv1809.03062MaRDI QIDQ5037569FDOQ5037569

Authors: Julius Berner, Arnulf Jentzen, Philipp Grohs 

Publication date: 1 March 2022

Published in: SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Abstract: The development of new classification and regression algorithms based on empirical risk minimization (ERM) over deep neural network hypothesis classes, coined deep learning, revolutionized the area of artificial intelligence, machine learning, and data analysis. In particular, these methods have been applied to the numerical solution of high-dimensional partial differential equations with great success. Recent simulations indicate that deep learning-based algorithms are capable of overcoming the curse of dimensionality for the numerical solution of Kolmogorov equations, which are widely used in models from engineering, finance, and the natural sciences. The present paper considers under which conditions ERM over a deep neural network hypothesis class approximates the solution of a $d$-dimensional Kolmogorov equation with affine drift and diffusion coefficients and typical initial values arising from problems in computational finance up to error $varepsilon$. We establish that, with high probability over draws of training samples, such an approximation can be achieved with both the size of the hypothesis class and the number of training samples scaling only polynomially in $d$ and $varepsilo{n}^{-1}$. It can be concluded that ERM over deep neural network hypothesis classes overcomes the curse of dimensionality for the numerical solution of linear Kolmogorov equations with affine coefficients.

Full work available at URL: https://arxiv.org/abs/1809.03062

Recommendations

• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Solving the Kolmogorov PDE by means of deep learning
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms

zbMATH Keywords

deep learningcurse of dimensionalitygeneralization errorempirical risk minimizationKolmogorov equation

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Neural nets and related approaches to inference from stochastic processes (62M45) Applications of stochastic analysis (to PDEs, etc.) (60H30) Numerical solutions to stochastic differential and integral equations (65C30)

Cites Work

• DGM: a deep learning algorithm for solving partial differential equations
• Learning Theory
• On the mathematical foundations of learning
• Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
• A distribution-free theory of nonparametric regression
• Local Rademacher complexities
• Title not available (Why is that?)
• Title not available (Why is that?)
• Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
• Title not available (Why is that?)
• Infinite dimensional analysis. A hitchhiker's guide.
• Neural Network Learning
• Error bounds for approximation with neural networks
• Foundations of machine learning
• Tools for computational finance.
• Stochastic simulation and Monte Carlo methods. Mathematical foundations of stochastic simulation
• Loss of regularity for Kolmogorov equations
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Solving high-dimensional partial differential equations using deep learning
• Title not available (Why is that?)
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• Error bounds for approximations with deep ReLU networks
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Asymptotic expansion as prior knowledge in deep learning method for high dimensional BSDEs
• Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ
• Optimal approximation with sparsely connected deep neural networks
• Solving the Kolmogorov PDE by means of deep learning
• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Deep optimal stopping
• A mean field view of the landscape of two-layer neural networks
• Provable approximation properties for deep neural networks
• Topological properties of the set of functions generated by neural networks of fixed size
• Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
• Deep Neural Network Approximation Theory

Cited In (70)

• Deep ReLU network expression rates for option prices in high-dimensional, exponential Lévy models
• Non intrusive reduced order modeling of parametrized PDEs by kernel POD and neural networks
• Uniform error estimates for artificial neural network approximations for heat equations
• A measure theoretical approach to the mean-field maximum principle for training NeurODEs
• Stable recovery of entangled weights: towards robust identification of deep neural networks from minimal samples
• DNN expression rate analysis of high-dimensional PDEs: application to option pricing
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Overcoming the curse of dimensionality in the numerical approximation of parabolic partial differential equations with gradient-dependent nonlinearities
• Asymptotic expansion as prior knowledge in deep learning method for high dimensional BSDEs
• Three ways to solve partial differential equations with neural networks — A review
• Sparse polynomial approximations for affine parametric saddle point problems
• An overview on deep learning-based approximation methods for partial differential equations
• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Solving the Kolmogorov PDE by means of deep learning
• Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks
• Physics-informed neural networks for approximating dynamic (hyperbolic) PDEs of second order in time: error analysis and algorithms
• A new efficient approximation scheme for solving high-dimensional semilinear PDEs: control variate method for deep BSDE solver
• Metric entropy limits on recurrent neural network learning of linear dynamical systems
• Error estimates for deep learning methods in fluid dynamics
• Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms
• Enhancing the momentum strategy through deep regression
• Variational Monte Carlo -- bridging concepts of machine learning and high-dimensional partial differential equations
• Imaging conductivity from current density magnitude using neural networks
• Computing Lyapunov functions using deep neural networks
• Neural network approximation and estimation of classifiers with classification boundary in a Barron class
• Convergence of a Robust Deep FBSDE Method for Stochastic Control
• Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation
• Lower bounds for artificial neural network approximations: a proof that shallow neural networks fail to overcome the curse of dimensionality
• Quantitative Approximation Results for Complex-Valued Neural Networks
• Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes
• Deep neural network surrogates for nonsmooth quantities of interest in shape uncertainty quantification
• Convergence bounds for empirical nonlinear least-squares
• Physics and equality constrained artificial neural networks: application to forward and inverse problems with multi-fidelity data fusion
• Towards fast weak adversarial training to solve high dimensional parabolic partial differential equations using XNODE-WAN
• Convergence of Physics-Informed Neural Networks Applied to Linear Second-Order Elliptic Interface Problems
• Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
• Active learning based sampling for high-dimensional nonlinear partial differential equations
• A theoretical analysis of deep neural networks and parametric PDEs
• Space-time error estimates for deep neural network approximations for differential equations
• Optimally weighted loss functions for solving PDEs with neural networks
• Numerical methods for backward stochastic differential equations: a survey
• Deep Weak Approximation of SDEs: A Spatial Approximation Scheme for Solving Kolmogorov Equations
• Solving Kolmogorov PDEs without the curse of dimensionality via deep learning and asymptotic expansion with Malliavin calculus
• Approximations with deep neural networks in Sobolev time-space
• RICAM, the Johann Radon Institute for Computational and Applied Mathematics
• Simultaneous neural network approximation for smooth functions
• Adaptive neural network surrogate model for solving the implied volatility of time-dependent American option via Bayesian inference
• Greedy training algorithms for neural networks and applications to PDEs
• Exponential ReLU neural network approximation rates for point and edge singularities
• Deep splitting method for parabolic PDEs
• Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs
• Deep empirical risk minimization in finance: Looking into the future
• Non-linear interactions and exchange rate prediction: empirical evidence using support vector regression
• Unbiased deep solvers for linear parametric PDEs
• Full error analysis for the training of deep neural networks
• Deep neural network expressivity for optimal stopping problems
• Deep learning based on randomized quasi-Monte Carlo method for solving linear Kolmogorov partial differential equation
• Error analysis for deep neural network approximations of parametric hyperbolic conservation laws
• Proof of the theory-to-practice gap in deep learning via sampling complexity bounds for neural network approximation spaces
• Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning
• Generalization of PINNs for elliptic interface problems
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Strong overall error analysis for the training of artificial neural networks via random initializations
• Monte Carlo simulation of SDEs using GANs
• Parameter identification by deep learning of a material model for granular media
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• Sampling complexity of deep approximation spaces
• Solving PDEs on unknown manifolds with machine learning
• Error analysis for empirical risk minimization over clipped ReLU networks in solving linear Kolmogorov partial differential equations
• An extreme learning machine-based method for computational PDEs in higher dimensions

Uses Software

• MNIST
• Adam
• DGM

This page was built for publication: Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5037569)